The Compressive Knowledge Graph Hypothesis: Which Graph Facts Matter for Scientific Hypothesis Generation?

Unknown

Conceptio › Archive › arXiv CS

arXiv CSopen access

The Compressive Knowledge Graph Hypothesis: Which Graph Facts Matter for Scientific Hypothesis Generation?

Unknown · 2026 · arxiv_cs

arXiv CS · Papers · License: Open Access · 2026

Open Source ↗Direct PDF ↓

artificialintelligence knowledgerepresentation reasoning

artificial intelligence, reasoning, knowledge representation

The Compressive Knowledge Graph Hypothesis: Which Graph Facts Matter for Scientific Hypothesis Generation? Shashwat Sourav1,2,3,4 , Viktoriia Baibakova5 , Sanjay Das2 , Ran Elgedawy2 , Maria Mahbub2 , Emily Herron2 , Tirthankar Ghosal2 1

arXiv:2605.27176v1 [cs.AI] 26 May 2026

Washington University in St. Louis 2 Oak Ridge National Laboratory 3 Lawrence Berkeley National Laboratory 4 UniverseTBD 5 Lila Sciences [email protected] Abstract

structure the model actually uses (Liu et al., 2023; Hagström et al., 2024). A model may benefit from a few salient entities, from the relation structure itself, or from only a small subset of the graph while ignoring the rest. If we do not separate these possibilities, it is difficult to know what role external knowledge graphs are really playing in hypothesis generation. Current work often treats knowledge graph prompting as a single intervention: provide the graph, then measure whether performance goes up or down (Pan et al., 2023; Wen et al., 2023). That view is too coarse for scientific discovery settings. A graph can vary in several ways at once. It can be dense or sparse, coarse or semantically rich, shallow or multi-hop (Jin et al., 2024; Mavromatis and Karypis, 2024). It can also be partly corrupted, shuffled, or compressed into a targeted subgraph (Li et al., 2023). When model behavior changes under these conditions, the main question is whether the graph helps, and which part of the graph helps, how that changes across models, and what kind of information remains useful as model capability increases (Zhang et al., 2022; Yasunaga et al., 2022). In this work, we study these questions in a battery-science hypothesis-generation setting. We compare three models with different capability levels: Mistral-7B, Llama-3.1-70B, and Gemini 2.5 Flash (Jiang et al., 2023; Grattafiori et al., 2024; Comanici et al., 2025). For each scientific problem, we generate hypotheses under a family of KG conditions that vary density, ontology richness, topology, and control structure (Jin et al., 2024; Mavromatis and Karypis, 2024; Ma et al., 2024). We then evaluate the generated outputs with metrics that separate entity use from relation use, and we add intervention-based analyses that test what

Knowledge graphs (KGs) can provide structured scientific context to language models, but it remains unclear which graph facts actually shape the generated hypotheses. We study KGguided hypothesis generation for battery materials across Mistral-7B, Llama-3.1-70B, and Gemini 2.5 Flash. We perturb local KGs by varying density, ontology richness, topology, and control structure, and evaluate outputs with both provided-graph and fixed-reference metrics. Across models, KG utility is selective and model-dependent: graph context changes outputs, but no-KG outputs also recover substantial graph content from model priors. Compact top-k subgraphs often approximate fullKG behavior, including when claimed-outcome triples are held out. At the same time, compression is not unique to one semantic ranking rule, random and topology-based subsets can also recover much of the signal. These results support a redundancy-aware Compressive KG hypothesis: useful KG signal is often recoverable from compact, scientifically structured subgraphs rather than requiring the full local graph.

Introduction

Knowledge graphs are increasingly used as a way to give language models structured context (Pan et al., 2023; Ando and Zhang, 2005). In scientific settings, this is especially appealing (Xiong et al., 2024; Kulkarni et al., 2025). A graph can organize a problem into explicit concepts such as the material system, the failure mode, the proposed intervention, the mechanism, and the target property. In principle, this should help a model move from a vague answer to a more grounded scientific hypothesis (Baek et al., 2024). In practice, however, it is still unclear how much of that graph 1

• We provide evidence for a redundancy-aware Compressive Knowledge Graph Hypothesis: compact subgraphs often recover much of the full-KG behavior, including under an outcome-held-out control, but compression is not unique to a single semantic ranking rule.

information is necessary, sufficient, and causally important (Chen et al., 2024). This lets us move beyond the usual question of whether KG prompting works, and toward a more precise account of how models use external structured knowledge. Our main claim is what we call the Compressive Knowledge Graph Hypothesis. We propose that full-KG behavior is often recoverable from compact subgraphs rather than requiring the entire local graph. This is a redundancy-aware claim, as it does not require one ranking rule to uniquely identify the important triples. Instead, useful signal may be distributed across mechanism, intervention, failuremode, and outcome-facing relations, and different compact subsets may preserve enough structure to recover similar hypothesis behavior (Zhang et al., 2025; Long et al., 2025). Across analyses, we find that KG influence is real but model-dependent. Problem identity remains the dominant source of variation, while KG condition has a smaller but measurable effect. Targeted and top-k subgraphs often recover much of the full-KG behavior; fixed-reference scoring shows that this is not merely an artifact of no-KG zeros; and an outcome-held-out control shows that compression is not only claimed-outcome leakage (Li et al., 2024; Linders and Tomczak, 2025). These findings matter for how knowledge graphs should be used in hypotheses generation. If the useful signal is concentrated in a compact subset, then supplying larger and denser graphs may not be the right design choice, especially for stronger models. A better approach may be to identify the small set of graph facts that actually steers generation. This has consequences not only for knowledge-graph prompting, but also for how we think about retrieval, structured context design, and evaluation in scientific language-model systems (Gurrapu et al., 2023; Wang et al., 2025). Our contributions are as follows:

Related Work

Prior work has studied many ways of combining knowledge graphs with language models, including using KGs as external memory, structured prompts, reasoning scaffolds, or sources of factual grounding. Early knowledge-enhanced language-model work incorporated KG entities, triples, or verbalized KG facts into pretraining and representation learning (Zhang et al., 2019; Peters et al., 2019; Agarwal et al., 2020). Surveys on knowledgeenhanced language models and LLM–KG integration argue that KGs can improve factuality, interpretability, and structured reasoning, while LLMs can help construct, complete, and verbalize KGs (Hu et al., 2022; Pan et al., 2023). Recent GraphRAG work extends this idea by using graph-based indexing, graph-guided retrieval, and graph-enhanced generation to provide relational context for downstream tasks (Peng et al., 2024; Han et al., 2024). However, in our work, we are trying to understand which graph facts are actually used during scientific hypothesis generation. Previous works have also developed methods that use KGs to guide hypothesis generation. KGCoI integrates external structured knowledge into a chain-of-ideas process and uses KG support to reduce hallucinations in hypotheses generation (Xiong et al., 2024). Related systems use scientific KGs for link prediction, literature-based discovery, or candidate hypothesis ranking (Spangler et al., 2014; Pu et al., 2023; Borrego et al., 2025; Kastrin et al., 2025). These approaches generally treat the KG as useful context or evidence. By contrast, we test the compressive view of KG utility: whether a small subset of high-value triples is sufficient to recover much of the full-KG behavior, whether removing that subset disrupts generation, and whether triple importance is governed by semantic role rather than graph topology alone.

• We study knowledge-graph-guided hypothesis generation across three models and multiple graph manipulations, including density, ontology, topology, random, shuffled, targeted, and compressed graph conditions. • We introduce an evaluation framework that separates provided-graph use from fixedreference recovery, allowing no-KG, top-k, random, shuffled, and full-KG outputs to be scored against the same full graph.

Task and Experimental Setup

We study KG-guided hypothesis generation in battery science (Chen et al., 2026). Each example contains a scientific problem statement and structured 2

Figure 1: Overview of the KG-guided generation pipeline. Battery-science fields are converted into a directed KG, verbalized as triples, and provided to language models under different graph conditions. Outputs are evaluated for entity recall, relation fidelity, graph coverage, and semantic distance.

fields such as material system, component, failure mode, intervention, mechanism, target property, and claimed outcome. In total we used 100 problems.1 From these fields, we construct a directed KG whose typed triples connect the problem to relevant scientific concepts. The model is asked to generate a solution hypothesis either without graph context or with a verbalized set of subject-relationobject triples, a common strategy for injecting KG facts into language-model prompts (Liu et al., 2019; Agarwal et al., 2020; Baek et al., 2024). All KG variants are derived from the same local graph Gp = (Vp , Ep ), with 15-18 typed triples per problem. Density variants vary subset size; ontology variants vary relation granularity; topology variants range from 2-hop context to full problem-to-outcome paths. Random, shuffled, and targeted/top-k controls respectively test irrelevant triples, broken relations, and compact relevanceranked subsets. Full definitions and size/context statistics are given in Appendices B.6 and G. Battery materials are a useful testbed because the problems are inherently relational and mechanismdriven. A good hypothesis must connect what material or component is involved, why it fails, how an intervention changes the mechanism, and which property or outcome should improve. In material science, the hypothesis must preserve links among

degradation, ion transport, interfacial stability, capacity retention, and materials design. These recurring structures are central to battery research and can be represented as KG triples, making the domain well suited for testing whether models use graph relations rather than only surface entities (Kumbhar et al., 2025). We compare Mistral-7B, Llama-3.1-70B, and Gemini 2.5 Flash in the main cross-family study, and run additional intra-family checks on Mistral7B/12B/22B and Llama-3.1-8B/70B. For each problem, we keep the prompt fixed and vary only the graph condition. The KG manipulations span three axes (2): density (sparse, medium, dense), ontology richness (coarse T1 versus richer T3 multihop relations), and topology (2-hop versus fullpath context). We also include control conditions: no KG, random KG, shuffled KG, targeted KG, and top-k compressed subgraphs. Detailed definitions of the random, shuffled, entity-only, and relationskeleton controls are given in Appendix B.3. We test three claims. First, real KG structure should affect outputs more than irrelevant or corrupted graph context. Second, if KG utility is compressed, then a small top-k subset should recover much of the full-KG behavior, while removing that subset should degrade the output. Third, graph influence should not be fully explained by simple topology alone: relation type, task relevance, and redundancy among local triples may all shape

1 The 100-problem evaluation dataset is available at https://huggingface.co/datasets/matter2mech/ battery-science-problems.

which compact subsets recover full-KG behavior.

entities made available by the graph. However, TRR alone does not show whether the model used the graph structure correctly, since an output can mention the right entities while ignoring or distorting their relations.

Evaluation Metrics

We design the evaluation to measure whether a knowledge graph changes the generated hypothesis, how it changes it. This distinction is important because a model may copy graph entities without using their relations, preserve relation language without recalling the correct objects, or generate a hypothesis that is semantically close to the full-KG output while covering only a small fraction of the graph. We therefore use a set of output-based metrics that separate entity recall, relation use, graph coverage, and semantic sensitivity. We distinguish two scoring views. Provided-graph metrics score an output against the graph supplied in that condition and measure whether the model used the given graph. These metrics are undefined for no-KG settings and can have different denominators for top-k and full-KG conditions. We therefore also report fixed-reference metrics, which score every output against the same full KG for that problem. Fixedreference scoring makes no-KG, random, shuffled, top-k, and full-KG outputs directly comparable and avoids artificial zero scores for no-KG baselines.

Relation Fidelity Score. Relation Fidelity Score (RFS) measures whether the generated hypothesis preserves the semantic role of the KG relations. Each graph relation is mapped to a scientific entity such as failure mode, mechanism, intervention, material component, property, or outcome. RFS then measures whether the language of the generated hypothesis expresses the same relation type. This allows us to distinguish shallow entity copying from relation-aware graph use. For example, mentioning an electrolyte additive contributes to TRR, but it contributes to RFS only if the output uses it in a role consistent with the supplied graph relation, such as an intervention that stabilizes an interface or improves ionic transport. KG Triple Coverage. KG Triple Coverage (KTC) measures broader coverage of the graph content. While TRR focuses on object entities, KTC measures the fraction of supplied triples whose object-side content is represented in the generated hypothesis. This metric is useful for comparing full-KG, no-KG, random-KG, and compressed-KG conditions because it directly measures how much of the graph context is reflected in the output. A high KTC score indicates that the model uses a larger portion of the graph, whereas a low KTC score indicates that the model either ignores the graph or uses only a small subset of it.

Human expert evaluation. Automatic graph-use metrics can show whether an output reflects KG content, but they do not fully measure whether the hypothesis is scientifically useful. We therefore add a blinded domain-rater assessment. A materials-science postdoctoral researcher rated five representative examples comparing No KG, Top8 KG, and Full KG outputs. The outputs were anonymized and condition labels were hidden during rating. The rater scored each hypothesis on a 1-5 scale for problem alignment, mechanistic specificity, intervention specificity, scientific plausibility, and evidence faithfulness, and also gave pairwise preferences against the No-KG output. The full rating instructions are provided in Appendix A.1.

Fixed-reference graph recovery. Providedgraph metrics answer whether a model used the graph it received. To compare conditions with different graph sizes, we also compute fixed-reference variants. Let Gpfull be the full KG for problem p. For any condition c, including no KG, random KG, shuffled KG, and top-k, we compute

Triple Recall Rate. Triple Recall Rate (TRR) measures how much of the provided KG content appears in the final hypothesis at the objectentity level. For a graph condition with triples G = {(si , ri , oi )}ni=1 and generated hypothesis y, we define 1 X TRR(y, G) = 1[o ∈ y], |OG |

Analogous fixed-reference versions are computed for relation fidelity and graph coverage. These metrics ask how much of the full problem graph is recovered in the output, regardless of which graph was provided to the model.

where OG is the set of object entities in the supplied graph. TRR captures whether the model recalls the

Semantic distance to full-KG behavior. For sufficiency and comprehensiveness experiments, we

TRRref (yp,c ) = TRR(yp,c , Gpfull ).

o∈OG

Figure 2: KG perturbation design. We vary the external knowledge graph along three axes: density, ontology richness, and topology. Density controls how many graph facts are supplied; ontology richness controls whether relations are coarse or semantically detailed; topology controls whether the model receives local 2-hop context or longer full-path structure.

where p indexes problems and c, c′ index KG conditions. Smaller values of ρ indicate that problem identity dominates over graph condition, while larger values indicate stronger sensitivity to graph context.

also measure semantic distance between an ablated output and the corresponding full-KG output. Let yfull be the hypothesis generated with the full KG and yc be the hypothesis generated under condition c, such as top-k triples or full KG with top-k triples removed. We compute

Statistical testing. We use paired permutation tests for the main condition contrasts because each problem is evaluated under multiple KG conditions. We also report bootstrap confidence intervals for key deltas, including ∆TRR(real-random), ∆RFS(real-shuffled), and fixed-reference recovery differences. Because we test multiple modelmetric-condition contrasts, we report both uncorrected and Holm/BH-corrected p-values in Appendix C. We show numerical zeros as p < 0.0001 and near-one values as p > 0.999. The main text emphasizes effect sizes, confidence intervals, and consistent directional patterns rather than isolated significance thresholds. Implementation details for RFS, KTC, and the deterministic top-k triple-ranking rule are provided in Appendix B, with additional details in Appendices B.1 and B.2. We compute semantic distance with a Sentence-Transformers encoder and verify (Appendix H) that the sufficiency trend is robust to replacing MiniLM-L6 with MPNet-base. The two encoders give highly correlated distances (Spear-

dsem (yc , yfull ) = 1 − cos (e(yc ), e(yfull )) , where e(·) is a sentence embedding function. Lower semantic distance means that the ablated condition better recovers the behavior induced by the full graph. This is our main metric for testing whether a small subset of triples is sufficient to approximate full-KG behavior. Additional metric implementation details are provided in Appendix B. Intra-problem versus inter-problem variation. To measure how strongly graph condition affects generation relative to the problem itself, we compare two sources of semantic variation. Intraproblem variation measures how much outputs change for the same scientific problem when the KG condition changes. Inter-problem variation measures how much outputs differ across different scientific problems. We summarize this using the variance ratio Ep d(yp,c , yp,c′ ) , ρ= Ep̸=p′ d(yp,c , yp′ ,c′ ) 5

man ρ = 0.965) and preserve the same monotonic decrease from k = 1 to k = 8.

decomposition shows that model identity explains more variance than KG condition across TRR, RFS, and KTC; we report the full table in Appendix C. These results show that external KG utility does not vanish across models, but it becomes more selective. Stronger models do not benefit uniformly from larger or richer graphs. Instead, they appear to benefit most from smaller, higher-signal graph contexts.

Results

Our results support the redundancy-aware Compressive Knowledge Graph Hypothesis. Across all our experiments, we observe three main patterns. First, KG utility is model-dependent. Second, compact subgraphs often approximate full-KG behavior, including when claimed-outcome triples are held out. Third, graph influence is not explained by a single selector: relation role matters, but random and topology-based subsets can also recover much of the signal when enough triples are retained. 5.1

Materials-science prompting guidance. The best graph condition differs by model, but the top examples share a common materials-science structure: the graph preserves a chain from failure mode to intervention, mechanism, target property, and outcome. Gemini 2.5 Flash performs best with sparse, semantically rich full-path graphs, suggesting that it can use compact mechanism-to-outcome chains. Llama-3.1-70B performs best with sparse coarse 2-hop graphs, suggesting that concise local grounding is sufficient. Mistral-7B benefits more from dense rich 2-hop graphs, suggesting that smaller open models may need more explicit local scaffolding. Representative top graph examples for each model are shown in Appendix A.3.

Cross-family KG utility is selective

Table 1 summarizes the main cross-family results. Gemini shows the strongest response to real KG structure, with ∆TRR(real–random) = 0.290 and ∆KTC(real–noKG) = 0.757. Llama-3.1-70B shows a smaller but positive effect, especially in graph coverage. Mistral-7B shows weak and brittle graph use, with almost no gain over random KG context and negative ∆RFS under the real-versusshuffled comparison. The preferred graph structure also differs by model. Gemini 2.5 Flash benefits most from sparse, semantically rich, full-path graph context. Llama3.1-70B benefits most from sparse, coarse, shorterrange context. Mistral-7B prefers denser scaffolding. This supports the selective-utility view: stronger models do not necessarily need more graph context, but they can benefit from a smaller set of high-signal facts. Full permutation tests and bootstrap confidence intervals are reported in Appendix C. We observe this in both the control comparisons and the structural ablations. In Table 9, Gemini 2.5 Flash shows large and statistically stable gains when the real KG is compared against a random KG, especially for TRR and KTC. It also shows a strong gain in RFS when the real KG is compared against a shuffled KG, indicating that relation structure is being used in a meaningful way. Llama-3.1-70B shows a smaller but still positive pattern, with the clearest gain appearing in KTC under the real-versus-random comparison. By contrast, Mistral-7B shows little benefit relative to the random KG and is strongly degraded by shuffled structure, suggesting that its graph use is weak and brittle rather than robust. A mixed-effects variance

5.2

Compressed subgraphs recover full-KG behavior

We next test whether the full graph is necessary. The necessity analysis compares no KG, entityonly context, relation skeletons, targeted KG, and full KG; full results are reported in Appendix B.4. Entity-only context does not recover the full-KG effect, showing that graph utility is not just lexical exposure to scientific terms. Relation skeletons preserve some relation-level signal but lose entityspecific grounding. Targeted KG recovers much of the full-KG behavior, indicating that useful graph signal can be preserved by compact subsets rather than requiring the entire graph. Appendix D.1 further shows that this compression effect is not unique to our semantic ranking heuristic: random and topology-based top-k selectors also approach full-KG behavior as k increases, although no single selector dominates across all models and k values. The top-k sufficiency and comprehensiveness analyses test this directly. In the sufficiency setting, we keep only the top-k ranked triples and measure distance to the full-KG output. In the comprehensiveness setting, we remove the same top-k triples and measure degradation. Figure 3 shows the key pattern: as k increases, top-k triples become in6

Table 1: Model-level summary of KG utility. ∆TRR(real−random) measures sensitivity to replacing the real KG with a random KG; ∆RFS(real−shuffled) measures sensitivity to shuffling KG structure; ∆KTC(real−noKG) measures the gain from using the real KG over no KG context. The variance ratio is intra-problem semantic variation across KG conditions divided by inter-problem variation across scientific problems; lower values indicate that problem identity dominates more strongly over KG condition. Model

∆TRR

∆RFS

∆KTC

Variance ratio

Best density

Best ontology

Best topology

Gemini Llama-3.1-70B Mistral-7B

0.2900 0.0380 0.0080

0.1867 0.0100 -0.0800

0.7569 0.0621 0.0054

0.3125 0.3838 0.5148

sparse sparse dense

T3_multihop T1_coarse T3_multihop

full_path 2hop 2hop

Table 2: Outcome-held-out control. Removing claimed-outcome triples reduces relation fidelity but does not eliminate graph signal. Compact top-8 subgraphs without outcome triples preserve substantial relation and mechanism/intervention signal.

creasingly sufficient to recover full-KG behavior, while removing them causes larger disruption. This is the central empirical signature of the Compressive Knowledge Graph Hypothesis. Because our top-k ranking uses lexical overlap with the problem statement, we also test whether compression is merely an artifact of this heuristic. We compare semantic top-k subsets against matched random-k and topology-based subsets selected by degree, betweenness, and PageRank. The compression trend remains across ranking methods, so our main claim is not that the lexical ranking is optimal, but that full-KG behavior can be approximated by compact subsets of triples. Full results are reported in Appendix D.1. The expert scores in Table 3 support the compression pattern. No-KG outputs are often plausible, but they are less mechanistically specific and less faithful to the supplied evidence. Top-8 KG nearly matches Full KG on mechanistic specificity and evidence faithfulness, while keeping scientific plausibility high. Because the rater saw anonymized outputs without condition labels, this provides a small sanity check that compact graph context recovers useful scientific grounding rather than merely increasing surface overlap. Full rating instructions are given in Appendix A.1. On average, each per-problem KG contains 16.1 triples (median 16, range 15-18), so the top-8 subset corresponds to only 49.7% of the full local graph and top-4 to 24.8%. Full KG-size and context-length statistics are reported in Appendix G. This shows that the observed recovery is not due to using nearly the whole graph, but to a compact subset of graph facts.

Model

Condition

RFS

Mech./Int.

Gemini

Full KG 0.580 Full KG – out. 0.439 Top-8 – out. 0.458

0.075 0.075 0.073

Full KG 0.677 Llama-70B Full KG – out. 0.511 Top-8 – out. 0.550

0.107 0.093 0.110

Table 3: Human Expert Evaluation A materialsscience postdoc blindly scored five representative examples on a 1–5 scale. Top-8 KG closely tracks Full KG on mechanism and evidence grounding, while both improve over No KG. Criterion

No KG Top-8 KG Full KG

Problem alignment Mechanistic specificity Intervention specificity Scientific plausibility Evidence faithfulness

3.2 2.4 2.6 3.8 2.0

4.0 3.8 3.6 3.8 3.8

4.2 4.0 3.8 4.0 4.2

Pairwise vs No KG Top-8 close to Full

– –

4/5 4/5

5/5 –

lation fidelity, as expected, but does not eliminate graph signal. For Gemini, full-KG RFS drops from 0.580 to 0.439, while top-8 without outcome triples still reaches 0.458. For Llama-3.1-70B, full-KG RFS drops from 0.677 to 0.511, while top-8 without outcome triples reaches 0.550. Mechanism/intervention coverage is also preserved under top8 no-outcome conditions. Thus, outcome-facing triples are high-leverage, but the compression effect is not only outcome leakage.

Outcome-held-out control. Because the local KG includes claimed-outcome triples, we test whether compression is merely an artifact of exposing part of the target hypothesis. We remove all outcome-facing triples and rerun full-KG and top-k conditions for Gemini and Llama-3.1-70B. Table 2 shows that removing outcome triples reduces re-

5.3

Topology alone does not explain graph influence

Compression alone does not identify which triples matter. We therefore use contradiction, knockout, 7

Figure 3: Sufficiency and comprehensiveness support KG compression. Keeping only the top-k ranked triples increasingly recovers the full-KG output, while removing the same triples causes systematic degradation. Compact subsets are often sufficient to approximate full-KG behavior, while removing high-ranked subsets causes systematic degradation.

5.4

Intra-family scaling shows compression without a simple size law

The cross-family comparison could reflect differences in training data, alignment, or model family rather than scale alone. We therefore perform intrafamily experiments for Mistral and Llama. In the Mistral family, semantic distance to the full-KG output decreases as k increases for 7B, 12B, and 22B models, confirming that compressed subsets recover full-KG behavior. However, the trend is not monotonic in parameter count. The 22B model is not consistently closer to full-KG behavior than the 12B or 7B models.

Figure 4: Triple importance depends on semantic role. Removing triples grouped by relation type shows that the most disruptive graph facts are not determined by topology alone. Outcome-facing and task-relevant relations often produce larger changes than purely structural bridge status, supporting the view that the compressed useful subset is semantically organized.

and ranking-baseline experiments to test whether graph influence is driven by relevance, relation type, topology, or redundancy. In contradiction experiments, high-relevance incorrect triples cause larger semantic shifts than low-relevance contradictions and reduce TRR more strongly. This shows that models are not treating the KG as passive background context: corrupting a central fact can change the generated hypothesis.

Figure 5 shows the same compression pattern within Llama-3.1 8B and 70B. Both models move closer to full-KG behavior as more top-ranked triples are added. The 70B model is less driven by broad graph recall under full, random, and shuffled settings. Thus, scaling changes the form of KG dependence, but does not produce a simple rule that larger models always need fewer triples. The full Mistral-family sufficiency and TRR tables are reported in Appendix E.

Relation-type analysis shows that outcomefacing and task-relevant triples are often influential, but topology-based selectors and random subsets can also recover full-KG behavior when enough triples are retained. This suggests that triple importance is not reducible to any single ranking rule; rather, the full graph contains redundant routes to similar hypothesis behavior, and compact subsets can preserve enough of that signal. Additional knockout statistics for bridge, peripheral, random, and relation-type removals are shown in Appendix F.

We also studied whether these effects are sensitive to generation randomness. On a repeatedsampling subset, between-condition TRR differences are much larger than within-condition sampling variation. About 18.1× larger for Llama-3.170B and 14.7× larger for Mistral-7B. This shows that the main effects are not driven by sampling noise. Detailed repeated-sampling statistics are reported in Appendix D.2. Encoder-sensitivity results are reported in Appendix H. Additional knockout statistics for bridge, peripheral, and random removals are shown in Appendix F. 8

Figure 5: Within-family scaling for Llama-3.1 8B→70B. Panel (A) shows semantic distance between top-k outputs and the full-KG output; lower values mean that the compressed subset better recovers full-KG behavior. Panel (B) shows TRR recovery relative to the full-KG condition. Panel (C) compares TRR across KG conditions. Both models show compression, but the 70B model is less driven by broad graph recall under full, random, and shuffled graph contexts.

Discussion

Conclusion

We studied how language models use external knowledge graphs for hypotheses generation. Across Mistral-7B, Llama-3.1-70B, and Gemini 2.5 Flash, KG utility is model-dependent. Compact subgraphs often approximate full-KG behavior, and outcome-held-out results show that this compression is not just claimed-outcome leakage. At the same time, fixed-reference and ranking-baseline analyses show that compression is not unique to one semantic selector; local scientific KGs contain redundant routes to similar hypothesis behavior. Our results suggest that KG-guided hypothesis generation should focus less on expanding graph size alone and more on diagnosing which compact pieces of structured context actually change model behavior.

We find that external graphs influence hypothesis generation, but their useful signal is not proportional to graph size. Compact subgraphs often approximate full-KG behavior, and removing highranked subsets can disrupt generation. However, the revised fixed-reference and ranking-baseline analyses show that this compression should be interpreted as redundancy-aware rather than selectorunique. No-KG outputs already recover some fullKG entities from model priors, and random or topology-based compact subsets can also recover much of the full-KG signal when enough triples are retained. The outcome-held-out control further shows that compression is not merely claimed-outcome leakage. Removing outcome-facing triples reduces relation fidelity, but top-8 no-outcome subgraphs still preserve substantial relation and mechanism/intervention signal. Thus, outcome triples are highleverage, but useful KG signal also flows through mechanism, intervention, and failure-mode relations. These findings suggest that graph compression should be guided by scientific role, but not only by role. Failure modes, interventions, mechanisms, and outcomes are useful signals, yet fixed-reference baselines show that redundancy and model priors also matter. The practical implication is that KG-guided generation should diagnose which compact pieces of context change behavior, rather than assuming that larger graph neighborhoods are always better.

Limitations

This study is limited to hypothesis generation in materials science, so the results may not be directly transferred to other scientific domains. Our graphuse metrics are diagnostic rather than full measures of scientific value. Fixed-reference scoring reduces denominator artifacts and avoids artificial no-KG zeros, but it still measures recovery of structured graph content rather than expert judgments of novelty, feasibility, or experimental usefulness. We therefore interpret the results as evidence about how models use structured context, not as a complete evaluation of hypothesis quality.

References

Lovisa Hagström, Sara Vera Marjanović, Haeun Yu, Arnav Arora, Christina Lioma, Maria Maistro, Pepa Atanasova, and Isabelle Augenstein. 2024. A Reality Check on Context Utilisation for Retrieval-Augmented Generation. arXiv e-prints, arXiv:2412.17031.

Oshin Agarwal, Heming Ge, Siamak Shakeri, and Rami Al-Rfou. 2020. Knowledge Graph Based Synthetic Corpus Generation for Knowledge-Enhanced Language Model Pre-training. arXiv e-prints, arXiv:2010.12688.

Haoyu Han, Yu Wang, Harry Shomer, Kai Guo, Jiayuan Ding, Yongjia Lei, Mahantesh Halappanavar, Ryan A. Rossi, Subhabrata Mukherjee, Xianfeng Tang, Qi He, Zhigang Hua, Bo Long, Tong Zhao, Neil Shah, Amin Javari, Yinglong Xia, and Jiliang Tang. 2024. Retrieval-Augmented Generation with Graphs (GraphRAG). arXiv e-prints, arXiv:2501.00309.

Rie Kubota Ando and Tong Zhang. 2005. A framework for learning predictive structures from multiple tasks and unlabeled data. Journal of Machine Learning Research, 6:1817–1853. Jinheon Baek, Sujay Kumar Jauhar, Silviu Cucerzan, and Sung Ju Hwang. 2024. ResearchAgent: Iterative Research Idea Generation over Scientific Literature with Large Language Models. arXiv e-prints, arXiv:2404.07738.

Linmei Hu, Zeyi Liu, Ziwang Zhao, Lei Hou, Liqiang Nie, and Juanzi Li. 2022. A Survey of Knowledge Enhanced Pre-trained Language Models. arXiv eprints, arXiv:2211.05994.

Agustín Borrego, Danilo Dessì, Daniel Ayala, Inma Hernández, Francesco Osborne, Diego Reforgiato Recupero, Davide Buscaldi, David Ruiz, and Enrico Motta. 2025. Research hypothesis generation over scientific knowledge graphs. Know.-Based Syst., 315(C).

Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothée Lacroix, and William El Sayed. 2023. Mistral 7B. arXiv e-prints, arXiv:2310.06825.

Jianguo Chen, Yu Wang, Dongxu Guo, Zhiyong Liu, Yiduo Wang, Suran Li, Wendong Xu, Linglong Qian, Yifan Shen, Tao Sun, Xuebing Han, Minggao Ouyang, and Yuejiu Zheng. 2026. Advancing battery research through large language models: A review. The Innovation, 7(2):101091.

Bowen Jin, Chulin Xie, Jiawei Zhang, Kashob Kumar Roy, Yu Zhang, Zheng Li, Ruirui Li, Xianfeng Tang, Suhang Wang, Yu Meng, and Jiawei Han. 2024. Graph Chain-of-Thought: Augmenting Large Language Models by Reasoning on Graphs. arXiv eprints, arXiv:2404.07103.

Xuexin Chen, Ruichu Cai, Zhengting Huang, Yuxuan Zhu, Julien Horwood, Zhifeng Hao, Zijian Li, and Jose Miguel Hernandez-Lobato. 2024. Feature Attribution with Necessity and Sufficiency via Dual-stage Perturbation Test for Causal Explanation. arXiv eprints, arXiv:2402.08845.

Andrej Kastrin, Bojan Cestnik, and Nada Lavrač. 2025. Recent Advances and Future Directions in LiteratureBased Discovery. arXiv e-prints, arXiv:2506.12385. Adithya Kulkarni, Fatimah Alotaibi, Xinyue Zeng, Longfeng Wu, Tong Zeng, Barry Menglong Yao, Minqian Liu, Shuaicheng Zhang, Lifu Huang, and Dawei Zhou. 2025. Scientific Hypothesis Generation and Validation: Methods, Datasets, and Future Directions. arXiv e-prints, arXiv:2505.04651.

Gheorghe Comanici, Eric Bieber, Mike Schaekermann, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, Luke Marris, Sam Petulla, Colin Gaffney, Asaf Aharoni, Nathan Lintz, Tiago Cardal Pais, Henrik Jacobsson, Idan Szpektor, Nan-Jiang Jiang, and 181 others. 2025. Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities. arXiv e-prints, arXiv:2507.06261.

Shrinidhi Kumbhar, Venkatesh Mishra, Kevin Coutinho, Divij Handa, Ashif Iquebal, and Chitta Baral. 2025. Hypothesis Generation for Materials Discovery and Design Using Goal-Driven and Constraint-Guided LLM Agents. arXiv e-prints, arXiv:2501.13299.

Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad AlDahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Korenev, Arthur Hinsvark, and 181 others. 2024. The Llama 3 Herd of Models. arXiv e-prints, arXiv:2407.21783.

Mufei Li, Siqi Miao, and Pan Li. 2024. Simple Is Effective: The Roles of Graphs and Large Language Models in Knowledge-Graph-Based Retrieval-Augmented Generation. arXiv e-prints, arXiv:2410.20724. Xingxuan Li, Ruochen Zhao, Yew Ken Chia, Bosheng Ding, Shafiq Joty, Soujanya Poria, and Lidong Bing. 2023. Chain-of-Knowledge: Grounding Large Language Models via Dynamic Knowledge Adapting over Heterogeneous Sources. arXiv e-prints, arXiv:2305.13269.

Sai Gurrapu, Ajay Kulkarni, Lifu Huang, Ismini Lourentzou, Laura Freeman, and Feras A. Batarseh. 2023. Rationalization for Explainable NLP: A Survey. arXiv e-prints, arXiv:2301.08912.

Jasper Linders and Jakub M. Tomczak. 2025. Knowledge Graph-extended Retrieval Augmented Generation for Question Answering. arXiv e-prints, arXiv:2504.08893.

Han Wang, Archiki Prasad, Elias Stengel-Eskin, and Mohit Bansal. 2025. Retrieval-Augmented Generation with Conflicting Evidence. arXiv e-prints, arXiv:2504.13079.

Nelson F. Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang. 2023. Lost in the Middle: How Language Models Use Long Contexts. arXiv e-prints, arXiv:2307.03172.

Yilin Wen, Zifeng Wang, and Jimeng Sun. 2023. MindMap: Knowledge Graph Prompting Sparks Graph of Thoughts in Large Language Models. arXiv e-prints, arXiv:2308.09729.

Weijie Liu, Peng Zhou, Zhe Zhao, Zhiruo Wang, Qi Ju, Haotang Deng, and Ping Wang. 2019. K-BERT: Enabling Language Representation with Knowledge Graph. arXiv e-prints, arXiv:1909.07606.

Guangzhi Xiong, Eric Xie, Amir Hassan Shariatmadari, Sikun Guo, Stefan Bekiranov, and Aidong Zhang. 2024. Improving Scientific Hypothesis Generation with Knowledge Grounded Large Language Models. arXiv e-prints, arXiv:2411.02382.

Xiao Long, Liansheng Zhuang, Aodi Li, Minghong Yao, and Shafei Wang. 2025. EPERM: An Evidence Path Enhanced Reasoning Model for Knowledge Graph Question and Answering. arXiv e-prints, arXiv:2502.16171.

Michihiro Yasunaga, Antoine Bosselut, Hongyu Ren, Xikun Zhang, Christopher D Manning, Percy Liang, and Jure Leskovec. 2022. Deep Bidirectional Language-Knowledge Graph Pretraining. arXiv eprints, arXiv:2210.09338.

Shengjie Ma, Chengjin Xu, Xuhui Jiang, Muzhi Li, Huaren Qu, Cehao Yang, Jiaxin Mao, and Jian Guo. 2024. Think-on-Graph 2.0: Deep and Faithful Large Language Model Reasoning with Knowledge-guided Retrieval Augmented Generation. arXiv e-prints, arXiv:2407.10805.

Xikun Zhang, Antoine Bosselut, Michihiro Yasunaga, Hongyu Ren, Percy Liang, Christopher D. Manning, and Jure Leskovec. 2022. GreaseLM: Graph REASoning Enhanced Language Models for Question Answering. arXiv e-prints, arXiv:2201.08860. Xu Zhang, Xiaojun Kang, Hong Yao, and Lijun Dong. 2025. Measuring the inferential values of relations in knowledge graphs. Algorithms, 18(1).

Costas Mavromatis and George Karypis. 2024. GNNRAG: Graph Neural Retrieval for Large Language Model Reasoning. arXiv e-prints, arXiv:2405.20139.

Zhengyan Zhang, Xu Han, Zhiyuan Liu, Xin Jiang, Maosong Sun, and Qun Liu. 2019. ERNIE: Enhanced Language Representation with Informative Entities. arXiv e-prints, arXiv:1905.07129.

Shirui Pan, Linhao Luo, Yufei Wang, Chen Chen, Jiapu Wang, and Xindong Wu. 2023. Unifying Large Language Models and Knowledge Graphs: A Roadmap. arXiv e-prints, arXiv:2306.08302.

Boci Peng, Yun Zhu, Yongchao Liu, Xiaohe Bo, Haizhou Shi, Chuntao Hong, Yan Zhang, and Siliang Tang. 2024. Graph Retrieval-Augmented Generation: A Survey. arXiv e-prints, arXiv:2408.08921.

Qualitative Case Study

Table 5 shows one representative example. The noKG output gives a plausible but generic additivebased hypothesis. The full-KG output is more specific: it identifies a low-molecular-weight fluorinated ether additive, connects it to dense electrical double layer ordering, and links that mechanism to desolvation impedance. This illustrates why entity recall alone is insufficient: the important change is the intervention–mechanism–failure connection.

Matthew E. Peters, Mark Neumann, Robert L. Logan, IV, Roy Schwartz, Vidur Joshi, Sameer Singh, and Noah A. Smith. 2019. Knowledge Enhanced Contextual Word Representations. arXiv e-prints, arXiv:1909.04164. Yiyuan Pu, Daniel Beck, and Karin Verspoor. 2023. Graph embedding-based link prediction for literaturebased discovery in alzheimer’s disease. Journal of Biomedical Informatics, 145:104464.

A.1

Human expert Assessment Instructions

We asked one materials-science postdoc to rate five representative examples. Each example contained the problem context and anonymized hypotheses from the No KG, Top-8 KG, and Full KG conditions. The condition labels were hidden during rating. The rater was instructed not to infer which system produced which output and to judge the hypotheses only by their scientific content. The rater was recruited through professional research contact and was not recruited through a crowdsourcing platform; no paid crowdwork was used.

Scott Spangler, Angela D. Wilkins, Benjamin J. Bachman, Meena Nagarajan, Tajhal Dayaram, Peter Haas, Sam Regenbogen, Curtis R. Pickering, Austin Comer, Jeffrey N. Myers, Ioana Stanoi, Linda Kato, Ana Lelescu, Jacques J. Labrie, Neha Parikh, Andreas Martin Lisewski, Lawrence Donehower, Ying Chen, and Olivier Lichtarge. 2014. Automated hypothesis generation based on mining scientific literature. In Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’14, page 1877–1886, New York, NY, USA. Association for Computing Machinery.

A.2

The rater received the following instructions:

Human expert Case Study Examples

Five case-study examples used for the domain-rater check

For each example, please read the problem/context and the anonymized hypotheses. Please rate each hypothesis on a 1–5 scale, where 1 means poor or absent, 3 means acceptable, and 5 means strong.

We selected five examples where the full-KG output recovered graph content that the no-KG output missed. In all five cases, the full-KG output had TRR = 1.0, the no-KG output had TRR = 0.0, and ∆TRR = 1.0. The goal was not to run a large human evaluation, but to check whether higher graph recovery corresponded to qualitatively better scientific hypotheses.

Problem alignment: Does the hypothesis address the stated battery/materials problem?

1. Paper 1: Thermal management for lithium-ion UUV batteries. The full-KG output specifies a pressure-compensating phase-change material system and links it to thermal instability, capacity fade, and impedance growth.

Mechanistic specificity: Does the hypothesis give a concrete mechanism rather than generic improvement language?

2. Paper 2: Additive manufacturing of sodium-ion battery components. The full-KG output focuses on thixotropic, shear-thinning inks and volatile cosolvent control to reduce coffee-ring effects and non-uniform morphology. 3. Paper 3: Polymer-gel electrolyte design. The fullKG output specifies a low-molecular-weight fluorinated ether additive and links it to dense electrical double-layer ordering, screening, and desolvation impedance.

Intervention specificity: Does the hypothesis name a concrete material, process, coating, additive, architecture, or design intervention?

4. Paper 4: High-nickel NCM cathode stabilization. The full-KG output proposes a conformal LiAlO2 coating and connects it to hydrofluoric-acid neutralization, parasitic surface reactions, and cycling stability.

Scientific plausibility: Is the hypothesis chemically or materials-science plausible?

5. Paper 5: Lithium-ion battery recycling. The fullKG output proposes mild electrochemical dissolution at the cathode-current collector interface to weaken binder adhesion and reduce material loss and cross-contamination.

Evidence faithfulness: Is the hypothesis faithful to the supplied structured evidence and graph facts? After scoring, please also answer two pairwise questions: (i) is the Top-8 KG hypothesis better than the No-KG hypothesis, worse than it, or about the same? (ii) is the Full KG hypothesis better than the No-KG hypothesis, worse than it, or about the same?

A.3 Model-Specific Top Graph Examples and Prompting Guidance Table 4 summarizes the model-specific prompting guidance derived from the best-performing graph conditions. The representative examples in Box A.3 show that the strongest graph prompts preserve a materials-science chain from failure mode to intervention, mechanism, target property, and outcome.

Please do not reward length alone. Prefer hypotheses that clearly connect the problem, intervention, mechanism, and target property in a scientifically plausible way.

Representative top graph prompts by model Gemini: Sparse + T3 multihop + full path. The selected example concerns metal–iodine batteries, where rapid capacity degradation and self-discharge arise from polyiodide dissolution and migration. The graph-conditioned output achieves RFSref = 1.000, TRRref = 0.750, and an RFS gain of +0.800 over no KG. Materials-science chain in the supplied graph:

We report the mean scores across the five examples in Table 3. Because this assessment uses one rater and five examples, we interpret it only as a qualitative check that graph-conditioned outputs are more mechanistically grounded in representative cases.

• Failure mode → polyiodide shuttle effect and low electronic conductivity

Table 4: Model-specific graph prompting guidance. The most effective graph style differs by model, but useful prompts preserve the materials-science chain from failure mode to intervention, mechanism, and target property. Model

Best graph style

Practical guidance

Gemini

Sparse, rich, full-path

Llama-3.1-70B

Sparse, coarse, 2-hop

Mistral-7B

Dense, rich, 2-hop

Use concise typed chains linking failure, intervention, mechanism, target property, and outcome. Avoid large undifferentiated graph dumps. Use short local graphs centered on the problem, with simple relation labels. Concise local grounding appears sufficient. Provide more explicit local scaffolding with clear relation labels. Avoid noisy or shuffled relations.

• Intervention → N,P-doped hierarchical porous carbon hosts with quaternary ammonium coating

• Target property → cycle life and rate capability • Outcome → suppressed self-discharge and enhanced electrochemical stability

• Mechanism → synergistic chemisorption by heteroatoms and physical confinement

Prompting takeaway: Mistral-7B benefits from denser local scaffolding with explicit relation labels.

• Target property → cycle life and rate capability • Outcome → suppressed self-discharge and enhanced electrochemical stability

Prompting takeaway: Gemini benefits from compact, semantically rich paths that connect failure, intervention, mechanism, and outcome.

Metric Implementation and KG Controls

Data accounting. The core dataset contains 100 unique battery-science problems. Larger row counts in the appendix arise from aggregation over models, KG conditions, top-k values, or repeated samples. For example, the main pipeline contains 100 problems × 11 KG conditions × 3 models = 3,300 generations. The contradiction experiment contains 50 problems × 9 conditions × 3 models = 1,350 generations. The sufficiency experiment contains 100 problems × 12 graph settings × 3 models = 3,600 generations. The centrality analysis contains 30 problems × 4 values of k × 5 selection methods × 2 models = 1,200 generations. We specify the aggregation unit in each table caption.

Llama-3.1-70B: Sparse + T1 coarse + 2-hop. The selected example concerns high-capacity lithium-ion anodes with volumetric expansion and unstable SEI formation. The graph-conditioned output achieves RFSref = 1.000, TRRref = 0.375, and an RFS gain of +0.600 over no KG. Materials-science chain in the supplied graph: • Failure mode → mechanical pulverization and continuous electrolyte decomposition • Intervention → polydopamine-derived nitrogendoped carbon nanoshell overcoating • Mechanism → mechanically resilient and ionically conductive barrier • Target property → cycling stability and rate performance

Compute and infrastructure. Open-model inference was run on H200 GPUs and Gemini 2.5 Flash was accessed through an API. We do not train or fine-tune models. The exact provider-side compute and parameter counts for Gemini 2.5 Flash are not publicly available.

• Outcome → enhanced capacity retention and prevention of active-material pulverization Prompting takeaway: Llama-3.1-70B benefits from concise local graphs with simple relation labels. Mistral-7B: Dense + T3 multihop + 2-hop. The selected example again concerns metal-iodine batteries with the polyiodide shuttle effect. The graph-conditioned output achieves RFSref = 0.600, TRRref = 0.875, and an RFS gain of +0.400 over no KG. Materials-science chain in the supplied graph:

B.1

Computing RFS and KTC

Relation Fidelity Score (RFS) is a deterministic, rule-based metric that measures whether relation roles present in the supplied KG are reflected in the generated hypothesis. Each KG edge is typed when the graph is built from structured scientific fields. We map typed edges into broad scientific roles, including failure, intervention, mechanism, property, component, system, and outcome. For each relation role present in the KG context, we

• Failure mode → polyiodide shuttle effect and low electronic conductivity • Intervention → N,P-doped hierarchical porous carbon hosts with quaternary ammonium coating • Mechanism → synergistic chemisorption by heteroatoms and physical confinement

Table 5: Qualitative example. The full-KG condition makes the hypothesis more specific by grounding the intervention in a mechanism involving electrical double layer screening and desolvation impedance. Condition

Output excerpt

No KG

The incorporation of tailored amphiphilic additives into a polymer-gel electrolyte will lower the Li-ion desolvation energy barrier and reduce interfacial resistance by disrupting the dense electrical double layer’s screening effect. Full KG The incorporation of a low-molecular-weight fluorinated ether additive into the polymer-gel electrolyte will disrupt dense electrical double layer ordering and reduce its screening effect, thereby mitigating electrical double layer screening and desolvation impedance. Observation The no-KG output is plausible but generic. The full-KG output specifies the additive class and ties it more directly to the failure mechanism and target impedance pathway.

check whether the generated hypothesis contains language associated with that role. Each role is associated with a small manually defined keyword inventory. For example, mechanism relations are associated with cues such as because, via, through, mechanism, and mediated; failure relations with cues such as degradation, capacity fade, instability, dissolution, and plating; intervention relations with cues such as coating, doping, modification, engineering, and treatment; and outcome/property relations with cues such as improve, enhance, reduce, suppress, and achieve. RFS is computed as:

inventory to test whether relation-role language appears in the hypothesis. KTC measures the overlap of the normalized content on the object-side between the supplied KG and the output. Because the labels are produced by fixed rules, inter-annotator agreement is not applicable. We interpret these metrics as proxies for graph use rather than expert judgments of scientific quality.

Degenerate graph conditions. TRR, RFS, and KTC are defined relative to the graph information supplied in a condition. In the no-KG condition, no KG objects or relations are provided, so graph-use metrics are set to 0 by convention and interpreted as X 1 “no graph signal available,” not as a general judgRFS(y, G) = 1[signals(r)∩y ̸= ∅], ment of output quality. Similarly, in the entity-only |R(G)| r∈R(G) condition, object strings are provided but relation where R(G) is the set of relation roles in the sup- labels are removed; RFS is therefore 0 because plied KG and y is the generated hypothesis. Be- there are no supplied relation roles to preserve. In cause RFS is rule-based and deterministic, inter- the relation-skeleton condition, relation roles are provided but concrete object entities are masked, annotator agreement is not applicable; we interpret so RFS can be nonzero while TRR and KTC reit as an operational diagnostic of relation-language main near zero. This behavior is intentional: the preservation, not as an expert semantic parser. metrics diagnose which kind of graph information KG Triple Coverage (KTC) measures how much is available and used. of the KG object-side content is reflected in the generated hypothesis. For each KG triple supplied B.2 Top-k Triple Ranking (si , ri , oi ), we extract normalized content words The targeted and compressed KG variants use a from the object oi and compare them to normalized deterministic relevance score to rank triples before content words in the generated hypothesis: selecting the top subset k. For each edge, we score |terms(OG ) ∩ terms(y)| the object label against the problem statement: KTC(y, G) = , |terms(OG )| |words(oe ) ∩ words(p)| s(e) = × b(e), where OG is the set of object strings in the sup|words(oe )| plied KG. KTC differs from TRR in granularity: where oe is the object label for edge e, p is the TRR asks whether a triple’s object entity is recalled, problem statement, and b(e) is a relation-type boost. while KTC measures broader object-side term covWe set b(e) = 1.3 for mechanism, failure, and erage. intervention triples, because these roles are central RFS/KTC reliability. RFS and KTC are deter- to scientific hypothesis generation, and b(e) = 1.0 ministic diagnostic metrics, not human annotation otherwise. This ranking is intentionally simple metrics. RFS uses a fixed relation-role keyword and transparent; it is used to test whether a small 14

relevance-ranked subset of graph facts can recover full-KG behavior. B.3

the benefit of the full graph and in several cases matches or slightly exceeds the full-KG condition. Table 6 reports these results.

Random, Shuffled, Entity-Only, and Relation-Skeleton Controls

B.5

Table 7 reports the distribution of KG context lengths for the necessity variants. These values are used to check whether the intervention results could be explained only by prompt length. The variants differ in length because they intentionally remove or preserve different kinds of information. Full KG is longest, targeted KG is shorter but still contains concrete entity and relation content, while entity-only and relation-skeleton variants are much shorter. No KG is a fixed short baseline. These length differences are important because long-context models may not use all parts of the context uniformly. Prior work on long-context language models shows that performance can depend on where relevant information appears in the input, with models often struggling to use information placed in the middle of long contexts. We therefore interpret the length-controlled controls, especially random and shuffled KG, together with the targeted and top-k experiments.

We use several KG variants to separate different possible sources of graph utility. The random KG condition replaces the problem’s real KG with a KG subgraph sampled from a different problem. Where possible, we sample a random subgraph with the same number of triples as the corresponding real condition. This controls for the number of explicit facts, although exact token length may still differ because object labels vary in length. The shuffled KG condition keeps the entities from the real KG but randomly permutes the relation labels. This preserves much of the surface content while breaking the mapping between entities and their scientific roles. The comparison between real KG and shuffled KG therefore tests whether the model is sensitive to relation structure rather than only entity co-occurrence. The entity-only condition removes relation labels and keeps only object-side entity strings. This tests whether graph gains can be explained by lexical exposure to scientific terms. The relationskeleton condition preserves relation types and typed placeholders while masking concrete entity identity. This tests whether relation-role information alone induces relation-consistent language. B.4

Context Length Distributions

B.6

KG Variant Definitions

All graph conditions are constructed from a perproblem local KG Gp = (Vp , Ep ). Each edge e = (s, r, o) ∈ Ep is a typed subject–relation–object triple derived from structured scientific fields such as material system, component, failure mode, intervention, mechanism, target property, and claimed outcome. The full local KG contains 15–18 triples per problem.

Necessity-Control Results

We next ask what kind of graph information is actually necessary for the observed gains. To test this, we compare the full KG against four reduced variants: no KG, entity-only context, relation skeleton, targeted KG, and full KG. This experiment isolates whether performance depends on raw entity strings, relation type information, or a compact targeted subset of graph facts. The results show that, entity-only context is not enough. Across models, providing only entity names produces near-zero performance on the main KG-grounding metrics, indicating that simple lexical exposure to graph entities does not recover the full-KG effect. Additionally, relation skeleton preserves mostly relation fidelity. It recovers some RFS while leaving TRR and KTC near zero. This shows that relation-type information is meaningful on its own, but it is not sufficient to reproduce the full behavior of graph-conditioned hypothesis generation. Third, targeted KG recovers most of

Additional Statistical Results

C.1

Paired Permutation Tests

We use paired permutation tests for condition-level contrasts. For each problem, we compute the metric difference between two KG conditions and test whether the mean paired difference is larger or smaller than expected under random sign flips. This is a non-parametric repeated-measures test and does not assume normality of metric values. C.1.1 Control Comparisons Table 9 reports the two main control comparisons. The real-vs-random comparison tests whether models benefit from relevant graph facts rather than generic extra context. The real-vs-shuffled comparison keeps much of the surface content but corrupts 15

Table 6: Necessity-control results across KG variants. Entity-only context does not recover graph-grounded behavior. Relation skeletons recover relation-fidelity signal but not entity or coverage metrics. Targeted KG recovers much of the full-KG behavior, supporting the claim that useful graph signal is concentrated in a compact subset. Variant

No KG Entity-only Rel. skeleton Targeted KG Full KG

Gemini

Llama-3.1-70B

Mistral-7B

TRR

RFS

KTC

TRR

RFS

KTC

TRR

RFS

KTC

0.000 0.000 0.000 0.780 0.594

0.000 0.000 0.544 0.640 0.546

0.000 0.000 0.000 0.598 0.526

0.000 0.000 0.010 0.730 0.661

0.000 0.000 0.570 0.629 0.546

0.000 0.000 0.000 0.546 0.541

0.000 0.000 0.045 0.855 0.799

0.000 0.000 0.574 0.600 0.596

0.000 0.000 0.000 0.664 0.640

Table 7: Context-length distribution by KG variant. We report the distribution of KG context lengths used in the necessity-control experiment. These lengths help check whether gains are driven simply by longer prompts. Variant Entity-only Full KG No KG Relation skeleton Targeted KG

Count

Mean

Std.

Min

75%

Max

Table 12 supports the main claims in the paper. Gemini 2.5 Flash has stable positive gains for real KG over random KG, real KG over no KG, and real KG over shuffled KG on relation fidelity. Llama3.1-70B shows smaller positive effects, especially for KTC. Mistral-7B shows weak real-vs-random gains and a negative real-vs-shuffled RFS effect, consistent with brittle use of relation structure.

Structural Ablations

Table 11 reports paired tests for the structural KG ablations. These comparisons test whether graph density, ontology richness, and topology change how much KG information appears in the generated output. The sparse-vs-medium comparison evaluates whether adding more graph facts helps or hurts. The T1-vs-T3 comparison tests coarse versus richer ontology structure. The 2-hop-vsfull-path comparison tests whether deeper topology adds useful context beyond the shorter local graph. These results support the main-text claim that more graph context is not uniformly better. C.2

50%

200 648.1 36.6 555.0 623.5 650.5 669.0 745.0 200 1348.1 36.6 1255.0 1323.5 1350.5 1369.0 1445.0 200 64.0 0.0 64.0 64.0 64.0 64.0 64.0 200 664.6 7.9 644.0 659.0 664.5 670.2 685.0 200 1080.3 40.2 999.0 1053.5 1085.0 1105.0 1197.0

relation structure, so it tests whether models are sensitive to the organization of the graph. These results support the main-text claim that Gemini shows the strongest and most stable use of KG structure, Llama-3.1-70B shows smaller gains, and Mistral-7B is weaker and more brittle under corrupted graph structure. C.1.2

25%

Robustness Checks

We include two additional robustness checks. First, we compare our semantic top-k ranking against matched random and topology-based top-k selectors. This tests whether the compression result is driven only by semantic relevance, by shorter prompts, or by graph centrality. Second, we estimate sampling variability by generating multiple outputs for a subset of conditions. D.1 Top-k Semantic Ranking Versus Topology Baselines

Bootstrap Confidence Intervals

The main experiments use a simple semantic relevance score to select top-k triples. To test whether the compression effect depends on this particular ranking rule, we compare semantic top-k selection against four matched baselines: random-k, degree-k, betweenness-k, and PageRank-k. All methods select the same number of triples, so the comparison controls for the size of the compressed graph. Degree, betweenness, and PageRank provide topology-based alternatives to the semantic

Bootstrap confidence intervals provide uncertainty estimates for the main effect sizes reported in the paper. Unlike the permutation tests, which test whether a paired difference is unlikely under a null hypothesis, the bootstrap intervals estimate the likely range of each effect size. We use them to check whether the main directional effects are stable across resampled problem sets. Intervals that do not cross zero indicate stable directional effects. 16

Table 8: Definitions of KG variants. All variants are generated from the same per-problem local KG. Density, ontology, and topology variants change the amount or form of graph context, while control variants test whether models use relevant graph content and relation structure. Variant group Condition

Definition

No graph Density Density Density Ontology Ontology Topology Topology Controls Controls Controls Controls Compression Compression

Prompt contains the scientific problem but no graph triples. Small relevance-ranked subset of Ep with few graph facts. Intermediate subset of Ep with more graph facts than sparse. Expanded verbalization of the local graph with the largest graph context. Coarse relation labels over the same problem fields. Richer relation labels and multihop scientific roles. Triples within a local two-hop neighborhood of the problem node. Longer problem-to-field/path view retaining extended graph paths. Same triple count as the matched real condition, but sampled from another problem. Same entities as the real KG, but relation labels are randomly permuted. Keeps object-side entity strings but removes relation labels. Keeps relation types/placeholders but masks concrete entity identity. Relevance-ranked subset selected by the deterministic score in Appendix B.2. Keeps the top k ranked triples, with k ∈ {1, 2, 4, 8}.

No KG Sparse Medium Dense T1 coarse T3 multihop 2-hop Full path Random KG Shuffled KG Entity-only Relation skeleton Targeted KG Top-k KG

ranking. Table 14 reports semantic distance to the full-KG output. Lower values mean that the selected top-k subset better approximates the behavior induced by the full graph. We observe that compression is robust. For both Llama-3.1-70B and Mistral-7B, increasing k consistently reduces semantic distance for all ranking methods. However, semantic ranking is not always the best selector. Random and topology-based methods sometimes match or outperform the semantic ranking at particular values of k. We therefore do not claim that our semantic ranking is optimal. Instead, these results show that the central compression effect is not an artifact of one ranking heuristic: several small matched subsets can recover much of the full-KG behavior. We also recompute the ranking-baseline comparison using fixed-reference TRR. This avoids the smaller-denominator issue for top-k subsets. The result reinforces the main interpretation: compression is robust, but not selector-unique. Random and topology-based subsets can match or exceed semantic top-k under fixed-reference scoring, especially when k = 8. D.2

pling variation. For Llama-3.1-70B, the betweencondition range is 0.9100 and the within-condition standard deviation is 0.0504, giving a signal-tonoise ratio of 18.1×. For Mistral-7B, the betweencondition range is 0.9167 and the within-condition standard deviation is 0.0625, giving a signal-tonoise ratio of 14.7×. This suggests that the main effects are not artifacts of single-sample generation noise. Tables 16 and 17 further show per-condition mean TRR and standard deviations in the repeatedsampling check. The condition ranking remains broadly stable under repeated sampling, and withincondition variance is small relative to betweencondition differences across all conditions.

Additional Intra-Family Scaling Results

The main text includes within-family checks for Llama and Mistral to test whether the compression result is only a cross-family artifact. Here we report the full Mistral-family numbers. The purpose of this analysis is not to claim a simple parameter-count scaling law. Instead, it checks whether the top-k compression effect remains visible when model family is held fixed. Tables 18 and 19 report TRR and semantic distance results respectively. All three Mistral models move closer to the full-KG output as k increases, confirming that the compression effect is present within the family. However, the ordering is not monotonic in model size: the 22B model is not consistently closer to full-KG behavior than the 12B or 7B models.

Sampling Variability

We also test whether the observed condition effects are larger than generation noise. For a subset of problems and conditions, we generate multiple samples and compute the within-condition standard deviation of TRR. We compare this to the betweencondition range in mean TRR across conditions. Table 15 shows that between-condition differences are much larger than within-condition sam17

Table 9: Paired permutation tests for the two most important control comparisons: replacing the real KG with a random KG, and shuffling KG structure while keeping content. Positive ∆ indicates that the first condition outperforms the second. Significance: ∗ p < 0.05, ∗∗ p < 0.01, ∗∗∗ p < 0.001. ∆

Comparison

p-value Sig.

Model

Metric

Gemini 2.5 Flash Gemini 2.5 Flash Gemini 2.5 Flash Gemini 2.5 Flash Gemini 2.5 Flash Mistral-7B Mistral-7B Mistral-7B Mistral-7B Mistral-7B Llama-3.1-70B Llama-3.1-70B Llama-3.1-70B Llama-3.1-70B Llama-3.1-70B

TRR Real KG vs Random KG +0.2900 RFS Real KG vs Random KG +0.0033 KTC Real KG vs Random KG +0.2559 FGS Real KG vs Random KG +0.0272 CHAS Real KG vs Random KG +0.0685 TRR Real KG vs Random KG +0.0080 RFS Real KG vs Random KG +0.0067 KTC Real KG vs Random KG +0.0054 FGS Real KG vs Random KG +0.0061 CHAS Real KG vs Random KG +0.0062 TRR Real KG vs Random KG +0.0380 RFS Real KG vs Random KG +0.0133 KTC Real KG vs Random KG +0.0331 FGS Real KG vs Random KG -0.0043 CHAS Real KG vs Random KG +0.0076

0.0000 0.8896 0.0000 0.0574 0.0064 1.0000 1.0000 1.0000 1.0000 1.0000 0.0708 0.5530 0.0426 0.8570 0.7648

∗∗∗

TRR Real KG vs Shuffled KG RFS Real KG vs Shuffled KG KTC Real KG vs Shuffled KG FGS Real KG vs Shuffled KG CHAS Real KG vs Shuffled KG TRR Real KG vs Shuffled KG RFS Real KG vs Shuffled KG KTC Real KG vs Shuffled KG FGS Real KG vs Shuffled KG CHAS Real KG vs Shuffled KG TRR Real KG vs Shuffled KG RFS Real KG vs Shuffled KG KTC Real KG vs Shuffled KG FGS Real KG vs Shuffled KG CHAS Real KG vs Shuffled KG

0.0198 0.0000 0.1842 0.4980 0.1842 0.0000 0.0000 0.0000 0.0000 0.0000 0.3656 0.7340 0.4534 0.5746 0.6782

∗

-0.0380 +0.1867 -0.0183 -0.0087 -0.0304 -0.0900 -0.0800 -0.0731 -0.0883 -0.0827 -0.0240 +0.0100 -0.0151 -0.0106 -0.0113

∗∗∗

∗∗

ns ns ns ns ns ns ns

∗

ns ns ∗∗∗

ns ns ns

∗∗∗ ∗∗∗ ∗∗∗ ∗∗∗ ∗∗∗

ns ns ns ns ns

Table 10: Mixed-effects variance decomposition across evaluation metrics. For all three metrics, model identity explains more variance than condition identity, indicating that performance differences are driven more strongly by the underlying LLM than by KG condition alone.

tion. Table 20 summarizes the knockout semanticdistance results. The relation-type analysis gives a clearer signal: outcome-facing and task-relevant triples often produce larger changes than bridge status alone.

Metric

Dominant factor

Model effect

Condition effect

F.1

TRR RFS KTC

model model model

0.1632 0.2095 0.2254

0.0288 0.0928 0.1358

Semantic Role Versus Topological Centrality

To test whether graph importance is explained by topology alone, we compare the effect of removing three types of triples: bridge triples, peripheral triples, and randomly selected triples. Bridge triples are selected based on graph position, while peripheral triples are selected from less central graph locations. Random removals provide a baseline for removal effects not tied to either semantic role or graph centrality. Table 22 summarizes the full knockout results with per-model TRR and ∆TRR alongside semantic distance. The results do not support a simple topological-centrality explanation. Removing bridge triples changes outputs, and for Gemini 2.5 Flash it slightly reduces TRR. However, peripheral and random removals can be as disruptive or more disruptive in semantic distance. This indicates that graph position alone does not

Topology and Semantic-Role Ablations

To test whether important triples are identified by graph topology alone, we compare removals of bridge, peripheral, and random triples. Bridge triples provide a simple topology-driven baseline: they connect major parts of the problem graph and should be important if graph position alone determines influence. The results do not support a bridge-only explanation. Peripheral and random removals often produce comparable or larger semantic shifts than bridge removals, especially for Gemini and Llama-3.1-70B. This suggests that triple importance is not reducible to simple graph posi18

Table 11: Paired permutation tests for KG ablations. Positive ∆ indicates that the first condition outperforms the second. Significance: ∗ p < 0.05, ∗∗ p < 0.01, ∗∗∗ p < 0.001. ∆

p-value

Sig.

Sparse vs Medium density Sparse vs Medium density Sparse vs Medium density Sparse vs Medium density Sparse vs Medium density Sparse vs Medium density Sparse vs Medium density Sparse vs Medium density Sparse vs Medium density Sparse vs Medium density Sparse vs Medium density Sparse vs Medium density Sparse vs Medium density Sparse vs Medium density Sparse vs Medium density

+0.1010 +0.0100 +0.1440 +0.0091 -0.0346 +0.0420 +0.0483 +0.0446 +0.0394 +0.0430 +0.0420 +0.0317 +0.0475 +0.0138 +0.0243

0.0000 0.7052 0.0000 0.4162 0.1490 0.0302 0.0302 0.0302 0.0302 0.0302 0.0940 0.1486 0.0336 0.4814 0.2840

∗∗∗

TRR RFS KTC FGS CHAS TRR RFS KTC FGS CHAS TRR RFS KTC FGS CHAS

T1-coarse vs T3-multihop T1-coarse vs T3-multihop T1-coarse vs T3-multihop T1-coarse vs T3-multihop T1-coarse vs T3-multihop T1-coarse vs T3-multihop T1-coarse vs T3-multihop T1-coarse vs T3-multihop T1-coarse vs T3-multihop T1-coarse vs T3-multihop T1-coarse vs T3-multihop T1-coarse vs T3-multihop T1-coarse vs T3-multihop T1-coarse vs T3-multihop T1-coarse vs T3-multihop

-0.0080 +0.0067 +0.0012 +0.0076 -0.0168 -0.0080 -0.0067 -0.0038 -0.0008 +0.0006 +0.0340 +0.0100 +0.0234 +0.0268 +0.0290

0.6318 0.7676 0.9316 0.4336 0.3218 1.0000 1.0000 1.0000 1.0000 1.0000 0.1838 0.7106 0.2126 0.2298 0.2348

ns ns ns ns ns ns ns ns ns ns ns ns ns ns ns

TRR RFS KTC FGS CHAS TRR RFS KTC FGS CHAS TRR RFS KTC FGS CHAS

2-hop vs Full-path 2-hop vs Full-path 2-hop vs Full-path 2-hop vs Full-path 2-hop vs Full-path 2-hop vs Full-path 2-hop vs Full-path 2-hop vs Full-path 2-hop vs Full-path 2-hop vs Full-path 2-hop vs Full-path 2-hop vs Full-path 2-hop vs Full-path 2-hop vs Full-path 2-hop vs Full-path

-0.0320 -0.0200 -0.0259 -0.0003 +0.0044 +0.0100 +0.0100 +0.0072 +0.0073 +0.0093 +0.0140 +0.0233 +0.0150 +0.0132 +0.0270

0.0346 0.3746 0.0416 0.9836 0.7836 0.5098 0.7540 0.7540 0.5098 0.5098 0.4312 0.2032 0.3534 0.5216 0.2604

∗

Model

Metric

Comparison

Gemini Gemini Gemini Gemini Gemini Mistral-7B Mistral-7B Mistral-7B Mistral-7B Mistral-7B Llama-3.1-70B Llama-3.1-70B Llama-3.1-70B Llama-3.1-70B Llama-3.1-70B

TRR RFS KTC FGS CHAS TRR RFS KTC FGS CHAS TRR RFS KTC FGS CHAS

Gemini Gemini Gemini Gemini Gemini Mistral-7B Mistral-7B Mistral-7B Mistral-7B Mistral-7B Llama-3.1-70B Llama-3.1-70B Llama-3.1-70B Llama-3.1-70B Llama-3.1-70B Gemini Gemini Gemini Gemini Gemini Mistral-7B Mistral-7B Mistral-7B Mistral-7B Mistral-7B Llama-3.1-70B Llama-3.1-70B Llama-3.1-70B Llama-3.1-70B Llama-3.1-70B

identify the triples most important for generation. The relation-type analysis in the main paper gives a clearer signal: outcome-facing and task-relevant triples often produce larger changes than bridge status alone. This result motivates our main claim that the compressed useful subset is semantically organized. Topological centrality is informative, but it is not sufficient. In future work, one could compare the relevance ranking used here against explicit degree, betweenness, and PageRank rankings. Betweenness centrality measures how often a node lies on shortest paths between other nodes, while degree centrality and PageRank capture different notions of graph importance. These baselines are useful, but our current knockout results suggest that scientific role is a stronger predictor of output

∗∗∗

ns ns

∗ ∗ ∗ ∗ ∗

ns ns

∗

ns ns

∗

ns ns ns ns ns ns ns ns ns ns ns ns

disruption than simple graph position.

KG Size and Context Length

The targeted condition can have a larger length proxy than some full-path variants because selected triples are verbalized with more explicit naturallanguage context. We therefore treat this as a context-length proxy rather than a direct triplecount measure.Table 23 summarizes per-problem KG size statistics. Each problem has a compact local KG concentrated around 16 triples, meaning top-8 uses approximately half of the full local graph and top-4 uses about one quarter. Table 24 shows the distribution of triple counts across problems, confirming that local KGs are tightly concentrated. 19

Table 12: Bootstrap 95% confidence intervals for the main KG-effect contrasts. Intervals that do not cross zero indicate stable directional effects. These intervals support the main-text claim that Gemini 2.5 Flash has the strongest and most stable KG gains, Llama-3.1-70B shows smaller positive gains, and Mistral-7B remains weak or brittle under several graph manipulations. Model

Quantity

Estimate

95% CI lower

95% CI upper