GPTKB 2.0: Browsing, Querying, and Auditing a Disambiguated LLM-Derived Knowledge Base

GPTKB 2.0: Browsing, Querying, and Auditing a Disambiguated LLM-Derived Knowledge Base Yujia Hu1

Tuan-Phong Nguyen2

Simon Razniewski1

1

2

ScaDS.AI Dresden/Leipzig & TU Dresden, Germany Institute for AI, VNU University of Engineering and Technology, Hanoi, Vietnam {yujia.hu,simon.razniewski}@tu-dresden.de

tuanphong@vnu.edu.vn

arXiv:2608.06992v1 [cs.CL] 7 Aug 2026

Abstract We present a web demo for exploring a largescale disambiguated knowledge base (KB) materialized from a large language model (LLM). GPTKB 2.0 contains 38.4M triples over 1.6M canonical entities, together with 207.6K consolidated relations and 66K consolidated classes. Unlike prior LLM-derived knowledge bases that largely identify entities by surface strings, GPTKB 2.0 performs context-guided disambiguation during recursive KB construction, separating homonyms and merging synonymous mentions as facts are elicited. The demo makes this process inspectable: users can browse entities, follow links across the KB, and audit the provenance of individual facts, including surface forms, candidate matches, source triples, and disambiguation decisions. The interface further supports structured SPARQL queries, natural-language questions translated to SPARQL, and entity linking from userprovided text to canonical GPTKB 2.0 entries. GPTKB 2.0 is available at https://gptkb. org/, with the full KB downloadable for offline use.

1

Figure 1:

Introduction

Budapest in GPTKB 2.0.

modes: without further triple context, they conflate homonymous entities that share identical entity label (e.g., Munich in [Europe, hasMajorCity, Munich] vs. Munich in [Mathieu Amalric, notableWork, Munich]) and fragment synonymous strings that denote the same entity (e.g., New York City and The Big Apple). We present GPTKB 2.0, a disambiguated, general-domain KB built entirely from an LLM, together with a web interface that makes both the knowledge and its provenance explorable. GPTKB 2.0 comprises 38.4M triples over 1.6M canonicalized entities, 207.6K consolidated relations, and 66K consolidated classes. Disambiguation is performed on the fly during construction, driven by a simple observation: the eliciting triple, together

General-domain KBs (Vrandecic and Krötzsch, 2014; Auer et al., 2007; Suchanek et al., 2007) are important and longstanding backbones for AI applications, and entity disambiguation recognized as a core challenge in KB construction. More recently, large language models (LLMs) have emerged as implicit repositories of factual knowledge (Petroni et al., 2019), inspiring a line of work that constructs knowledge bases directly from LLMs by materializing their parametric knowledge in structured form (Nguyen et al., 2024; Cohen et al., 2023; Hu et al., 2025, 2026a; Parović et al.). However, these approaches largely rely on surface strings as entity identifiers. Yet surface strings are inadequate identifiers, exhibiting two complementary failure 1

with textual descriptions of existing entities, typically provides sufficient context to distinguish new entities from known ones without external resources. The demo’s distinguishing feature is transparency. Through the interface, users can (i) browse and query the full KB via SPARQL or natural language; (ii) inspect the derivation and consolidation provenance behind any fact: all its trajectories from the seed entity, and the surface forms, candidate matches, and context considered during consolidation, so that both homonymy and synonymy resolution become auditable; and (iii) link named entities in their own input text to canonical GPTKB 2.0 entries. To our knowledge, GPTKB 2.0 is the first demo to combine disambiguation provenance, SPARQL, and natural-language querying over an LLM-derived KB.

2

Entities 1,592,185 Triples 38,450,135 Relations 207,633 Classes 66,523 Triple objects (entities) 15.8M Triple objects (literals) 22.6M Avg. triples per entity 38.4 Avg. outlinks per entity 15.8 Entities novel to Wikidata* 36.8% * Validated on 1,000 samples.

Table 1: Overall statistics of GPTKB 2.0.

target entity label, its source triple, and the labels and descriptions of the nearest candidates by embedding similarity; it either matches an existing entity, marks it new, or abstains. A match inherits the incoming label as an alias. For instance, Munich (city) and Munich (film) are kept distinct through their differing source-triple context; The Big Apple is mapped to New York based on the source triple [Emerald City, contrastsWith, The Big Apple].

Construction

GPTKB 2.0 methodology (Hu et al., 2026b) represents entities, relations, and classes as first-class elements with unique identifiers, so that disambiguation can operate across homonymous and synonymous mentions. Following curated KBs such as Wikidata (Vrandecic and Krötzsch, 2014), besides unique ID, each element also carries a textual description, which serves as the context signal for disambiguation. An example for Budapest is shown in Figure 1. Starting from a seed entity, GPTKB 2.0 is built by a recursive, contextguided pipeline that expands the KB along the subject–object frontier, disambiguating each new mention on the fly. The pipeline proceeds through four context-guided steps: elicitation, named entity recognition (NER), and disambiguation.

3

Overview

As shown in Table 1, GPTKB 2.0 contains 38.4M triples and 1.6M disambiguated entities, together with 207.6K relations and 66.5K classes.

4

Web Interface

GPTKB v2 is served at https://gptkb.org/, which provides a user interface for keyword-based entity search and link-based exploration to discover connected entities. The interface is built with the Django framework and served via Nginx, with the KB stored in an OpenLink Virtuoso triple store. To our knowledge, this is the first demo interface to make the consolidation process fully traceable while combining SPARQL querying, natural-language (text-to-SPARQL) querying, and entity linking over an LLM-derived KB.

Elicitation and NER Both steps rest on one principle: a bare label is ambiguous, so we condition the LLM on context. Given an entity, the LLM is prompted for triples; since facts about Munich could concern the city or the 2005 film, we include the entity’s description so that elicited facts pertain to the intended entity. Each elicited object is then classified as a literal or a named entity, 01099 is a literal in [Neustadt Dresden, postalCode, 01099] but an entity in [Frisch, partOfBand, 01099]—using the source triple as context.

4.1

Access

Entities can be reached through several routes: 1. The start page links directly to featured entities such as Vannevar Bush and San Francisco. 2. A search field in the top-right corner supports string-based lookup.

Disambiguation Named-entity objects are resolved against the KB to merge synonyms and separate homonyms. The LLM is prompted with the

3. When an entity’s ID is known, its page can be accessed directly at https://gptkb.org/ entity/<ID> (e.g., E14 for the USA). 2

Figure 2: Hyde Park in London is recognized as distinct from other synonymous entities.

4.2

Traceable decision-making

Each entity’s page exposes the disambiguation decisions behind it, making both homonymy and synonymy auditable. To trace homonymy handling, users can inspect the candidates presented to the LLM when this entity was disambiguated, revealing how it was kept distinct from same-label candidates and judged new to the KB at the point it first appeared. As shown in Figure 2, clicking Show under How this entity was disambiguated reveals the process by which Hyde Park in London is separated from its homonyms existing in GPTKB 2.0. To trace synonymy handling, the surface forms consolidated into this entity are listed as its aliases; clicking one opens the prompt for the corresponding disambiguation step, showing how that form was merged. As shown in Figure 3, the alias list of New York City lets users click an alias (here, The Big Apple) to reveal the mentions mapped under that label; the info button then opens the prompt used at that step, listing the target entity, its source triple, and the retrieved candidates.

4.3

Linking is also context-dependent: in the example, the two occurrences of Mercury resolve to distinct entities, the planet ( Mercury) and the Roman deity ( Mercury), despite sharing a surface form.

Each decision is auditable through a disambiguation-evidence panel that lists every candidate the linker weighed. Candidates are retrieved in two ways: surface form matches, ranked by how often each entity has been observed under that surface form, and a BM25 text-relevance fallback that fills the remaining slots. Notably, context can override raw frequency: although the planet is the most frequent entity linked to Mercury (observed 53 times), the mythological mention is correctly resolved to the Roman deity (observed only 16 times), which the surrounding context favors. This transparency lets

Entity Linking

Beyond showing how entities within GPTKB v2 were linked during construction, the interface integrates LELA (Haffoudhi et al., 2026a,b), an entity-linking system, over GPTKB v2. A user enters text and clicks Link entities; LELA detects named-entity mentions and links each to its corresponding entity in GPTKB v2, or marks it NIL when no match exists. 3

Click one alias to browse mapped entities

Click to check corresponding prompt

Figure 3: The Big Apple is mapped to existing entity New York City.

users see not only which entity was chosen but which alternatives were considered and rejected. 4.4

SPARQL Query

Because GPTKB 2.0 is a fully materialized, structured KB, its contents can be queried directly rather than through repeated prompting of the underlying model. The KB is stored in an OpenLink Virtuoso triple store and exposed through a SPARQL endpoint at https://gptkb.org/query/, supporting structured queries, filtering, and aggregation over all 38.4M triples. This turns questions about the model’s knowledge that would otherwise require bespoke, large-scale prompting (e.g., thousands of templated probes (Kotek et al., 2023)) into single declarative queries evaluated with mature database technology. In the following, we show enabled analyses. Most frequent classes The most frequent classes in GPTKB 2.0 can be queried via: PREFIX disamkbp: <https://gptkb.org/prop/> SELECT ?o (COUNT(*) AS ?freq) WHERE { ?s disamkbp:P0 ?o . } GROUP BY ?o ORDER BY DESC(?freq) LIMIT 20 label human municipality tourist attraction fictional beaver song neighborhood building film

4

freq 166328 66232 51891 29751 25408 24317 23436 23376

Entities with most aliases Entities with most aliases in GPTKB 2.0 can be queried via:

PREFIX rdfs: <http://www.w3.org/2000/01/rdfschema#> PREFIX skos: <http://www.w3.org/2004/02/skos/ core#> SELECT ?e ?eLabel (COUNT(?form) AS ?nforms) WHERE { ?e (rdfs:label|skos:altLabel) ?form } GROUP BY ?e ?eLabel ORDER BY DESC(?nforms) LIMIT 20

Consistent with the rest of the interface, every step is transparent: the demo exposes the agent’s full reasoning trace, the generated SPARQL query (which users can open in the query editor), and the KB entries the agent resolved, each linked to its entity page.

As shown above, 128 distinct surface labels map to the entity United States of America. This illustrates the value of resolving entities during construction rather than afterward: because each merge is available to subsequent elicitation steps, later facts are gathered against the consolidated entity, avoiding the fragmentation that a post-hoc pass would first have to detect and then repair.

4.5

Query in Natural Language

For users unfamiliar with SPARQL, the demo integrates GRASP (Walter and Bast, 2025), an agent that answers plain-English questions over GPTKB 2.0. Given a question, the agent iteratively explores GPTKB 2.0 through tool calls, resolving the entities, properties, and classes it needs, then composes and executes a SPARQL query to produce the answer. For example, “What is Budapest known for?” is resolved by locating the Budapest entity (E13406) and the traditionallyKnownFor property (P67817), yielding a concise list of answers.

5

Evaluation

5.1

Disambiguation

Reliable link-based exploration depends on correct disambiguation: linked entities must be neither wrongly merged nor wrongly split. We manually assess both error directions on samples of 100 cases each (Table 2). Merge precision measures whether entities combined into one are truly the same: 98% of same-label objects merged into an existing entity 5

Judge

Human

LLM

LLM−

NED precision Merge correct Split correct

94.5% 97.5%

— —

— —

Triple precision True Plausible Implausible False %

94.5% 2.5% 1.0% 2.0%

92.8% 5.5% 1.2% 0.5%

80.0% 10.3% 2.0% 7.7%

Entity factuality Verifiable Plausible Unverifiable

96.0% 2.0% 2.0%

92.3% 6.7% 1.0%

— — —

6

Related Work

Disambiguated Knowledge Bases Managing entity identity is a longstanding challenge in KB construction. Curated KBs (DBpedia (Auer et al., 2007), YAGO (Suchanek et al., 2007), Wikidata (Vrandecic and Krötzsch, 2014)) inherit canonicalization from human-maintained resources, and mention-grounding methods reuse such inventories (Hoffart et al., 2011), but both bound coverage to what those resources contain; text-based systems (Carlson et al., 2010) broaden coverage yet often set consolidation aside. LLM-derived KBs (Cohen et al., 2023; Hu et al., 2025; Parović et al.) are tied to neither a fixed corpus nor a closed schema, but inherit no external identifiers and must disambiguate from scratch.

Table 2: Manual and automatic evaluation of GPTKB 2.0. Human evaluation uses n = 400 for NED precision and n = 200 for triple precision and entity factuality; LLM evaluation uses n = 1,000.

are correct (homonymy), as are 91% of differentlylabeled aliases resolved to an existing entity (synonymy); false merges (9%) in the latter are the main error mode. Split precision measures whether entities kept separate are truly distinct: all 100 samelabel pairs assigned distinct IDs are genuinely different (homonymy), and for 95% of entities no synonymous duplicate appears among their top-20 label-embedding neighbors (synonymy; an approximation, since duplicates with dissimilar surface forms may be missed). The pipeline thus handles both homonymy and synonymy reliably in both directions, with false synonym merges the largest residual error.

Traceable KB Interfaces Public KBs are typically exposed through browsers, entity pages, and SPARQL endpoints that present the KB as a finished artifact showing what it contains, not how each entry arose. For LLM-derived KBs, whose entries are generated and consolidated automatically, the provenance behind each fact and disambiguation decision is itself evidence of trustworthiness. GPTKB 2.0’s interface makes this process traceable, surfacing for each fact its derivation trajectory, the candidates weighed during disambiguation, and the context behind each decision.

5.2

We presented GPTKB 2.0, a disambiguated knowledge base materialized entirely from LLM, together with a web interface that makes both its knowledge and the disambiguation decisions behind it explorable. The KB comprises 38.4M triples over 1.6M entities, 207.6K relations, and 66K classes, all consolidated on the fly during a recursive construction pipeline. The distinguishing feature of this demo is transparency: every fact carries provenance recording the surface forms, candidate matches, and context behind its disambiguation. Through the interface, users can browse and trace entities, query the KB in SPARQL or natural language, and link named entities in their own text against it. By turning an LLM’s latent parametric knowledge into a structured, queryable, and inspectable resource, GPTKB v2 offers both a usable general-domain KB and a step toward making the knowledge inside LLMs open to scrutiny.

7

Overall Quality of GPTKB 2.0

Since exhaustive evaluation of a 38M-triple KB is infeasible, we assess randomly sampled subsets. Following YAGO (Suchanek et al., 2007) and recent factuality metrics such as FactScore (Min et al., 2023) and VeriScore (Song et al., 2024), we verify samples against web-retrieved evidence, using two paradigms: 200 items validated by humans and 1,000 items judged by an agentic pipeline. Precision exceeds 90% for both triples and entities (Table 2). For triples (labeled true, plausible, implausible, or false), 94.5% are true and 2.0% false on the human sample, and 92.8% true and 0.5% false on the automatic sample. In an ablation where the judge sees each triple without the subject entity’s description, the true rate drops to 80%, underscoring how much descriptions contribute to interpretability. For entities (labeled verifiable, plausible, or unverifiable), 96% are verifiable on the human sample and 92.3% on the automatic sample. 6

Conclusion

References

Yixiao Song, Yekyung Kim, and Mohit Iyyer. 2024. VeriScore: Evaluating the factuality of verifiable claims in long-form text generation. In Findings of EMNLP.

Sören Auer, Christian Bizer, Georgi Kobilarov, Jens Lehmann, Richard Cyganiak, and Zachary G. Ives. 2007. DBpedia: A nucleus for a web of open data. In ISWC.

Fabian M. Suchanek, Gjergji Kasneci, and Gerhard Weikum. 2007. Yago: a core of semantic knowledge. In WWW.

Andrew Carlson, Justin Betteridge, Bryan Kisiel, Burr Settles, Estevam R. Hruschka Jr., and Tom M. Mitchell. 2010. Toward an architecture for neverending language learning. In AAAI.

Denny Vrandecic and Markus Krötzsch. 2014. Wikidata: a free collaborative knowledgebase. Commun. ACM.

Roi Cohen, Mor Geva, Jonathan Berant, and Amir Globerson. 2023. Crawling the internal knowledgebase of language models. In Findings of EACL.

Sebastian Walter and Hannah Bast. 2025. GRASP: generic reasoning and SPARQL generation across knowledge graphs. In ISWC.

Samy Haffoudhi, Fabian M. Suchanek, and Nils Holzenberger. 2026a. LELA: An end-to-end LLM-based entity linking framework with zero-shot domain adaptation. In IJCAI-ECAI. Samy Haffoudhi, Fabian M. Suchanek, and Nils Holzenberger. 2026b. LELA: an LLM-based entity linking approach with zero-shot domain adaptation. In ISWC. Johannes Hoffart, Mohamed Amir Yosef, Ilaria Bordino, Hagen Fürstenau, Manfred Pinkal, Marc Spaniol, Bilyana Taneva, Stefan Thater, and Gerhard Weikum. 2011. Robust disambiguation of named entities in text. In EMNLP. Yujia Hu, Tuan-Phong Nguyen, Shrestha Ghosh, Moritz Müller, and Simon Razniewski. 2026a. GPTKB v1.5: A massive knowledge base for exploring factual LLM knowledge. In AAAI Demos. Yujia Hu, Tuan-Phong Nguyen, Shrestha Ghosh, and Simon Razniewski. 2025. Enabling LLM knowledge analysis via extensive materialization. In ACL. Yujia Hu, Tuan-Phong Nguyen, and Simon Razniewski. 2026b. GPTKB 2.0: Direct construction of disambiguated knowledge bases from large language models. Preprint, arXiv:2608.03729. Hadas Kotek, Rikker Dockum, and David Sun. 2023. Gender bias and stereotypes in large language models. In ACM CI. Sewon Min, Kalpesh Krishna, Xinxi Lyu, Mike Lewis, Wen-tau Yih, Pang Koh, Mohit Iyyer, Luke Zettlemoyer, and Hannaneh Hajishirzi. 2023. FActScore: Fine-grained atomic evaluation of factual precision in long form text generation. In EMNLP. Tuan-Phong Nguyen, Simon Razniewski, and Gerhard Weikum. 2024. Cultural commonsense knowledge for intercultural dialogues. In CIKM. Marinela Parović, Ze Li, and Jinhua Du. Generating domain-specific knowledge graphs from large language models. In Findings of ACL. Fabio Petroni, Tim Rocktäschel, Sebastian Riedel, Patrick Lewis, Anton Bakhtin, Yuxiang Wu, and Alexander Miller. 2019. Language models as knowledge bases? In EMNLP-IJCNLP.

7

Related documents

• GPTKB 2.0: Direct Construction of Disambiguated Knowledge Bases from Large Language Models#429465
• Stealing Reasoning Traces from Proprietary LLM APIs#441673
• Impossible to hide secret ...: Uncovering Security and Privacy Issues in LLM-native IDEs#411112
• Codifying the Judge: Scalable Evaluation via Program Distillation#1117526
• ExplAIner: A Declarative Query Language for Explaining Classification Models#346554