Curated AI beats frontier LLMs at pharma asset discovery Łukasz Kidziński
Kevin Thomas
Gosset Research May 7, 2026
Abstract
Verified drugs surfaced (across 10 queries)
arXiv:2605.04908v1 [cs.AI] 6 May 2026
General-purpose LLMs with web search are increasingly used to scout the competitive landscape of pharmaceutical pipelines. We benchmark Gosset — an AI platform with a chat interface backed by curated target-, modality-, and indication-level drug-asset annotations — against four frontier systems with web access (Claude Opus 4.7, GPT 5.5, Gemini 3.1 Pro, Perplexity sonar-pro) on ten niche oncology / immunology targets where most of the pipeline lives in the long tail of preclinical and Asian-developed assets. All five systems receive the same natural-language query and the same JSON output schema. Across 10 targets Gosset returns 3.2× more verified drugs per query than the best frontier system, at perfect precision and 100% recall against the cross-system union of verified drugs. The same curated index is exposed as a Gosset MCP server that any frontier model can call as a tool, suggesting that each of these systems can close most of the recall gap by swapping generic web search for a curated index behind the same chat interface.
500
Verified drugs surfaced by each system 451
400 300 200 100 0
Gosset
124
140
Claude + web
GPT + web
109
Gemini + web
76 Perplexity
Figure 1: Total verified drugs returned across the 10 niche-target queries (after alias-aware deduplication). Gosset surfaces every drug in the cross-system union; the next-best frontier system recovers under a third.
1
Why this matters
A pharma analyst asking "list all drugs targeting TL1A" wants three things from a tool: every real program (recall), no fabricated ones (precision), and an answer in seconds (latency). Frontier LLMs handle late-stage anchor drugs well — the few names that appear in press releases. But the typical target has a 10× longer preclinical-and-early-clinical tail of small biotechs, Chinese developers, and academic programs that web-indexed sources cover only sparsely; the recall problem is exacerbated by the well-documented hallucination tendency of generative models when asked for exhaustive named-entity lists [5], even under retrieval augmentation [6]. Curated indexes of clinical-stage assets and trial registries [7] are designed for exactly that tail. 1
Global precision and recall across 10 niche-target queries 1.0
1.00 1.00
1.00
0.99
1.00
0.97
Score
0.8 0.6
Precision Recall (vs cross-system union)
0.4
0.27
0.2 0.0
0.31
0.24
0.17
Gosset Claude + web GPT + web Gemini + web Perplexity
Figure 2: Global precision and recall across the 10 niche-target queries. Gosset matches the frontier LLMs on precision and clears them on recall by 3× or more. Recall is each system’s count of verified drugs over the cross-system union of 451 verified drugs (after alias-aware deduplication). The union is the discoverable universe — drugs traceable to sponsor websites, company materials, conferences, patents, papers, or press releases — not the absolute pipeline. Within that universe Gosset reaches 100% recall; programs that no source has surfaced (purely internal pre-IND research, undisclosed academic work) are out of scope for any of the five systems. What changes is how that index is exposed. Gosset wraps it in the same chat experience users already get from Claude, GPT, Gemini, and Perplexity — ask in natural language, get an answer in seconds — so the comparison below is between four AI platforms with the same front door, distinguished by what sits behind the chat. We test the question with a controlled head-to-head: same prompt, same judging rubric, same target list. Targets are chosen for diversity (immunology, oncology, mid-tier interest) and for having known long tails: TL1A, OX40L, IL-36R, TROP-2, B7-H3, ROR1, NaPi2b, Claudin 18.2, FAP, and GPRC5D.
2
Methodology
2.1
Systems under test
• Gosset — AI chat interface backed by Gosset’s curated drug-asset index. The user types a natural-language query; the system parses it into structured filters and queries the index directly. Sub-second; no live web access. Returns up to 200 rows sorted by phase (Approved → Phase 4 → . . . ) and most-recent industry trial. • Claude + web — claude-opus-4-7 [1] via the Anthropic API with the hosted web_search tool (20 search budget). • GPT + web — gpt-5.5-1 [2] via the Azure OpenAI Responses API with the web_search_preview tool (20 budget). • Gemini + web — gemini-3.1-pro-preview with native Google Search grounding [3] (20 budget). • Perplexity — sonar-pro [4] model with search_context_size="high". All five receive the same prompt asking for a JSON list of drugs with {name, sponsor, modality, phase, indication}. Frontier systems are free to use their web tools as they see fit; Gosset has no live web access. 2.2
Validation
Every verdict in this paper is signed off by a human expert reviewer with a pharma-pipeline background. The reviewer is the authoritative judge; the AI judges and deterministic auto-pass described below are scaffolding to 2
focus that human attention on the cases where it matters most. The pipeline runs in three layers: • Deterministic auto-pass. Drugs with curated industry-grade evidence (active clinical trials, FDA approvals, or sponsor commitments matching the queried target) are tagged verified without any LLM call. This clears the bulk of the data so reviewer attention is not spent on uncontroversial cases. • Three-AI-judge cross-check. Everything that survives the auto-pass is independently graded by Claude Opus 4.7, GPT 5.5, and Gemini 3.1 Pro, each with web search, in the LLM-as-a-judge pattern [9]. A 2-of-3 majority produces a preliminary verdict of verified, hallucinated, or unsure (ties resolve to unsure). The disagreements and unsure cases are precisely where the human reviewer focuses next. • Human expert sign-off. The reviewer audits the flagged residual — canonical-name collisions where two distinct molecules share a developer code (e.g. IMB101, SAR446309) and AI judges’ web search surfaces the other molecule; TNF-superfamily pathway annotations where binding evidence is indirect; wrong-target attributions (e.g. rocatinlimab mis-cited for OX40L when it binds the OX40 receptor) — and reclassifies as needed. All numbers in this paper reflect the post-review state. The role of the AI judges is to triage, not to decide. Where they agree with each other and the auto-pass, the reviewer spot-checks; where they disagree, the reviewer adjudicates from primary sources. To prevent the recall proxy from overcounting systems whose output style packs multiple aliases into a single string ("Tecotabart vedotin (LM-302, TPX-4589, BMS-986476)" — one molecule, four surface names), every system returns an explicit aliases list in its JSON schema and we apply alias-aware union-find at scoring time: two drugs are the same if their alias sets share any non-trivial member. Per-system metrics. Let Vs be system s’s verified-drug count and Hs its hallucinated-drug count, both as confirmed by the human reviewer. • Precision = Vs /(Vs + Hs ). S • Recall (proxy) = |Vs |/| systems V |. The union of verified drugs across all five systems is the best groundtruth proxy short of curating each target’s pipeline manually. • Hallucination rate = Hs /(Vs + Hs ). • Latency — per-query wall-clock.
3
Results System Gosset GPT + web Claude + web Gemini + web Perplexity
Verified
Hallucinated
Precision
Recall (proxy)
451 140 124 109 76
0 0 1 0 2
1.000 1.000 0.992 1.000 0.974
1.000 0.310 0.275 0.242 0.169
Table 1: Aggregate counts across all 10 queries after the human-reviewed validation pipeline (Section 2.2). Gosset surfaces 3.2× more verified drugs than the best frontier system at perfect precision. Recall is global: each system’s verified-drug count over the cross-system union of 451 verified drugs (after alias-aware deduplication).
3.1
Findings
1. Long-tail recall. Gosset returns 451 verified programs across the 10 queries — 3.2× the next-best system (GPT + web at 140). The recall gap widens for targets with >10 preclinical assets (e.g. TL1A, B7-H3, NaPi2b, Claudin 18.2) where most of the pipeline lives in early-stage Asian-developed and academic programs that frontier-LLM web indexes cover thinly. 2. Late-stage parity. On the small subset of approved / Phase 3 anchors, all five systems agree — frontier LLMs do recover what’s been heavily covered in the press.
3
True-positive drugs (judged)
Coverage per query
verified drugs returned by each system
70 60 Gosset Claude + web GPT + web Gemini + web Perplexity
50 40 30 20 10 0
A
TL1
0 OX4
L
r pto
ece 36 r
IL-
P-2
TRO
H3
i2b 18.2 n i NaP d Clau
1 ROR
B7-
FAP
C5D GPR
Hallucinated drugs (false-positives)
Figure 3: Verified true-positive drug count returned by each system per target. Gosset returns the most verified drugs on most targets, driven by long-tail preclinical coverage that frontier systems miss.
Hallucinations per query
1.0
claims judged not real / off-target
0.8 Gosset Claude + web GPT + web Gemini + web Perplexity
0.6 0.4 0.2 0.0
A
TL1
0 OX4
L
r pto
IL-
ece 36 r
P-2
TRO
H3
B7-
i2b 8.2 in 1 NaP d u Cla
1 ROR
FAP
C5D GPR
Figure 4: Per-target false-positive count. Frontier systems occasionally fabricate drug names or assign unrelated assets to a target; Gosset’s structured filters cannot hallucinate names.
Per-query median latency Gosset 1.2s Claude + web
103.1s
GPT + web
804.8s
Gemini + web
106.8s
Perplexity
15.3s
0
200
400 600 Median wall-clock (seconds)
800
Figure 5: Median per-query wall-clock. Gosset answers in roughly the time of one Mongo round-trip; frontier systems pay the cost of multiple web searches plus generation. 3. Hallucinations are rare in the frontier LLMs. After validation, Gosset, GPT + web, and Gemini + web all reach P = 1.000. Claude + web has 1 hallucinated drug (P = 0.992) and Perplexity has 2 (P = 0.975) — small absolute counts, but the separation in recall (5–7×) is the dominant axis.
4
4. Latency. Gosset is roughly two orders of magnitude faster than the web-augmented LLMs, which is the difference between an interactive workflow and a wait-and-see one. 3.2
Closing the gap with Gosset MCP
The recall gap is not a model-quality problem; it is an index problem. The frontier systems we benchmarked are strong reasoners, but they are reading the open web, and the open web does not catalog the long tail. Gosset exposes its curated asset index as an MCP [8] server, which any of these models can call as a tool — the same way they already call web search. In that configuration the LLM keeps its strengths (natural-language understanding, summarization, multi-step reasoning, audience-appropriate output) and offloads enumeration to a system designed for it. We expect each of the four frontier models to close most of the recall gap when wired to Gosset MCP, without any change to the model itself: the same prompt, routed through a curated index instead of generic web search, lifts verified-drug counts toward Gosset’s own headline numbers. A follow-up paper will benchmark each frontier system in both configurations.
4
Limitations
The headline 100% recall is a statement about the discoverable universe of drugs, not the absolute pipeline. Our cross-system union is built from what Gosset, Claude, GPT, Gemini, and Perplexity can surface — which in turn traces back to sponsor websites, company materials, conferences, patents, peer-reviewed papers, and press releases. Programs that have never appeared in any of those channels (purely internal pre-IND research, undisclosed academic work, programs behind a Chinese sponsor’s firewall) are invisible to every system here, and no benchmark of this shape can count them. The right reading is that Gosset achieves perfect recall within the publicly traceable universe; absolute-pipeline recall remains unmeasurable. The judges themselves are LLMs and so inherit some calibration error; we mark unsure verdicts neutrally and human-review the residual borderline cases. Target selection biases toward areas where Gosset is well populated; we expect smaller margins on widely-covered targets such as PD-1 or HER2 where the press has saturated frontier-LLM training data.
References [1] Anthropic. Claude Opus 4.7. https://www.anthropic.com/news/claude-opus-4-7, 2026. [2] OpenAI. GPT-5.5 system card. https://openai.com/research/gpt-5-5, 2026. [3] Google DeepMind. Gemini 3.1 Pro: technologies/gemini/, 2026.
native search-grounded reasoning. https://deepmind.google/
[4] Perplexity AI. Sonar Pro: a real-time web search model. https://docs.perplexity.ai/docs/model-cards, 2026. [5] Z. Ji, N. Lee, R. Frieske, et al. Survey of hallucination in natural language generation. ACM Computing Surveys, 55(12):1–38, 2023. [6] P. Lewis, E. Perez, A. Piktus, et al. Retrieval-augmented generation for knowledge-intensive NLP tasks. NeurIPS, 2020. [7] U.S. National Library of Medicine. ClinicalTrials.gov. https://clinicaltrials.gov/, accessed 2026. [8] Anthropic. Introducing the model-context-protocol, 2024.
Model
Context
Protocol.
https://www.anthropic.com/news/
[9] L. Zheng, W.-L. Chiang, Y. Sheng, et al. Judging LLM-as-a-judge with MT-Bench and Chatbot Arena. NeurIPS Datasets and Benchmarks, 2023.
5