VISTA Architect: A graph database-oriented health AI system demonstrated in multidisciplinary tumor boards

Unknown

Conceptio › Archive › arXiv CS

arXiv CSopen access

VISTA Architect: A graph database-oriented health AI system demonstrated in multidisciplinary tumor boards

Unknown · 2026 · arxiv_cs

arXiv CS · Papers · License: Open Access · 2026

Open Source ↗Direct PDF ↓

databases datamanagement sql storage

databases, sql, data management, storage

VISTA Architect: A graph database-oriented health AI system demonstrated in multidisciplinary tumor boards Tuomo Kiiskinen1,∗ , Jason Fries1 , Philip Adamson1 , David Wu2 , Timothy John Ellis-Caleo2 , Aaron Fanous1 , Balasubramanian Narasimhan1 , Joel Neal2 , Sylvia Plevritis1 , Manuel A. Rivas1,∗

arXiv:2606.22692v1 [cs.AI] 21 Jun 2026

1 Department of Biomedical Data Science, Stanford University School of Medicine

2 Department of Medicine, Stanford University School of Medicine ∗ Corresponding authors: [email protected], [email protected]

Abstract We introduce VISTA Architect, a database-oriented AI architecture for integrating large language models (LLMs) with longitudinal electronic health records (EHRs). At ingestion, it transforms complex clinical documentation into a persistent, provenance-linked knowledge graph, eliminating repeated reprocessing of raw records at query time. The architecture has two layers: a source-faithful MEDS Graph preserving granular EHR structure with full provenance, and a clinically abstracted Timeline Object Architecture (TOA) that uses graph-guided LLM extraction to synthesize a concise timeline of deduplicated, temporally coherent clinical events. This addresses key limitations of direct long-context prompting and retrieval-augmented generation (RAG), which often miss temporal relationships and incur high cost and latency from repeated raw-text processing. By precomputing clinical synthesis once, downstream queries access an organized patient state and traverse to source documentation only when detailed verification is needed. We demonstrate the system in multidisciplinary thoracic oncology tumor boards at Stanford Medicine, where precise reconstruction of patient histories is critical. Across 1,180 patients, VISTA Architect achieved 96.4% accuracy (mean 9.75/10) on 15 tumor board–salient variables (17,700 evaluations; 95% CI 96.1– 96.7%), surpassing a matched BM25 RAG baseline and recent benchmarks for LLM-based clinical extraction. An agentic interface reduced preparation for a 30-patient held-out cohort to about 2.2 minutes without sacrificing accuracy. While configured here for thoracic oncology, the modular design adapts to other specialties through customizable event definitions, episode structures, and agentic tools; validation beyond thoracic oncology remains future work.

Introduction

Multidisciplinary tumor boards exemplify these challenges.[4, 5, 6] These real-time care conferences require rapid and precise synthesis of complex Clinicians routinely rely on electronic health records longitudinal patient trajectories—from initial diag(EHR) to reconstruct complex patient narratives that nosis through treatments, responses, and disease guide critical clinical decisions. However, EHR data progression—to inform critical medical decisionare primarily organized for documentation rather making without direct patient interaction. Thoracic than for clinical reasoning, fragmenting patient traoncology tumor boards, for example, rely heavily on jectories across isolated records such as progress accurate temporal narratives, including distinctions notes, laboratory results, and imaging reports.[1] between baseline disease, treatment effects, emergThe burden of manually integrating these fraging resistance mutations, and toxicity-related commented data into coherent clinical narratives falls plications. Currently, preparing a single patient case heavily on clinicians, who often spend considerable for tumor board discussion demands approximately time—up to 4 hours daily—on chart review alone,[2] 15–30 minutes per clinician and suffers from signifiexacerbating burnout and increasing the risk of cogcant inter-observer variability, potentially impacting nitive errors.[1, 3] 1

the consistency and quality of clinical decisions.[4, 5] domain-specific synthesis problem: events must be temporally aligned, repeated documentation must Traditional computational approaches attempting to be reconciled, and clinically meaningful phases must address this challenge face a fundamental tension be inferred. VISTA Architect addresses this by makbetween the scale of longitudinal patient records ing longitudinal synthesis explicit and amortized and the capacity of current AI methods. For indiat ingestion rather than re-derived for each downvidual clinical documents, large language models stream application.[16] (LLMs) have demonstrated strong summarization capabilities, in some cases matching or exceeding Here we introduce VISTA Architect, a databasemedical experts.[7, 8] However, tumor board prepa- oriented AI system that addresses a general chalration requires synthesis over longitudinal documen- lenge in applying LLMs to large document repositotation that may span years of care, thousands of ries: how to provide accurate, structured, and comclinical events, and very large text volumes. This plete context from records that far exceed model cascale often exceeds the practical context available pacity, without sacrificing retrieval speed or tempoto current LLM workflows, particularly when the ral fidelity.[17] Our approach employs a two-tiered same patient record must be queried repeatedly graph architecture that pre-computes a structured across multiple clinical questions. When applied representation of the available longitudinal clinical to such longitudinal records, LLM performance de- record. The first tier, a hierarchical MEDS Graph, grades: temporal reasoning deteriorates as input preserves the granular documentation structure of length increases,[9] and hallucinations and omis- the source data. The second tier, a clinically fosions become clinically consequential.[10] Retrieval- cused Timeline Object Architecture (TOA), synthesizes augmented generation (RAG) offers a strategy for these into deduplicated, temporally ordered events navigating records beyond context length,[11] but and episodes—producing a queryable knowledge retrieves fragments by semantic similarity rather graph that any downstream application can traverse than temporal or clinical structure, producing dis- deterministically. Because the clinically organized connected context that may miss critical longitu- graph is constructed once and queried repeatedly, dinal relationships. Dense retrieval models fre- the expensive part of the workflow—longitudinal quently underperform simple lexical baselines in synthesis into events, episodes, and current-state specialized domains,[12] and even optimized clini- artifacts—is amortized. At query time, applications cal RAG pipelines require substantially more com- operate over this precomputed clinical context and putation per query while achieving suboptimal ac- traverse provenance links only when targeted source curacy relative to structured approaches.[13] Em- evidence is needed, rather than repeatedly searchpirically, reported accuracy levels for LLM-based ing or rereading the raw record. The contribution clinical information extraction reflect these limita- is therefore not that a graph store categorically retions: a recent meta-analysis of 56 oncology studies places SQL, but that VISTA encodes a clinically reported an average overall accuracy of 76.2% (di- meaningful, provenance-linked hierarchical tree-like agnostic accuracy 67.4%),[14] agentic multi-agent compression of the longitudinal record, allowing orchestration achieved 84% strict recall on tumor downstream agents to navigate from patient state board–salient facts across 71 patients,[15] and opti- to targeted source evidence rather than reconstructmized clinical RAG pipelines for note-level extrac- ing clinical context from raw records at query time. tion reported F1 scores of 0.79–0.90 depending on On top of this persistent graph layer, every interretrieval strategy.[13] These results were typically ob- action with VISTA Architect—patient build, dashtained on note-level or short-document tasks; whole- board population, chat, and cross-patient retrieval— record longitudinal extraction is harder still. Rela- is served by an agentic interface in which the retional databases can support deterministic queries ceiving component plans its own graph traversals and provenance, but reconstructing a patient-level and parallel sub-agent invocations, supporting both clinical state from normalized EHR tables remains a on-demand cohort preparation and conversational 2

clinical reasoning from a single shared graph. We demonstrate this architecture in the context of electronic health records (EHR) and multidisciplinary tumor boards, where the demands on accuracy, temporal precision, and speed are particularly acute.

processing methodologies such as direct LLM prompting, retrieval-augmented generation (RAG) over raw EHR text, and programmatic deterministic queries over relational EHR schemas, by positioning graph databases at the core of clinical data representation.[17] The architecture is organized into three interconnected layers: (1) a structured multilayered graph database derived directly from electronic health record (EHR) data; (2) an AI bridge that manages complex query translation and data retrieval; and (3) a generative and agentic user interface that dynamically generates display items directly from the TOA data layer.

At our own institution, a manual LLM-based pretumor-board summarization workflow has been developed, evaluated against physician-authored references, and deployed into routine thoracic tumor board practice.[18] VISTA Architect builds on this operational foundation: it pre-computes a complete structured patient timeline, from which the same clinician-facing summary of critical information is retrieved automatically rather than summarized from The graph database is constructed in three sequential a fixed window of recent notes for each query. steps during patient ingestion, each producing a graph-resident layer that downstream applications We validate the VISTA Architect system in the can query deterministically. The two subsections highly demanding context of thoracic oncology tuthat follow describe how these layers are assembled mor boards at Stanford Medicine. Across the full into the runtime context for an agentic AI bridge, cohort of 1,180 patients, VISTA Architect achieved and how that bridge is exposed through the user96.4% accuracy (mean score 9.75/10; 95% CI 96.1– facing applications. 96.7%) on 15 tumor board–salient clinical variables, with 17,063 of 17,700 variable evaluations rated as Correct. Automated case preparation completed 2.1.1 Step 1: Turning the EHR into a graph in 2.2 minutes for a held-out 30-patient cohort undatabase der the fully agentic build, matching and extending recent efforts exploring agentic LLM orchesFirst, VISTA Architect transforms the source health tration for patient summarization in tumor-board record into a persistent graph database. In the imworkflows.[15] plementation evaluated here, the source data were Through its structured yet flexible design, VISTA represented as MEDS XML, a hierarchical representaArchitect offers a model for connecting large doc- tion derived from the Medical Event Data Standard ument repositories to LLM-powered applications. (MEDS).[19] However, the architecture does not deThe architecture is adaptable via specialty-specific pend on MEDS XML specifically. The same graph configurations—defining context-appropriate node layer can be constructed from other structured, semitypes, episode structures, and variable dictionaries— structured, or document-based clinical sources, promaking it applicable beyond thoracic oncology to vided that the source data can be represented as other clinical specialties and, more broadly, to any nodes, edges, timestamps, and provenance links. domain where large, longitudinal document collecThis first graph layer, the MEDS Graph, represents tions must be queried with high accuracy and comthe EHR as a graph. It preserves the granular plete provenance. structure of the source record: patients, encounters, notes, measurements, procedures, medications, 2 Results imaging records, and other clinical entries become graph nodes, while temporal, visit-level, and source2.1 Architecture document relationships become graph edges. The We designed VISTA Architect using a database- goal of this layer is not to summarize the record, but oriented AI framework, diverging from text- to make the original EHR computationally address3

Figure 1: VISTA Architect high-level system design. VISTA Architect converts source EHR data into a persistent graph database, consisting of a source-faithful MEDS Graph and a clinically abstracted TOA Graph. The agentic AI bridge operates over TOA events, episodes, current-state artifacts, and targeted MEDS Graph retrieval through tool calls. The user interface exposes the same graph-resident artifacts through display items, natural-language interaction, and provenance-linked EHR access.

able.

is given in Appendix C).

This step is deterministic and does not require an LLM. Because the MEDS Graph maintains links back 2.1.2 Step 2: Timeline Object Architecture (TOA) to the original source documents, later AI-generated assertions can be traced to the underlying EHR ev- Next, VISTA Architect constructs a compact cliniidence. The graph can also link to multimodal as- cal timeline graph on top of the source-level graph. sets such as imaging studies, audio recordings, and We refer to this layer as the Timeline Object Archiwaveform data without storing them directly, keep- tecture (TOA) Graph. Whereas the MEDS Graph ing the graph lightweight to operate. The graph represents the EHR as documented, the TOA Graph database therefore becomes the primary data struc- represents the patient trajectory as a coherent timeture of the system: the LLM is not asked to re- line of clinically meaningful events; both graph laypeatedly reread the full record from scratch, but ers are shown for a single representative patient in instead operates over a structured representation Supplementary Fig. S6. of the record. In our thoracic oncology cohort, perpatient MEDS Graphs contained a median of 3,608 The central operation in this step is transforming nodes (IQR 562–11,629; the full MEDS Graph schema EHR data from documents into clinical events. For example, a diagnosis, surgery, imaging result, molec4

ular test, treatment start, adverse event, or change in clinical status may be mentioned across multiple notes. In the TOA layer, these mentions are resolved into a single event object linked back to the source evidence in the MEDS Graph. This converts the longitudinal record from a documentation-centered graph into a decision-centered clinical timeline.

makes complex longitudinal records easier for humans to understand by organizing long sequences of events into clinically meaningful phases. Second, it is itself a precomputed reasoning step that aids later AI operations over the graph. By storing phase boundaries, transition points, treatment contexts, and termination reasons, the system gives downstream agents a structured map of the timeline rather than only a flat sequence of events. Later graph operations can therefore start from a clinically organized representation and traverse into finer-grained evidence only when needed.

This step uses graph-guided LLM extraction. Rather than prompting an LLM with an entire raw EHR, the system traverses the MEDS Graph, splits the EHR into compact graph-derived segments, presents those segments to the model, and asks it to produce standardized, structured clinical event objects. The The same ingestion process also produces currentLLM therefore performs a bounded abstraction task state graph-resident artifacts, such as structured variover graph-derived context, while the graph pre- able snapshots, summaries, and display-ready fields. serves provenance and temporal structure. A key These artifacts represent the system’s latest synthedistinction is that timestamps in the TOA Graph rep- sized view of the patient at the relevant decision resent the inferred time of the actual clinical occur- point. They are not merely UI elements; they are rence, whereas timestamps in the source EHR typi- stored outputs of prior reasoning over the graph. cally reflect when that information was documented. Dashboards, agents, documentation tools, cohort In some cases these dates are effectively identical; in queries, and cross-patient retrieval can reuse these others, the difference may span months, years, or artifacts without reconstructing the clinical context even decades when historical events are recorded from the raw EHR for every interaction. retrospectively. The TOA Graph therefore represents the patient’s actual biomedical and clinical course rather than the order in which facts happened to 2.1.4 Connecting the graph to an agentic AI bridge appear in notes. The event-extraction and episodesynthesis prompts used in the thoracic oncology con- At runtime, the AI bridge operates with the TOA figuration are reproduced in Appendices E.1 and E.2. layer, episode structure, current-state artifacts, and a small recent EHR context available to the model. This gives the agent a compact but clinically orga2.1.3 Step 3: Forming episodes and current-state nized view of the patient before any additional reartifacts trieval is needed. User or application requests can therefore often be answered directly from the graphFinally, the TOA graph is expanded by generating resident clinical representation. higher-order clinical structures from the timeline obWhen additional detail is required, the agent uses ject event stream. Individual TOA events provide a the TOA layer to navigate the lower-level MEDS detailed timeline, but episode formation compresses Graph and retrieve targeted pieces of the original that stream into a compact clinical story: meaningEHR. For example, the TOA Graph may identify ful phases with clear transitions, anchors, and reathe relevant clinical event or episode, after which sons for change. In the thoracic oncology configuthe agent can traverse provenance edges to recover ration, these episodes include baseline history, diagthe exact source note, imaging report, laboratory nostic workup, treatment lines, and post-oncological value, or document fragment. Retrieval is therefore phases, but the episode vocabulary is configurable guided by the clinical abstraction layer rather than for other domains. performed as an unguided search over raw text. In Episode formation serves two purposes. First, it this sense, graph traversal is used as a clinical navi5

gation operation: the agent starts from TOA events, boundary of the architecture. The architecture can episodes, or current-state artifacts and follows clini- be adapted to other clinical domains through concally meaningful edges to the relevant source-level figurable event types, episode structures, display nodes, rather than issuing a broad search over the artifacts, edge types, and agentic tool capabilities. full patient record. What is shared is the architectural core: construct a source-faithful EHR graph, abstract it into a clinically This design changes the role of retrieval. Convenmeaningful TOA graph, store reusable reasoning artional RAG systems retrieve fragments from raw tifacts, and expose the result through an agentic AI documents at query time. VISTA Architect instead bridge and user interface. performs much of the semantic and temporal organization once, stores it as graph structure, and then uses that structure to retrieve only the targeted 2.2 Application to Thoracic Tumor Boards source evidence needed for a given task. The AI 2.2.1 Clinical Context bridge uses natural language, tool calls, and agentic orchestration, but the durable context remains the Multidisciplinary tumor boards require clinicians graph: TOA for clinical navigation, MEDS Graph for to rapidly synthesize comprehensive patient histosource-level detail, and original EHR documents for ries from fragmented and complex electronic health auditability. record (EHR) data.[4, 5, 6] These real-time care conferences integrate expertise from radiologists, pathologists, medical oncologists, surgeons, and geneticists to develop precise, patient-specific treatment strategies. Accurate and temporally precise reconstruction of clinical events—including diagnostic evaluations, therapeutic interventions, molecular testing, and toxicity management—is essential to informed clinical decision-making in oncology, particularly in thoracic malignancies where patients frequently experience multiple treatment stages and complex care trajectories.

2.1.5 Exposing the architecture through userfacing applications

The final step is the user-facing application layer. Users see many of the same display artifacts that the AI bridge holds in context: structured patient fields, timeline summaries, episode-level views, and current-state summaries. This makes the interface and the AI bridge operate over the same underlying representation rather than over separate data products. In the thoracic oncology configuration, this layer exposes Overview, Clinical Timeline, au2.2.2 Implementation Details tomated tumor board note, and chat views, all rendered from graph-resident artifacts (Supplementary We implemented VISTA Architect specifically within Figs. S2, S3, S4, and S5). thoracic oncology tumor boards, leveraging its flexFrom the user perspective, this enables both direct ible architecture through configurable profiles and review and interactive extension. A user can inspect specialized prompts. These profiles define how display items, ask questions about them through the MEDS Graph and Timeline Object Architecture the AI bridge, and, when needed, the agent can per- (TOA) layers process and structure clinical data form additional graph operations. These operations specifically for thoracic oncology decision-making. can be purely retrieval-oriented, such as finding the For thoracic oncology tumor boards, the profile consource evidence for a clinical assertion, or generative, figuration involved defining clinically meaningful such as drafting a note, producing a summary, createpisodes and events tailored explicitly for oncoling a plot, or assembling a domain-specific view. ogy scenarios. Episodes included baseline assessHere we demonstrate VISTA Architect in multidis- ments, diagnostic evaluations, distinct numbered ciplinary thoracic tumor boards. The tumor-board treatment lines, and post-treatment surveillance. application is a configuration of the system, not the Imaging events such as CT, MRI, and PET scans 6

Figure 2: Synthetic example of TOA-based timeline organization. VISTA Architect maps source-level imaging events from the EHR into graph-resident clinical episodes. The central timeline shows how granular events contribute to higher-order episode summaries, preserving temporal structure and provenance while creating a compact representation for display, agentic retrieval, and source-evidence navigation.

served as anchors for defining these episodes, with graph at ingestion, rather than re-deriving them events grouped around these anchors within clini- from per-query note retrieval. cally meaningful temporal windows (±90 days overall, ±7 days for co-occurrence). 2.3 Clinical Accuracy Additionally, we established a focused variable dictionary capturing the most clinically relevant patient information frequently discussed during tumor board meetings. These variables included demographic details (date of birth, sex, smoking status), tumor-specific factors (diagnosis, histology, metastasis, lymph node involvement, and genetic testing panel), treatment information (previous surgeries, current medical therapy, radiation therapy, last imaging dates), clinical status (ECOG performance status, therapy toxicity/comorbidities), and safety-critical details (allergies). The full dictionary, with variable types, is provided in Supplementary Table S2.

The 1,180 patients in this study were those in the Data Lake who presented at thoracic multidisciplinary tumor boards between (de-identified) January 2020 and November 2025 (Table 1). The cohort was predominantly NSCLC (82.0%), with representation of thymoma/thymic tumors (8.0%), SCLC (2.9%), mesothelioma (2.9%), and other thoracic malignancies (4.2%). Molecular profiling results were documented in 67.3% of patient records, with EGFR (17.8%), KRAS (8.2%), and ALK (4.1%) the most prevalent actionable drivers across the full cohort. Each patient contributed an EHR data export, converted to MEDS XML (Section 2.1), with a median The variable dictionary and the pre-tumor-board record time span of 8.5 years (IQR 4.4–12.7), a mesummary note were not defined de novo: they fol- dian of 3,255 clinical events (IQR 362–10,922), and low the tumor-board–salient fields and the concise a median of 3,608 MEDS Graph nodes (IQR 562– summary format already in operational use at the 11,629), encompassing demographics, oncologic proStanford thoracic tumor board.[18] VISTA Architect cedures, and longitudinal clinical text. From each paproduces these same artifacts from the precomputed tient, we extracted 15 multidisciplinary tumor board (MTB)-salient variables spanning demographics, tu7

mor characteristics, treatment history, clinical status, challenging variable was Date of Last CT (9.52/10, imaging, and safety (Table 3), yielding 17,700 total 92.1%; 95% CI 90.4–93.5%), reflecting ambiguity bevariable evaluations. tween internal and external imaging dates. We evaluated VISTA Architect across the full 1,180patient thoracic oncology cohort using 15 tumor board–salient variables per patient (17,700 total evaluations). Ground truth was established from raw EHR XML truncated at the documented tumor board date, ensuring evaluation reflects only information available at the clinical decision point. The production pipeline used GPT-4.1 for chunk-level event extraction and GPT-5 for downstream patient-info synthesis; an independent GPT-5 LLM-as-judge (Appendix E.5) scored each extracted variable against the ground truth, with full evaluation protocol and clinician validation described in Methods.

Independent clinician validation of the LLM-asjudge was performed on a randomly selected 30patient subset from the full 1,180-patient cohort. The original clinician review covered 480 paired evaluations from the initial 16-variable set; the primary endpoint reported here excludes DNR/code status, yielding 450 endpoint evaluations in the same 30 patients. The subset was subsequently checked to ensure that it was representative of the broader cohort rather than an outlier sample. Clinician review classified each of the 480 paired judge decisions as Agree, Uncertain, or Disagree: 95.4% were rated Agree (95% CI 93.2–97.0%), 4.6% Uncertain, and 0% Disagree (zero of 480; 95% CI upper bound 0.79%). The Uncertain cases clustered on variables with inherent clinical ambiguity. The judge correctly identified all six VISTA extraction failures in the sample (100% sensitivity), grading each as Partial or Incorrect.

Overall accuracy was high, achieving an approximate mean quality score of 9.75/10 with 17,063/17,700 (96.4%; 95% CI 96.1–96.7%) variables rated as Correct (Table 2).1 As a sensitivity analysis addressing the concern that two of the 15 reported variables (Date of Birth and Sex) are reliably available from structured OMOP fields and serve here as positive controls for upstream hallucination (Section 4.2), the same headline excluding those two positive controls is 9.71/10 mean (95.85% correct; 14,703/15,340 evaluations across 13 variables, Wilson 95% CI 95.52–96.15%).

On the same 30-patient cohort, we additionally evaluated the fully agentic extraction pipeline (Section 2.1.4), in which chunk-level extraction is parallelized within and across patients and coordinated by an agent. This agentic build also used a different underlying model family from the sequential pipeline—Claude Opus 4.6 as orchestrator, with Gemini 3.5 Flash for per-chunk event extraction and episode synthesis (Appendix H)—so the agreement reported here additionally indicates that extraction quality is not tied to a single model provider. Extraction accuracy remained consistent with the sequential pipeline reported above: the GPT-5 judge scored the two pipelines within confidence-interval overlap on the 16-variable rubric, with no systematic pervariable regression. The wall-time gains achieved under this configuration are reported in Section 2.5.

Performance was consistent across all six clinical categories (Table 3). Demographics achieved the highest accuracy: Date of Birth and Sex were both extracted perfectly (10.00/10, 100% correct across all 1,180 patients; 95% CI 99.7–100%). Safety- and treatment-related variables were extracted with high reliability, including Radiation Therapy at 9.94/10 (99.1%; 95% CI 98.3–99.5%) and Allergies at 9.70/10 (94.8%; 95% CI 93.4–96.0%). Tumor characterization variables were reliably extracted, with Histology at 9.89/10 (97.8%), Genetic Testing Panel at 9.75/10 (95.4%), and both Metastasis and Lymph Node In- Error analysis across the 637 incorrect extractions volvement at 9.68–9.69/10 (94.8–95.0%). The most (3.6%) revealed systematic patterns rather than ran1 DNR / code status, evaluated in the original 16-variable dom failures. The most common error source was schema and excluded from the primary endpoint because it is imaging chronology (Date of Last CT), arising from deterministically available from OMOP-coded fields, was ex- ambiguity between internal and external imaging tracted at mean 9.92/10 and 99.5% correct (1,174/1,180; Wilson dates or PET-CT versus standalone CT attribution. 95% CI 98.9–99.8%).

Table 1: Cohort characteristics (N=1,180), summarized from VISTA-generated structured artifacts. Values combine source-structured fields and text-derived artifacts produced during pipeline processing; this descriptive table is not intended as a standalone benchmark of LLM extraction from unstructured text. Per-gene rows report positive results; patients with multiple actionable drivers are counted in more than one row, so the per-gene counts sum to more than the 613 patients with any actionable driver. Characteristic

Value

Patient Demographics Age at tumor board, median (IQR), y Male Female Clinical Characteristics Smoking status Current / Former / Never Unknown Pack-years, median (IQR) ECOG performance status 0 / 1 / 2 / 3–4 Unknown

69 (61–76) 547 (46.4%) 633 (53.6%)

74 / 470 / 575 61 (5.2%) 30 (12–45) 254 / 379 / 76 / 29 343 (29.1%)

Tumor Characteristics Primary diagnosis NSCLC Thymoma/thymic tumors SCLC Mesothelioma Other thoracic malignancies Metastatic disease Yes / No Unknown or suspected Lymph node involvement Yes / No Unknown or suspected

968 (82.0%) 94 (8.0%) 34 (2.9%) 34 (2.9%) 50 (4.2%) 523 / 500 157 (13.3%) 572 / 437 171 (14.5%)

Molecular Testing Results documented Any actionable driver mutation EGFR / KRAS / ALK MET / HER2 / BRAF ROS1 / RET / NTRK PD-L1 results documented

794 (67.3%) 613 (51.9%) 210 / 97 / 48 35 / 29 / 26 19 / 19 / 5 671 (56.9%)

Treatment History Current systemic anticancer therapy History of radiation therapy

385 (32.6%) 389 (33.0%)

Safety Known drug allergies

545 (46.2%)

Previous Surgery errors stemmed from boundary cases in surgical procedure classification (e.g., diagnostic VATS biopsy vs. oncologic resection). Allergies and Current Medical Therapy errors reflected documentation fragmentation across the longitudi-

nal record. Metastasis errors arose from ambiguous staging scenarios where imaging was “suspicious but not confirmed.” No errors occurred in Date of Birth or Sex across all 1,180 patients. Worked examples of a perfect extraction and of a representative

Metric

Value

Patients evaluated Variables per patient Total evaluations Mean quality score Correct Incorrect

1,180 15 17,700 9.75/10 17,063/17,700 (96.4%; 95% CI 96.1–96.7%) 637/17,700 (3.6%)

Table 2: Aggregate extraction accuracy across 15 MTB-salient variables for 1,180 patients after excluding DNR/code status from the primary endpoint. Variable

Category

VISTA

RAG (GPT-4.1)

RAG (GPT-5)

Demographics Tumor Clinical Treatment Imaging Safety

9.96 (100.0%) 9.72 (96.0%) 9.65 (96.7%) 9.72 (96.7%) 9.77 (96.7%) 9.67 (93.3%)

8.26 (81.1%) 7.97 (70.0%) 6.10 (46.7%) 7.26 (60.0%) 7.17 (66.7%) 8.00 (66.7%)

8.34 (80.0%) 8.04 (70.0%) 7.92 (68.3%) 6.83 (52.2%) 6.93 (63.3%) 7.43 (56.7%)

Overall

9.76 (96.9%)

7.58 (66.7%)

7.73 (66.9%)

Table 4: VISTA Architect vs. RAG baseline accuracy by clinical category after excluding DNR/code status from the primary endpoint (N=30 patients, 450 evaluations per system). RAG uses BM25 retrieval over raw EHR XML with focused per-variable queries—the configuration most favorable to lexical retrieval. Full per-variable results in Appendix G.

Figure 3: Single-patient MEDS Graph construction performance for VISTA Architect. Build time and peak memory usage are shown for the programmatic XML-to-graph conversion step for representative small (<1 MB XML, ∼500 MEDS Graph nodes), mid-sized (∼2 MB XML, ∼2,200 nodes), and large (∼5 MB XML, ∼10,000 nodes) patients. All MEDS Graphs are constructed in under one second and under 20 MB of memory, showing approximately linear scaling with patient record size. Note that the subsequent LLM-based TOA event extraction step requires additional time; per-patient end-to-end build times under the fully agentic pipeline are reported in Section 2.5 (mean 88 s, median 87 s, max 131 s on the test30 cohort).

Regime

Wall Time

Speedup

Sequential (one patient at a time, sequential chunks) Across-patient (10 patients in parallel, sequential chunks within patient) Fully agentic (intra- and inter-patient parallel, measured)

74 min 10 min 2.2 min

1.0× 7× 34×

Table 5: Parallelization impact for the 30-patient test cohort. All rows report measured wall time. The first two rows are the sequential pipeline under single-patient and batched-across-patient regimes; the bottom row is the fully agentic build (Section 2.1.4) on the held-out test30 cohort, including all chunk extraction, unification, episode synthesis, and display generation stages.

the pre-computed graph artifacts is already a work- illustration that the architecture readily supports ing retrieval system—not a final product, but an cross-patient analyses without re-touching the raw

EHR. This capability operationalizes the emerging on treatment line number, therapeutic intent (cuparadigm of smart patient retrieval—AI systems that rative vs. palliative), regimen drug class overlap surface similar prior cases with known outcomes to (14 classes: platinum doublet, immunotherapy, provide contextual evidence on disease trajectories, EGFR TKI, etc.), and prior surgery/radiation statreatment responses, and trial opportunities during tus. tumor board deliberation—offering a concrete start3. Narrative component: BM25 ranking on each paing point for the kind of retrieval recently advocated tient’s pre-tumor-board summary note. This note for precision oncology.[20] is generated by VISTA Architect at patient ingesThe module is not designed to maximize a single notion following the standard Stanford pre-tumortion of overall similarity. Different clinical questions board summary procedure—a concise clinical privilege different facets of a patient’s record: a quessummary of the same format presented at current tion about expected toxicity from a novel EGFR-MET Stanford tumor board meetings—and is stored combination requires patients matched on driver bias a graph-resident artifact alongside the strucology; a question about whether to attempt salvage tured TOA layer. Because the note is part of the surgery after progression on first-line immunotherarchitecture and is generated for every patient at apy requires patients matched on treatment trajecingestion time, it is available as a query input for tory; a question about an unusual presentation patany patient (including the index patient, before tern requires matching on the narrative descriptions that patient’s actual board meeting). The narrain the patient’s pre-tumor-board summary note. The tive component captures clinical descriptors not module therefore exposes explicit weights over disalways represented in structured fields (e.g., “bitinct retrieval components, so that the same underlateral involvement,” “brain metastases,” “poor lying graph can serve different retrieval objectives functional status”). without re-indexing. The pipeline operates in four stages (Figure 4). Stage 1 (Hard Gate) filters the cohort by diagnosis category (NSCLC, SCLC, mesothelioma, thymoma) and metastatic status, reducing 1,180 candidates to a clinically compatible subset (typically 30–500 patients). Stage 2 (Weighted Multi-Component Retrieval) applies three parallel scoring components, each targeting a distinct facet of clinical similarity. The relative contribution of each component is controlled by weights wbio , wtraj , wnarr , which for this demonstration were set to equal contribution (wbio = wtraj = wnarr = 1/3):

Each component returns its top-5 candidates; after deduplication, 10–15 candidates advance to Stage 3 (Context Assembly), where the index patient’s full clinical profile and each candidate’s compressed profile are packed into a single prompt (∼15–20K tokens). Stage 4 (LLM Clinical Judge) uses a reasoning model to evaluate candidates and select 1–5 final matches, providing per-match rationale, key differences, and the matched patient’s post-tumor-board trajectory. Critically, Stages 1–3 require no LLM calls: the retrieval index is pre-computed from the same patient_info.json, episodes.json, and summary.json artifacts generated during per-patient TOA construction. Index construction for 1,180 patients completes in under 5 seconds, and retrieval executes in under 100 ms per query. Only the final clinical judge step (Stage 4) requires a single LLM call.

1. Biology component: Weighted Jaccard similarity on driver mutation profiles (EGFR, ALK, KRAS, ROS1, BRAF, MET, RET, NTRK, HER2) with secondary terms for histology and PD-L1 concordance. Matching actionable driver mutations (e.g., both EGFR-positive) receive 3× weight, reflecting their outsized influence on treatment seAs an illustrative query, querying an EGFR-positive lection. metastatic adenocarcinoma patient on second-line 2. Trajectory component: Composite score based palliative therapy retrieved 14 candidates from 554 13

after hard gating. The top-ranked match—also EGFR-positive adenocarcinoma on second-line palliative intent—achieved a biology score of 1.0 and trajectory score of 0.87, with multi-component retrieval surfacing this patient through both the biology and trajectory components independently. The equal-weight setting used here is a default rather than a fixed choice. If a specific retrieval task warranted emphasizing one facet over another—for example, weighting biology more heavily when the clinical question concerns mutational analogues, or weighting narrative when targeting an unusual presentation pattern—the same underlying graph artifacts support arbitrary weight configurations without re-indexing.

Discussion

an organized patient state and traverses downward only when source-level evidence is needed. This changes the role of retrieval. A relational query can return rows satisfying predefined predicates, and a RAG system can retrieve text fragments matching a query. In VISTA, the TOA layer provides a clinical map of the record: diagnosis, treatment phases, progression events, imaging anchors, toxicities, and current state are represented as connected objects. The agent can therefore retrieve by following clinically meaningful edges and targeting relevant nodes, rather than scanning or reassembling the patient history from raw tables or documents. This resembles how clinicians navigate a chart: they begin with the known clinical story, then drill into the specific note, report, image, or lab value needed to verify a claim. Viewed from an information-processing perspective, the architecture functions as a log-compressed clinical tree built to minimize information loss while maximizing navigability. The TOA and episode layers are structured compressions of the longitudinal EHR that preserve clinically salient state transitions, relationships, and provenance while organizing information at progressively higher levels of abstraction. For an average downstream query, many high-level facts are already present in context, and the model only needs to retrieve incremental details. This reduces the amount of text passed into the model, the amount of reasoning required to reconstruct patient state, and the number of source-level lookups, while retaining access to the original EHR when needed.

VISTA Architect addresses critical challenges encountered by multidisciplinary tumor boards, particularly the difficulty of synthesizing fragmented clinical narratives derived from complex electronic health record (EHR) data.[1, 2, 3, 21] Traditional methods often require repetitive re-processing of extensive EHR records at query time, introducing substantial computational overhead and latency. In contrast, VISTA Architect structures clinical data into a persistent, temporally structured, and contextspecific Timeline Object Architecture (TOA) data plane, inherently capturing critical information required for clinical decision-making. This approach reduces computational demands and enables nearBecause the representation is graph-based, new node instantaneous query response. types, edge types, provenance links, or specialtyThe architectural advantage of VISTA Architect is specific episode definitions can be added without renot that graph databases categorically replace reladefining the entire system. The same source-faithful tional storage. The lowest EHR layer could in pringraph can therefore support multiple clinical conciple be backed by SQL, document stores, or other figurations and downstream tools, including dashclinical data systems. The key distinction is repreboards, similarity search, automated note writing, sentational: VISTA transforms longitudinal docuplotting, and conversational agents. mentation into a hierarchical, clinically navigable structure. The MEDS Graph preserves source-level Pre-computing the graph also exposes a clean subdetail; the TOA Graph abstracts that detail into tem- strate for agents. Because TOA assertions are porally resolved clinical events; and the episode and provenance-tracked and graph-resident, agents can current-state layers compress the event stream into operate over deterministic, audited context withclinically meaningful phases and decision-ready ar- out performing their own raw-text retrieval. This tifacts. Query-time reasoning therefore begins from allowed the build pipeline, dashboard generation, 14

Stage 1 Hard Gate

1,180 → 30–500

Stage 2 Weighted Multi-Component Retrieval Biology (wbio )

Trajectory (wtraj )

Narrative (wnarr )

Stage 3 Context Assembly

10–15 candidates, ∼20K tokens

Stage 4 LLM Clinical Judge

1–5 final matches + rationale

Figure 4: Graph-guided patient similarity pipeline. Stages 1–3 operate entirely on pre-computed TOA artifacts without LLM calls. Three retrieval components each score a candidate on a distinct facet of clinical similarity, combined through tunable weights: wbio weights biology (Jaccard overlap of driver-mutation profiles, with histology and PD-L1 concordance); wtraj weights treatment trajectory (agreement in line number, therapeutic intent, and regimen drug-class); and wnarr weights narrative (BM25 lexical match over each patient’s pre-tumor-board summary note). Because these weights are applied at query time over the same pre-computed index, the graph can serve different retrieval objectives without re-indexing. Only Stage 4 requires a single LLM invocation for final clinical judgment.

chat, and similarity retrieval to share a single agentic interface (Section 2.1.4) while inheriting the structural guarantees of the graph layer underneath it. This shared agentic interface was the configuration used to measure wall-time gains under full intraand inter-patient parallelization without degradation of extraction quality (Section 2.5). These performance characteristics should be contextualized against the broader landscape of LLMbased clinical information extraction. While LLMs have demonstrated strong capabilities for singledocument summarization tasks where the input fits within the model’s context window,[7] the challenge of extracting structured clinical variables from full longitudinal EHR records—spanning years of documentation and thousands of events—remains substantially harder. A recent meta-analysis of 56 studies evaluating LLM integration in oncology decision-making reported an average overall accuracy of 76.2%, with diagnostic accuracy at 67.4%.[14] Agentic multi-agent approaches such as the Health-

care Agent Orchestrator achieved 84% strict recall on high-importance tumor board facts across 71 patients,[15] and optimized clinical RAG pipelines for note-level extraction reported F1 scores of 0.79– 0.90 depending on retrieval strategy.[13] VISTA Architect achieved 96.4% accuracy across 17,700 evaluations in 1,180 patients on the longitudinal wholerecord extraction task. In our matched comparison on a 30-patient subset, a standard BM25 RAG baseline using the same LLMs achieved approximately 67% accuracy where VISTA Architect achieved 96.9% (Section 2.3). These results indicate that the evaluation task is non-trivial—a standard RAG approach does not trivially solve it—and that VISTA Architect’s high accuracy is not an artifact of an easy benchmark. VISTA Architect’s demonstrated thoracic tumor board use case can be viewed as an automation and extension of a summarization workflow already developed and deployed at our institution:[18] whereas that system summarizes a fixed lookback

window of recent notes into a single live artifact, the precomputed, provenance-linked timeline yields the same summary note as one of several reusable artifacts, alongside structured variables and crosspatient retrieval. That deployment experience also indicates that, for well-specified summarization tasks, tightly constrained workflows can match or exceed higher-autonomy agentic designs; VISTA Architect is consistent with this principle, as its agentic layer operates over a deterministic, provenancetracked graph rather than performing unconstrained retrieval, and its fully agentic build is used to parallelize processing without altering extraction accuracy (Section 2.3). While VISTA Architect was conceived with general clinical workflows in mind, the current implementation and validation have been demonstrated within thoracic oncology tumor boards. The architecture supports flexibility in defining new node and edge types through configurable profiles; however, further empirical validation in additional clinical contexts remains necessary to substantiate broader claims about specialty agnosticism.

stantially affected by these coverage gaps and would require complementary data sources. Future directions for VISTA Architect include expanding validation across multiple sites, integrating multimodal data such as imaging and genomic information, and enhancing capabilities for managing larger patient cohorts. Prospective studies evaluating impacts on clinical workflow efficiency, user satisfaction, and clinical decision quality are also planned to provide a comprehensive assessment of the system’s clinical utility. Several extensions are deferred to future work: ablation studies isolating the contribution of each TOA layer; explicit validation of the temporal-normalization step against chart-reviewed event dates; expanded clinician adjudication beyond the 30-patient calibration subset; and stronger retrieval baselines including denseembedding and long-context single-pass configurations. The BM25 comparison reported in Section 2.4 is best interpreted as a coarse graph-versus-no-graph ablation rather than as a head-to-head against an optimized clinical RAG system.

In summary, VISTA Architect demonstrates a preThe present evaluation is based on a single-center computation paradigm for integrating large, comimplementation using exclusively English-language plex document repositories with LLM-driven applidocumentation. Future expansions to multiple cen- cations: by constructing a structured, temporally ters, varied healthcare settings, and multilingual clin- aware knowledge graph once from raw source data, ical documentation will be important to comprehen- the system transforms the retrieval problem from a sively assess generalizability and robustness. Addi- per-query search into a deterministic graph travertionally, the architecture’s reliance on accurate and sal, addressing both the accuracy and the speed recomplete original EHR data is a notable limitation, quirements of high-stakes decision-making.[4, 5, 6] as incomplete documentation inherently impacts While validated here in thoracic oncology tumor the accuracy and completeness of the derived clin- boards, the underlying architecture—configurable ical narratives. A related limitation is that OMOP- node types, episode structures, and provenancederived data exports do not capture all sources that tracked event timelines—may be applicable to other clinicians reference in practice—Care Everywhere settings where longitudinal, heterogeneous docunotes, external lab results, and outside-institution ment collections must be made reliably accessible to records may be absent. This limitation is partially AI systems; such applications will require prospecmitigated by our evaluation cutoff: ground truth tive validation in their own clinical contexts. was truncated at tumor-board-date minus one day, which guarantees that the available record was at 4 Methods (Evaluation) minimum sufficient for the actual tumor board meeting to have proceeded. Downstream events occur- 4.1 Dataset ring outside the index institution (e.g., mortality ascertainment or recurrence detection) are more sub- Our evaluation cohort comprised 1,180 thoracic oncology patients drawn from the VISTA Oncology 16

Data Lake, a multi-modal clinical data resource established by Stanford Medicine’s Research Technology group (full description in Appendix B2 ). The Data Lake integrates clinical notes, genomic data, and imaging metadata together with the Stanford Cancer Registry, covering approximately 222,000 patients across the Stanford hospital ecosystems; the derivation of the 1,180-patient evaluation cohort is summarized in Supplementary Fig. S1. Patient inclusion in the Data Lake requires either a documented tumor board encounter in the source EHR or a record in the Stanford Cancer Registry (which feeds the California Cancer Registry). The clinical dataset is delivered as BigQuery tables transformed from the source Epic Clarity instances to the Observational Medical Outcomes Partnership (OMOP) Common Data Model, derived from Stanford’s STARR-OMOP electronic health record representation. For VISTA Architect, each patient’s OMOP-formatted record was exported from the Data Lake and converted into the MEDS XML representation (Section 2.1) that serves as input to the MEDS Graph. All records are Safe Harbor de-identified by Research Technology before extraction; in particular, all dates (including dates of birth, death, and service dates) are shifted by a per-patient offset of ±30 days (excluding zero), which preserves intra-patient temporal ordering while removing the original calendar dates. No protected health information left the institutional perimeter during analysis (Stanford IRB protocol 76049). All LLM inference was carried out through PHI-compliant services—Stanford SecureGPT (Azure OpenAI) for the sequential pipeline and Google Vertex AI for the agentic pipeline—so that patient data never left the institutional data perimeter at any stage. 4.2

Accuracy Assessment

We evaluated VISTA Architect’s clinical accuracy using a structured, variable-level evaluation protocol across all 1,180 patients. From each patient, we extracted 15 MTB-salient variables spanning six clinical categories: demographics (Date of Birth, Sex, Smoking Status), tumor characteristics (Diagnosis, 2 Public overview: https://susom.github.io/starroncology-data-lake-arpah/about.html.

Histology, Metastasis, Lymph Node Involvement, Genetic Testing Panel), clinical status (ECOG Performance Status, Therapy Toxicity/Comorbidities), treatment (Previous Surgery, Current Medical Therapy, Radiation Therapy), imaging (Date of Last CT), and safety (Allergies). This yielded 17,700 total variable evaluations. The variable set was designed to span both the structured–unstructured spectrum and the range of extraction difficulty encountered in tumor board preparation, from variables resolvable from a single field to variables that require careful collection across the entire EHR history. Although several variables (notably Date of Birth and Sex) are reliably available in structured OMOP fields and could in principle be extracted deterministically, we deliberately designed the accuracy assessment so that every reported value was produced by the generative pipeline: this exposes any hallucination upstream of the note-extraction challenge, with the structured variables serving as positive controls (a value other than 100% accuracy on Date of Birth or Sex would indicate a regression in the pipeline). The remaining variables fall into two qualitatively different difficulty regimes. Collection variables—Previous Surgery, Therapy Toxicity / Comorbidities, and Allergies— require sweeping the full longitudinal record and assembling a set; for Previous Surgery in particular, the evaluation deliberately includes non-oncological and remote surgical history (e.g., prior pediatric procedures relevant to anesthesia or current surgical planning), which typically appears only in free-text past medical history rather than in structured procedure tables. Time-varying state variables—ECOG Performance Status, Current Medical Therapy, Metastasis, Lymph Node Involvement, and Date of Last CT—require extracting the current status from multiple potentially conflicting mentions, where the temporal development of the variable matters. The 15 variables were selected to have a defined value for every oncology patient; absence of documentation is itself an informative value (e.g., no documented oncologic surgery is interpreted as “No Previous Surgery”), and the LLM-as-judge scored “no documentation” symmetrically against the ground-truth XML.

To establish ground truth, we used each patient’s Our scoring rubric balanced strictness with clinical raw EHR XML truncated at the documented tu- realism. A score of 10 required exact or clinically mor board date, ensuring evaluation reflects only equivalent matches between extracted and groundinformation available at the clinical decision point. truth values. Partial credit (scores 5–9) was assigned We employed an LLM-as-judge approach[22] using for clinically accurate but incomplete information— GPT-5 to perform automated, bidirectional compar- for example, correctly identifying adenocarcinoma ison between VISTA Architect’s structured JSON histology but missing a specific mutation subtype. outputs and the XML ground truth. The judge as- We adopted specialized handling for ambiguous clinsigned each variable a correctness label (Correct ical scenarios: metastasis and lymph node involveor Incorrect) and a 10-point quality score with de- ment were scored as Correct when imaging sugtailed rationale. This approach has been validated in gested disease without pathological confirmation, clinical settings, where LLM-based fact verification reflecting real-world MTB practice where treatment against EHR data achieves agreement with clinicians decisions must proceed on the basis of radiologic exceeding inter-clinician agreement.[23] An LLM-as- findings. judge fact-scoring pipeline using the same model The large evaluation cohort (N=1,180) provides stafamily (GPT-5) was independently validated against tistical power to estimate accuracy with narrow 95% physician fact-scoring for thoracic tumor board sumconfidence intervals. For mean scores, we report maries at the same institution, reporting judge– normal-approximation CIs; for proportion correct, physician agreement comparable to inter-physician we report Wilson score intervals. The scale of evaluaagreement,[18] supporting its use as a scalable evaltion also mitigates a key concern with LLM-as-judge uator in this setting. To calibrate the LLM-as-judge paradigms: at small sample sizes, correlated halluciin our setting, a clinician independently reviewed nations between extraction and evaluation models all 480 variable evaluations across the randomly secould systematically inflate accuracy. At N=1,180, lected, representative 30-patient subset described such random concordance errors contribute negabove, classifying each judge decision as Agree, Unligible bias (<0.1 percentage points), and systemcertain, or Disagree against chart-reviewed ground atic aligned errors are further mitigated by the artruth. All 30 patients were evaluated using the same chitectural separation between extraction (graph16-variable schema; variables without explicit docserialized input, deterministic retrieval context) and umentation were retained as valid evaluable states evaluation (raw XML ground truth, independent when clinically appropriate, such as the absence of judge prompt). documented oncologic surgery being interpreted as no Previous Surgery rather than as missing data. The evaluation pipeline uses distinct models across stages to avoid same-model self-evaluation: GPT-4.1 Decoding and determinism. All LLM calls in the is used for chunk-level TOA event extraction (where production pipeline (GPT-4.1 chunk extraction, GPTparallelization speed is prioritized), while GPT-5 5 patient-info synthesis, GPT-5 LLM-as-judge) and is used for downstream patient-info synthesis and, in the agentic pipeline (Gemini 3.5 Flash per-chunk independently, as the LLM judge for the accuracy extraction and episode synthesis, Claude Opus 4.6 orchestrator / display) use vendor-default decodevaluation reported here. ing settings: temperature 1.0, top-p 1.0, no explicit The same randomly selected, representative 30- random seed, and provider default max_tokens. patient subset was additionally used as the held-out Reasoning-model parameters (reasoning_effort test cohort for benchmarking the fully agentic par- for GPT-5, thinking_budget for Gemini 3.5 Flash) allelized build (Section 2.1.4) and the RAG retrieval are not adjusted from their defaults except that baseline (Section 2.4, Appendix G); no development thinking_budget is explicitly set to 0 for shortof either system was conducted on this subset. output Flash calls to avoid token starvation. No determinism guarantees are made or required for 18

the reported accuracy numbers, which are reported as single-run point estimates with binomial / Wilson confidence intervals appropriate to the evaluation sample size (Table 6).

The VISTA Oncology Data Lake from which the cohort was drawn is governed by Stanford Medicine TDS under the ARPA-H program; access procedures and eligibility are described at https://susom.github.io/starr-oncology-dataA completed TRIPOD-LLM reporting checklist (Gallake-arpah/about.html. Patient-level data cannot lifant et al., 2025) is provided as Supplemenbe redistributed by the authors. tary File S1, mapping each checklist item to the manuscript section in which it is addressed. 4.3

Efficiency Benchmarks

Acknowledgments

We measured VISTA Architect’s computational per- We thank the Stanford thoracic oncology team formance at both the cohort and single-query level. for clinical guidance throughout the development At the cohort level, we measured end-to-end build of VISTA Architect. We acknowledge Stanford wall time under two configurations: the sequential Medicine’s Technology and Digital Solutions (TDS) pipeline used to generate all results in Section 2.3 group, and in particular Somalee Datta, for build(running on one patient at a time, and additionally ing and maintaining the VISTA Oncology Data Lake with 10 patients in parallel as the production regime), from which the cohort used in this study was drawn. and the fully agentic pipeline (Section 2.1.4) running This research was funded, in part, by the Advanced with full intra- and inter-patient parallelization. At Research Projects Agency for Health (ARPA-H). The the single-query level, we measured graph query views and conclusions contained in this document latency across temporal, episodic, and measurement are those of the authors and should not be interoperations on the pre-computed TOA layer, and end- preted as representing the official policies, either to-end chat latency encompassing all three stages of expressed or implied, of the U.S. Government. T.K. was supported by the Finnish Cultural Foundation. the agentic chat pipeline.

Code and Data Availability

Competing Interests

The VISTA Architect backend—comprising MEDS XML to MEDS Graph conversion, TOA event ex- The authors declare no competing financial or nontraction, episode synthesis, and graph-resident financial interests. query infrastructure—is available as open source on GitHub at https://github.com/VISTA-Stanford/ vista-architect, together with example prompts Author Contributions (CRediT) sufficient to apply the pipeline to MEDS-formatted data from any source; the snapshot corresponding Conceptualization: T.K., M.A.R., S.P., J.F., P.A., D.W., to this manuscript is tagged v0.1.0-preprint (com- T.J.E.-C., A.F., B.N., and J.N. Methodology: T.K., J.F., mit 8837d44). The agentic implementation of the AI D.W., T.J.E.-C., and J.N. Software: T.K. and J.F. Vallayer (Section 2.1.4, Appendix H) is released along- idation: T.K., A.F., D.W., T.J.E.-C., and J.N. Formal side the backend in the same repository. The produc- analysis: T.K. Investigation: T.K. and P.A. Resources: tion frontend (Dash/Plotly user interface) is not part J.F., P.A., B.N., S.P., and M.A.R. Data curation: T.K., of this release; the exact production prompts used for J.F., P.A., and B.N. Visualization: T.K. Supervision: the thoracic oncology configuration are reproduced M.A.R. and S.P. Project administration: B.N., S.P., verbatim in Supplementary File S2. The meds2text and M.A.R. Funding acquisition: S.P. and M.A.R. library used in Step 1 is publicly available at https: Writing – original draft: T.K. and M.A.R. Writing – //github.com/VISTA-Stanford/meds2text. review and editing: all authors. 19

Pipeline role

Model identifier

Serving endpoint

Access window

Chunk-level event extraction (sequential pipeline) Patient-info synthesis (sequential pipeline) LLM-as-Judge (accuracy evaluation) RAG baseline answer generation (variant 1) RAG baseline answer generation (variant 2) Per-chunk event extraction & episode synthesis (agentic pipeline) Agentic orchestrator & display synthesis (agentic pipeline)

gpt-4.1

Stanford SecureGPT (Azure OpenAI; PHI-compliant)

Feb 2026

gpt-5

Stanford SecureGPT (Azure OpenAI)

Feb 2026

gpt-5

Stanford SecureGPT (Azure OpenAI)

Feb 2026

gpt-4.1

Stanford SecureGPT (Azure OpenAI)

Apr 2026

gpt-5

Stanford SecureGPT (Azure OpenAI)

Apr 2026

gemini-3.5-flash Google Vertex AI (Stanford GCP project som-nero-plevriti-deidbdf)

May–Jun 2026

Google Vertex AI (Stanford GCP project som-nero-plevriti-deidbdf)

May–Jun 2026

claude-opus-4-6

Table 6: Exact model identifiers, serving endpoints, and access windows for every LLM used in this study. Both endpoints are PHI-compliant: SecureGPT proxies Azure OpenAI inside the Stanford institutional perimeter; the Vertex AI project is a Stanford-managed GCP environment under the same data-use agreement. Decoding settings for all models are vendor defaults (§4.2 “Decoding and determinism”). Model identifiers are pulled verbatim from gsgpt.py (MODELS, VERTEX_MODELS, VERTEX_ANTHROPIC_MODELS) and from the gcpclaude launcher environment.

Ethics Approval This study was reviewed and approved by the Stanford University Institutional Review Board under protocol 76049 (same value as §4.1). The protocol covers secondary research use of de-identified EHR data extracted from the VISTA Oncology Data Lake. No protected health information left the institutional perimeter at any stage of analysis. The Data Lake is Safe Harbor de-identified by Stanford Medicine Research Technology before extraction, with perpatient date shifts of ±30 days (excluding zero) to preserve intra-patient temporal ordering while removing the original calendar dates. Patient consent was waived in accordance with 45 CFR 46.116(f) on the basis of minimal risk and infeasibility of obtaining individual consent for secondary use of deidentified records.

References [1] Jeffrey Budd. Burnout related to electronic health record use in primary care. Journal of primary care & community health, 14:21501319231166921, 2023. 20

[2] Christine Sinsky, Lacey Colligan, Ling Li, Mirela Prgomet, Sam Reynolds, Lindsey Goeders, Johanna Westbrook, Michael Tutty, and George Blike. Allocation of physician time in ambulatory practice: a time and motion study in 4 specialties. Annals of internal medicine, 165(11):753–760, 2016. [3] Shasha Han, Tait D Shanafelt, Christine A Sinsky, Karim M Awad, Liselotte N Dyrbye, Lynne C Fiscus, Mickey Trockel, and Joel Goh. Estimating the attributable cost of physician burnout in the united states. Annals of internal medicine, 170(11):784–790, 2019. [4] Richard D Hammer, Donna Fowler, Lincoln R Sheets, Athanasios Siadimas, Chaohui Guo, and Matthew S Prime. Digital tumor board solutions have significant impact on case preparation. JCO Clinical Cancer Informatics, 4:757–768, 2020. [5] Alex Nobori, Chayanit Jumniensuk, Xiang Chen, Dieter Enzmann, Sarah Dry, Scott Nelson, and Corey W Arnold. Electronic health

record–integrated tumor board application to save preparation time and reduce errors. JCO Clinical Cancer Informatics, 6:e2100142, 2022.

intensive nlp tasks. Advances in neural information processing systems, 33:9459–9474, 2020.

[6] Linda Chia-Fang Chang, Hsuan-Chih Kuo, [12] Nandan Thakur, Nils Reimers, Andreas Rücklé, Hung-Ming Wang, Yung-Chia Kuo, Ching-Ting Abhishek Srivastava, and Iryna Gurevych. Beir: Wang, Li-Chin Chen, and Jason Chia-Hsun A heterogenous benchmark for zero-shot evalHsieh. The use of an integrated digital tool uation of information retrieval models. arXiv to improve the efficiency of multidisciplinary preprint arXiv:2104.08663, 2021. tumor boards—a prospective trial in taiwan. [13] Ivan Lopez, Akshay Swaminathan, Karthik Cancers, 17(3):444, 2025. Vedula, Sanjana Narayanan, Fateme [7] Dave Van Veen, Cara Van Uden, Louis BlankeNateghi Haredasht, Stephen P Ma, April S meier, Jean-Benoit Delbrouck, Asad Aali, ChrisLiang, Steven Tate, Manoj Maddali, tian Bluethgen, Anuj Pareek, Malgorzata PoRobert Joseph Gallo, et al. Clinical entity lacin, Eduardo Pontes Reis, Anna Seehofnerová, augmented retrieval for clinical information et al. Adapted large language models can outextraction. NPJ digital medicine, 8(1):45, 2025. perform medical experts in clinical text summarization. Nature medicine, 30(4):1134–1142, [14] Yuexing Hao, Zhiwen Qiu, Jason Holmes, 2024. Corinna E Löckenhoff, Wei Liu, Marzyeh Ghassemi, and Saleh Kalantari. Large language [8] Lavender Yao Jiang, Xujin Chris Liu, model integrations in cancer decision-making: Nima Pour Nejatian, Mustafa Nasir-Moin, a systematic review and meta-analysis. NPJ Duo Wang, Anas Abidin, Kevin Eaton, Digital Medicine, 8(1):450, 2025. Howard Antony Riina, Ilya Laufer, Paawan Punjabi, et al. Health system-scale language [15] Matthias Blondeel, Noel Codella, Sam Preston, models are all-purpose prediction engines. Hao Qiu, Leonardo Schettini, Frank Tuan, WenNature, 619(7969):357–362, 2023. wai Yim, Smitha Saligrama, Mert Öz, Shrey Jain, et al. Healthcare agent orchestrator (hao) for pa[9] Hejie Cui, Alyssa Unell, Bowen Chen, Jatient summarization in molecular tumor boards. son Alan Fries, Emily Alsentzer, Sanmi Koyejo, arXiv preprint arXiv:2509.06602, 2025. and Nigam H Shah. Timer: Temporal instruction modeling and evaluation for longitudinal [16] George Hripcsak, Jon D Duke, Nigam H Shah, clinical records. npj Digital Medicine, 8(1):577, Christian G Reich, Vojtech Huser, Martijn J 2025. Schuemie, Marc A Suchard, Rae Woong Park, Ian Chi Kei Wong, Peter R Rijnbeek, et al. Ob[10] Elham Asgari, Nina Montaña-Brown, Magda servational health data sciences and informatDubois, Saleh Khalil, Jasmine Balloch, ics (ohdsi): opportunities for observational reJoshua Au Yeung, and Dominic Pimenta. searchers. Studies in health technology and inforA framework to assess clinical safety and matics, 216:574, 2015. hallucination rates of llms for medical text summarisation. NPJ digital medicine, 8(1):274, [17] Yong Shang, Yu Tian, Kewei Lyu, Tianshu Zhou, 2025. Ping Zhang, Jianghua Chen, and Jingsong Li. [11] Patrick Lewis, Ethan Perez, Aleksandra Electronic health record–oriented knowledge Piktus, Fabio Petroni, Vladimir Karpukhin, graph system for collaborative clinical decision Naman Goyal, Heinrich Küttler, Mike Lewis, support using multicenter fragmented medical Wen-tau Yih, Tim Rocktäschel, et al. Retrievaldata: design and application study. Journal of augmented generation for knowledgeMedical Internet Research, 26:e54263, 2024. 21

[18] Tim Ellis-Caleo, Timothy Keyes, Nerissa Ambers, Faraah Bekheet, Wen-wai Yim, Nikesh Kotecha, Nigam H. Shah, and Joel Neal. Development, evaluation, and deployment of a multi-agent system for thoracic tumor board. arXiv preprint arXiv:2604.12161, 2026. [19] Bert Arnrich, Edward Choi, Jason Alan Fries, Matthew BA McDermott, Jungwoo Oh, Tom Pollard, Nigam Shah, Ethan Steinberg, Michael Wornow, and Robin van de Water. Medical event data standard (meds): Facilitating machine learning for health. In ICLR 2024 Workshop on Learning from Time Series For Health, pages 03–08, 2024. [20] Yan-Ran Joyce Wang and Akshay S. Chaudhari. Ai-driven smart patient retrieval for precision oncology. Nature Reviews Cancer, 26(5):305–307, 2026. [21] Fred Schulte and Erika Fry. Death by 1,000 clicks: Where electronic health records went wrong. Kaiser Health News, 18, 2019. [22] Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, et al. Judging llm-as-a-judge with mtbench and chatbot arena. Advances in neural information processing systems, 36:46595–46623, 2023. [23] Philip Chung, Akshay Swaminathan, Alex J Goodell, Yeasul Kim, S Momsen Reincke, Lichy Han, Ben Deverett, Mohammad Amin Sadeghi, Abdel-Badih Ariss, Marc Ghanem, et al. Verifying facts in patient care documents generated by large language models using electronic health records. NEJM AI, 3(1):AIdbp2500418, 2025.

Supplementary Information Supplementary Contents A. Supplementary Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 B. VISTA Oncology Data Lake: Supplementary Note . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30 C. MEDS Graph Schema (Hierarchical XML Representation) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 D. TOA Graph Schema (Clinical Narrative) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34 E. LLM Prompts (Production) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37 F. MTB Variable Dictionary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43 G. RAG Baseline Comparison . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43 H. Agentic Implementation Details . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48

Supplementary Figures

This section collects the supplementary figures referenced throughout the main text. Supplementary Fig. S1 summarizes selection of the evaluation cohort. Supplementary Figs. S2, S3, S4, and S5 illustrate the VISTA Architect user interface—the Overview, Clinical Timeline, automated pre-tumor-board summary note, and chat views—each rendered directly from graph-resident artifacts with provenance-linked access to the source EHR. Supplementary Fig. S6 shows the two-tier MEDS + TOA graph structure for a single representative patient.

VISTA Oncology Data Lake ∼222,000 patients (Stanford Medicine, all malignancies)

Thoracic tumor board cohort documented thoracic tumor board encounter; first encounter on/after 1 January 2020 (de-identified) n = 1,180 (15 variables; 17,700 evaluations)

Validation subset, n = 30 clinician concordance review, agentic build benchmark, and BM25 RAG comparison Figure S1: Cohort flow. Selection of the 1,180-patient thoracic oncology evaluation cohort from the VISTA Oncology Data Lake, and of the 30-patient subset used for clinician concordance review, the agentic build benchmark, and the BM25 RAG comparison. The full cohort contributes 15 tumor board–salient variables per patient (17,700 evaluations).

Figure S2: VISTA Architect Overview view. A single-screen synopsis of the patient assembled from graph-resident artifacts: patient identity, staging assessment, primary diagnosis with molecular profile, current and prior treatment protocol, clinical history, and the most recent findings. All fields are populated from the TOA layer and the structured patient_info.json and summary.json artifacts produced during pipeline processing. Synthetic demonstration patient; not a real patient.

Figure S3: VISTA Architect Clinical Timeline view. The longitudinal record organized into clinical episodes— diagnosis and numbered treatment lines, each with start/end dates, intent, and termination reason—on the right, with the contributing imaging and assessment events aligned chronologically on the left. Toggles switch between the episode-level summary, the complete event log, and the raw EHR. The view is rendered directly from TOA episodes and events, preserving temporal structure and source provenance. Synthetic demonstration patient; not a real patient.

Figure S4: VISTA Architect automated tumor board note. Automated pre-tumor-board summary note writing incorporates the whole patient history together with recorded tumor board decisions, generated from the same graph-resident artifacts. Synthetic demonstration patient; not a real patient.

Figure S5: VISTA Architect chat interface. The chat interface answers queries either directly from the TOA data plane or by reaching into the MEDS Graph data plane through targeted agentic graph queries, with provenance-linked access to the source EHR. Synthetic demonstration patient; not a real patient.

Figure S6: Two-tier graph structure for a single representative patient. Lower plane: the source-faithful MEDS Graph, whose nodes preserve the original EHR records (here dominated by measurements, source fragments, observations, clinical notes, and visits). Upper plane: the abstracted TOA Graph, in which clinical events are grouped into a small number of episode nodes (baseline, diagnosis, treatment line). Gold sourced-from edges link every TOA event back to its supporting MEDS evidence, making provenance explicit; contains, anchored-by, precedes, and followed-by edges organize the clinical timeline. Legend labels denote the semantic meaning of each node and edge in this configuration, not a fixed type vocabulary (Section 2.1.3). For this patient the ∼1,750 MEDS-graph nodes are abstracted into ∼77 TOA nodes; counts are illustrative of a single smaller-than-median patient (cohort median 3,608 MEDS-graph nodes).

VISTA Oncology Data Lake: Supplementary Note

This appendix provides additional context on the VISTA Oncology Data Lake, the clinical data resource from which the 1,180-patient evaluation cohort in this study was drawn. The condensed version of this material in Section 4.1 covers the points directly relevant to the analyses reported in this paper; the description here documents the broader scope, source systems, and operational properties of the resource for readers seeking institutional context. Scope and modalities. The Virtual Intelligence for Specialized Tumor Board Assessment (VISTA) Oncology Data Lake is a multi-modal data repository designed to integrate diverse clinical modalities—including clinical notes, genomic data, and imaging data—to facilitate oncology research and AI modeling. To bridge fragmentation of care across different clinics, the Data Lake also integrates the Stanford Cancer Registry. The Data Lake contains approximately 222,000 patients in total; approximately 12% of patients in the Stanford Cancer Registry have a documented tumor board encounter. Cohort inclusion criteria. A patient is included in the Data Lake if at least one of the following is true: (i) a documented tumor board presentation or evaluation in the source Electronic Health Record (EHR); or (ii) a verified analytic or non-analytic case in the Stanford Cancer Registry (SCR). The SCR tracks comprehensive disease histories dating back to 1988 and feeds into the California Cancer Registry. SCR analytic cases include patients whose first course of cancer treatment occurred at Stanford regardless of diagnosis location, and patients diagnosed at Stanford regardless of where they were subsequently treated (with potentially incomplete data in the latter case). SCR non-analytic cases—introduced in 2024—capture additional cancer-related encounters such as patients treated at Stanford for recurrence only, patients followed at Stanford and found to be cancer-free, and consultation-only or end-of-life-care patients neither diagnosed nor treated at Stanford. Source systems. Source EHR data comes from the three Stanford hospital ecosystems—Stanford Health Care, Stanford Children’s Hospital (formerly Lucile Packard Children’s Hospital), and Stanford Healthcare Tri-Valley—together with the network of more than one hundred pediatric and adult care clinics affiliated with University HealthCare Alliance and Packard Children’s Health Alliance. These ecosystems use two independent Epic instances that share Epic Clarity data with Research Technology. Non-Epic clinical sources (radiology DICOMs, whole-slide imaging, genetic-testing results) are shared across the two hospital ecosystems and are processed from single source systems. The Stanford Cancer Registry is shared across both ecosystems and managed on the Neuralframe KACI vendor platform, with monthly database snapshots delivered to Research Technology. Data architecture. The Data Lake is delivered as BigQuery datasets on Google Cloud Platform. BigQuery is a fully managed, serverless, HIPAA-compliant enterprise data platform optimized for high-performance analytics on large multi-modal biomedical datasets, with decoupled storage and compute enabling complex SQL queries in seconds. Two independent BigQuery datasets (EHR and Registry) are linked by the unique patient identifier (Medical Record Number and Date of Birth). The clinical dataset is transformed from the two Epic Clarity instances to the unified Observational Medical Outcomes Partnership (OMOP) Common Data Model; the data includes Epic modules such as Epic Genetics, Epic Beaker, and Epic Beacon. DICOM and WSI metadata are included in custom tables alongside the OMOP CDM. Registry data undergoes minimal ETL to preserve the original California Cancer Registry data model. The Data Lake is refreshed 30

quarterly, with each release accompanied by updated metrics and release-specific information at the Data Lake’s public overview page; version-specific metadata such as gene lists per genetic-test version are also hosted there. Privacy and de-identification. The Data Lake is PHI-scrubbed using Safe Harbor methods: PHI is replaced with surrogates, and all dates—including dates of birth, death, and service dates—are shifted by a per-patient offset that is unique across all datasets and data types, is between −30 and +30 days, and is never zero. This preserves the intra-patient timeline while removing the original calendar dates. Because of the large amount of unstructured data (clinical notes, DICOMs), the Stanford University Privacy Office has determined that additional Expert Determination is required for the Data Lake to be declared formally de-identified; the security requirements currently applied are comparable to NIH dbGaP datasets.

MEDS Graph Schema (Hierarchical XML Representation)

The MEDS Graph represents raw EHR data as a hierarchical structure preserving all original detail with granular provenance. The node and edge types listed in this appendix are the instantiation used in the thoracic oncology configuration evaluated here; they describe the semantic role each source record plays in the graph rather than a fixed schema, and additional types can be defined for other source systems or clinical domains. Figure S6 shows both graph layers for a single representative patient. C.1 C.1.1

Node Types Person Node (Demographics)

{ "node_type": "Person", "patient_id": "cohort1_136040534", "birth_datetime": "1937-09-20", "gender": "Male", "race": "White", "ethnicity": "Not Hispanic" }

C.1.2

Visit Node (Encounter Container)

{ "node_type": "Visit", "visit_id": "visit_67890", "visit_start_date": "2023-01-18", "visit_end_date": "2023-01-18", "visit_type": "Outpatient|Inpatient|Emergency" }

C.1.3

Event Nodes (Typed by EHR Source)

// Note Event { "node_type": "Event", "event_type": "note", "event_id": "evt_note_5678", "note_id": "12345", "note_type": "Progress Note", "timestamp": "2023-01-18 09:30:00", "visit_id": "visit_67890", "care_site_id": "cs_oncology_clinic", "provider_id": "prov_oncologist_123" } // Measurement Event (Lab Panel) { "node_type": "Event", "event_type": "measurement",

"event_id": "evt_lab_panel_cbc_2023_01_15", "measurement_id": "panel_cbc_2023_01_15", "panel_name": "Complete Blood Count", "timestamp": "2023-01-15 08:00:00", "visit_id": "visit_67889" } // Individual Lab Value (Measurement Decomposition) { "node_type": "Event", "event_type": "measurement", "event_id": "evt_lab_wbc_2023_01_15", "measurement_id": "meas_wbc_2023_01_15", "lab_name": "WBC", "value": 5.2, "units": "K/uL", "abnormal": false, "reference_range": "4.5-11.0" } // Image Event { "node_type": "Event", "event_type": "image", "event_id": "evt_img_ct_chest_2023_01_20", "image_id": "img_ct_12345", "modality": "CT", "body_site": "chest", "timestamp": "2023-01-20 14:00:00" } // Radiology Report (linked to Image) { "node_type": "Event", "event_type": "note", "event_subtype": "radiology_report", "event_id": "evt_note_rad_ct_chest_2023_01_20", "note_id": "rad_report_67890", "linked_image_id": "img_ct_12345", "timestamp": "2023-01-20 16:30:00" }

C.1.4

XMLFragment Node (Provenance Anchor / Source Fragment)

Granular provenance at the note/procedure/measurement level. This is the node shown as source fragment in Figure S6. { "node_type": "XMLFragment", "fragment_id": "frag_note_2023_01_18_5678", "evidence_date": "2023-01-18", "fragment_type": "note",

"source_id": "note_5678" }

C.2

Edge Types

• VISIT_CONTAINS: Visit → Event (events grouped under their encounter) • IMAGE_REPORT_FOR: RadiologyReport → Image (report describes image) • PROCEDURE_FOR: Image → Procedure (image is part of procedure) • HAS_MEASUREMENT: LabPanel → LabValue (panel contains individual values) • PRECEDES: Event → Event (temporal ordering within visit) • SOURCED_FROM: TOAEvent → XMLFragment (provenance to EHR source) C.3

Key Features

1. Measurement Granularity: Lab panels decomposed into individual queryable values 2. Radiology Coupling: Explicit edges link imaging reports to images 3. Visit Grouping: All events within a visit are children of a Visit node via VISIT_CONTAINS 4. Provenance: Every event linked to exact EHR source via XMLFragment nodes 5. Fast Construction: Build time ∼0.10s for a representative ∼2,200-node patient (∼2MB XML) 6. Low Memory: Typically <5MB even for 8+ years of longitudinal care

TOA Graph Schema (Clinical Narrative)

The TOA Graph synthesizes MEDS Graph data into a deduplicated, temporally-corrected clinical narrative with episodes and events. The event types, episode kinds, and edge types described below are the thoracic oncology configuration and capture the semantic meaning of each element; they are freely modifiable within the architecture rather than a fixed, canonical vocabulary, and new event types, episode kinds, and edges can be defined for other clinical domains (Section 2.1.3; Figure S6). D.1

Event Node Schema

Events represent distinct clinical occurrences with ontological deduplication. { "event_id": "evt_12345", "date": "2023-01-20", // clinical occurrence date "evidence_date": "2023-01-20", // first documentation date "source_event_refs": ["E1", "E5", "E12"], // MEDS Graph event references (provenance) "type": "imaging|diagnostic|treatment|surgery|lab|symptom| examination|procedure|adverse_effect|baseline_information| critical_information", "subtype": "ct|mri|pet-ct|biopsy|blood_test|...|null", "modality": "ct|mri|xray|pet|pet-ct|ultrasound|null", // imaging only

"site": "chest|head|abdomen|pelvis|spine|bone|whole_body|null", "laterality": "left|right|bilateral|null", "description": "<=80 chars, compact and factual", "priority": "MAJOR|MINOR", "values": { // lab events only "Hgb": "9.8", "WBC": "17.6" } }

Event Types (Thoracic Oncology): • imaging: CT, MRI, PET-CT, X-ray (with findings) • diagnostic: Biopsies, pathology, molecular testing (EGFR, ALK, PD-L1) • treatment: Systemic therapies (chemotherapy, immunotherapy, targeted) • surgery: Resections, biopsies (lobectomy, wedge, pneumonectomy) • lab: Blood tests, tumor markers (when abnormal/decision-informing) • adverse_effect: Treatment toxicities, complications (PE, neutropenia, fractures) • baseline_information: Pre-existing conditions (smoking, COPD, prior cancers) • critical_information: Code status (DNR/DNI), goals of care, performance status D.2

Episode Node Schema

Episodes group events into clinically meaningful phases. { "episode_id": "ep_txline_1", "kind": "baseline|diagnosis|treatment_line|post_oncological", "start_date": "2023-01-15", "end_date": "2023-07-20", "anchor_event_id": "evt_carbo_pem_pembro_start", "event_ids": ["evt_12345", "evt_67890", ...], // all events in episode // Treatment line specific (kind = "treatment_line") "line_number": 1, "treatment": "carboplatin/pemetrexed/pembrolizumab", "intent": "curative|palliative|adjuvant|neoadjuvant|null", "clinical_context": "Carbo/Pem/Pembro: initial partial response (6.5->4.2cm), then bone progression on PET", "termination_reason": "progression|toxicity|completion|no_effect|null" }

Episode Kinds (Thoracic Tumor Board): 1. baseline: Pre-oncological background (smoking history, comorbidities, family history) 35

2. diagnosis: Initial diagnostic workup → treatment decision 3. treatment_line: Line of therapy (may include surgery, chemo, radiation as planned) 4. post_oncological: Hospice, palliative care only, end of active treatment D.3

Edge Types

• PRECEDES: Event → Event (temporal ordering) • CONTAINS: Episode → Event (episode membership) • ANCHORED_BY: Episode → Event (episode starts with this event) • FOLLOWED_BY: Episode → Episode (treatment sequence) • SAME_DAY: Event ↔ Event (co-occurrence, same encounter) • SOURCED_FROM: Event → MEDSXMLFragment (provenance to EHR source) D.4

Key Features

1. Temporal Correction: Distinguishes clinical occurrence date from documentation date 2. Deduplication: Same condition mentioned 50 times → single event node 3. Complete Provenance: Every event links to MEDS Graph source via source_event_refs 4. Episodic Structure: Treatment lines organize events into decision-relevant phases 5. Fast Queries: Graph traversal <1ms D.5

Deterministic retrieval coverage on the test30 cohort

To verify that the TOA Graph in fact supports the deterministic retrieval claimed in the main text, we measured the coverage of six representative graph-resident retrieval targets across the 30-patient test cohort (Table S1). The selected targets span a range of clinical-information types: standard structured fields (drug exposures), free-text-anchored content (metastasis mentions, lymph node mentions), the latest specific documents (latest oncology note, latest chest CT report), and a more challenging extraction target (driver mutations, which may be recorded under multiple gene-name aliases or only in narrative form). Each target was implemented as a deterministic graph query over the pre-computed TOA layer with provenance back to the MEDS Graph. All queries executed in under 0.01 s per patient, and coverage on the 30-patient cohort ranged from 73% to 100% depending on the information type. Retrieval Target Metastasis mentions Lymph node mentions Drug exposures Latest oncology note Latest chest CT report Driver mutations

Coverage

Query Time

100% (30/30) 97% (29/30) 100% (30/30) 93% (28/30) 87% (26/30) 73% (22/30)

<0.01s <0.01s <0.01s <0.01s <0.01s <0.01s

Table S1: Deterministic graph-resident retrieval coverage on the 30-patient test cohort. All queries executed via NetworkX traversal of the pre-computed TOA layer.

LLM Prompts (Production)

The blocks below summarize the structure and key instructions of each production prompt for readability. The complete, verbatim production prompts—byte-for-byte as used in the pipeline, with Source and Model headers—are reproduced in Supplementary File S2, which follows the same E.x numbering used here. E.1

Phase 1: Event Extraction (Chunk Processing)

Source: toa/prompts/timeline_compact.txt | Model: GPT-4.1 (fast, parallel) Extract structured clinical events from EHR XML chunk. Return valid JSON: {"events":[ { "date": "YYYY-MM-DD", // clinical occurrence "evidence_date": "YYYY-MM-DD", // first documentation "source_event_refs": ["E1","E5"], // REQUIRED: MEDS Graph provenance "type": "imaging|diagnostic|treatment|surgery|lab|symptom| adverse_effect|baseline_information|critical_information", "modality": "ct|mri|pet|pet-ct|xray|ultrasound|null", "site": "chest|head|abdomen|pelvis|spine|bone|null", "description": "<=80 chars", "priority": "MAJOR|MINOR", "values": {"Hgb":"9.8","WBC":"17.6"} // labs only } ]} DISTINCTIONS: - baseline_information: Pre-existing (smoking, COPD, prior cancers, family history) - may be mentioned later but predates diagnosis - critical_information: DNR/DNI, goals of care, performance status, prior external studies mentioned in notes VARIABLES (thoracic oncology): Binary: Lymph Node Involvement, Metastases, Pleural Effusion, Driver Mutation Categorical: Histology, TNM components, ECOG, PD-L1, Treatment Response Discrete: Pack-Years, Tumor Size (cm), Prior Lines, Metastatic Sites Continuous: CEA, FEV1 (%), LDH, Hemoglobin, Radiation Dose (Gy) Genetics: EGFR, ALK, KRAS, PD-L1, ROS1, BRAF, MET, RET, NTRK PROVENANCE: For EVERY event, cite MEDS Graph sources in source_event_refs using [E1], [E2] tags from SOURCE EVENTS section. RULES: - Extract each imaging study SEPARATELY (PET-CT, MRI, CT are 3 events) - Imaging + adverse effect = TWO events (imaging + adverse_effect) - Baseline: smoking, allergies -> baseline_information - Code status during treatment -> critical_information - Extract prior external studies -> critical_information - Labs: only abnormal/decision-informing values SAFETY (always extract): - Adverse: PE, DVT, sepsis, pneumonia, hemorrhage, fractures, ICU

- Bleeding risk: anticoagulation, thrombocytopenia, IVC filter - Goals of care: DNR, DNI, hospice, comfort care - Baseline diagnostics: all initial workup imaging (even normal)

E.2

Phase 2: Episode Synthesis (Treatment Lines)

Source: toa/prompts/episodes_from_events.txt | Model: GPT-4.1 Segment timeline events into EPISODES (treatment-line-based for thoracic): 1. BASELINE: Background predating diagnosis (smoking, COPD, allergies) Anchor: one day before first oncological event 2. DIAGNOSIS: Diagnostic workup -> treatment decision Includes ALL imaging, biopsies, pathology, molecular testing Anchor: specialist visit or first diagnostic procedure 3. TREATMENT LINES: Each line of therapy until change needed Includes systemic therapy, surgery, radiation (as planned) Contains all imaging, labs, symptoms during line CRITICAL: Event that ENDS line (progression, toxicity) stays IN episode Anchor: treatment start date 4. POST-ONCOLOGICAL: Hospice, palliative only, end of active treatment Anchor: transition decision or hospice enrollment OUTPUT (strict JSON, NO event_ids - auto-populated by date range): { "episodes": [ { "episode_id": "temp", "kind": "baseline|diagnosis|treatment_line|post_oncological", "start_date": "YYYY-MM-DD", "end_date": "YYYY-MM-DD", "anchor_event_id": "<event_id from input>", "clinical_context": "Summary of episode", // treatment_line specific: "line_number": 1, "treatment": "carboplatin/pemetrexed/pembrolizumab", "intent": "curative|palliative|adjuvant|neoadjuvant|null", "termination_reason": "progression|toxicity|completion|no_effect|null" } ], "notes": ["Segmentation rationale"] } INVARIANTS: Use ONLY dates/event_ids from input. Episodes time-monotonic. Event that ENDS line belongs to THIS episode, not next.

E.3

Chat System Prompt (Agentic AI Bridge)

Source: prompts/chat_system.txt | Model: User-selected (GPT-4.1/GPT-5) 38

You are "Vista," board-certified oncology decision-support AI for multidisciplinary tumor boards. DATA SOURCES: 1. CONVERSATION HISTORY - Recent chat context 2. CLINICAL EPISODES (TOA) - High-level narrative phases 3. DETAILED TIMELINE (TOA Events) - Complete chronological log 4. PATIENT DATA - Structured JSONs (patient_info, summary) 5. RECENT EHR DATA - Last 10k chars of raw XML UNDERSTANDING PATIENT: - Form timeline from TOA events (most reliable source) - Cross-reference episodes for treatment phases - Patient may have transferred to Stanford - history in notes - Contradictions exist - base decisions on data support SCOPE: Base answers on provided data + latest guidelines (NCCN, ESMO, IASLC, ASTRO) + peer-reviewed evidence. STYLE: - Concise: bulleted lists over prose - Clinical language: NSCLC, COPD, ECOG, PD-L1 (standard abbreviations) - Direct answer first; brief context if it changes interpretation - Present options with pros/cons without choosing - Cite guidelines when relevant ("NCCN v2.2024") SAFETY: If insufficient data, say so + suggest what’s needed. Output: Plain text with markdown - NO JSON, NO code blocks.

E.4

Phase 3: Dashboard JSON Generation (UI Population)

After TOA timeline extraction, dashboard-specific JSONs are generated for the UI. E.4.1

Patient Info Extraction

Source: prompts/patient_info.txt | Model: GPT-4.1 You are a highest-expert-level oncology data assistant working at Stanford Tumor Board meetings. Tumor board subtype: {TUMOR_TYPE} INPUTS: - XML chunk (final chunk of patient record) - Timeline context (complete timeline from all chunks) PROCESSING: 1. Extract comprehensive demographics, medical history, tumor information 2. Use timeline for chronological progression context 3. Provide complete TNM staging (latest IASLC guidelines) 4. Cross-reference timeline events to validate dates 5. Use medical abbreviations (NSCLC, COPD, ECOG, PD-L1) 6. If name unclear (de-identification): use "John/Jane Doe"

OUTPUT: Valid JSON with EXACT keys (spaces, NOT underscores): PATIENT DEMOGRAPHICS: name, date_of_birth, sex, height_cm, weight_kg previous_conditions # chronic diseases OR major past events # (e.g., "COPD, Stroke (2011)") allergies # list smoking_history # (e.g., "40 pack-years; quit 2015") medications # list of {name, dose, frequency} TUMOR INFORMATION: diagnosis tnm_staging histology driver_mutations latest_updates body_diagram_image TREATMENTS: current previous

alternatives surgical_candidate

E.4.2

# "CANCER_TYPE. Metastasis (sites). Recurrent." # (e.g., "cT4N2M1b", IASLC 8th edition) # (e.g., "Moderately differentiated SCC") # dict: EGFR, ALK, KRAS, PD-L1, ROS1, etc. # MAX 3 bullets: (1) recent imaging status, # (2) current clinical situation, (3) TB question # filename in assets/images

# ONLY oncological regimens (no supportive meds) # "Drug1+Drug2+Drug3 (start: YYYY-MM-DD)" # Previous ONCOLOGICAL regimens # "Drug1+Drug2 (YYYY-MM-DD to YYYY-MM-DD, response, # reason stopped)" OR "Surgery (YYYY-MM-DD, outcome)" # list of alternative treatments # {"eligible": bool, "description": "..."}

Pre-Tumor-Board Summary Note Generation

Source: prompts/tumor_board_note.txt | Model: GPT-4.1 This prompt generates the pre-tumor-board summary note—a concise clinical summary in the standard Stanford Epic format presented at thoracic tumor board meetings. The note is generated by VISTA Architect at patient ingestion (i.e., before the actual tumor board meeting) and is stored as a graph-resident artifact. The internal prompt-file name (tumor_board_note.txt) reflects the production codebase naming and is retained here for source-of-truth fidelity. You are writing a Stanford Epic tumor board note. Follow EXACT 4-line format. Include detailed medical information based ONLY on provided patient data. Target 750-999 characters for completeness. FORMAT: AIGen: [LASTNAME]: [AGEGENDER] with h/o [CANCER TYPE including detailed pathology, staging, molecular features] Prior therapy: [DETAILED therapy history with dates, responses, toxicities] Tumor board question: [SPECIFIC clinical question based on current status] Tumor board decision: [DETAILED decisions made including rationale] CRITICAL INSTRUCTIONS:

- Use ONLY information from provided patient data, summary, and decisions - DO NOT invent dates, treatments, or clinical details not in data - Extract from PATIENT DEMOGRAPHICS: name, age, sex - Extract from TUMOR INFORMATION: diagnosis, staging, mutations - Extract from TREATMENTS: current and previous (with dates/responses) - Use summary’s cancer_history for prior therapy details - Use tumor board decisions provided (if any) for decision line - If information missing, use general terminology but do not fabricate Generate note following exact 4-line format above.

Usage in Pipeline: 1. prepare_patient.py loads patient_info.json and tumor_info.json 2. Builds context: patient data + tumor board decisions (if any) 3. Calls tumor board note prompt with GPT-4.1 4. Saves to summary.json for UI display E.5

Accuracy Evaluation (LLM-as-Judge) Prompt

Source: quick_eval.py::judge_all_variables | Model: GPT-5 The system prompt below was used verbatim to score each variable evaluation in the primary accuracy assessment. Only the {N_VARS} placeholder (set to 16 as run; DNR/code status is scored by the judge but excluded from the reported 15-variable, 17,700-evaluation primary endpoint) and the literal numbered variable list are interpolated at call time; the rubric text is fixed. You are evaluating the quality of {N_VARS} tumor board variables extracted from a clinical pipeline. Compare the extracted values against the EHR ground truth provided. For EACH variable, evaluate: - Correctness: Correct/Incorrect/Partial/N/A - Score: 1-10 (10=correct, 1=wrong, 5=partially correct) - Brief explanation (1-2 sentences) - XML value (ground truth value from XML) === CORE PRINCIPLE: JUDGE TRUTH, NOT FORM === You are scoring whether the answer is factually correct given the EHR, NOT whether it matches a preferred style, length, ordering, or completeness threshold. Concretely: 1. Honest uncertainty is correct. When the EHR genuinely does not document a field, "Unknown" or "Not documented" or "No" (where appropriate per the rules below) is a SCORE 10 answer — not a 7. The pipeline is being honest. 2. Default-No is correct when not documented. For binary/safety fields

where the rubric specifies a default of "No" when undocumented (DNR, Metastasis with no findings, Radiation when none given), "No" without further qualification scores 10. 3. Equivalent phrasings score the same. "NKDA" = "No Known Allergies" = "None". "Former smoker, 30py, quit 2010" = "30 pack-year former smoker, quit 2010-04". Range vs single value for ECOG ("1-2" vs "1") both score 10 when both are documented. 4. Verbose != wrong. Rich context alongside the core answer (e.g., "No — PET ruled out initial concern") scores the same as the bare answer ("No"). Do not deduct for extra clinically relevant detail. 5. Order in lists does not matter as long as the content is right and the most relevant items are recognizable. 6. Form/style preferences are not deductions. Do not deduct for: bullets vs prose, dates as YYYY-MM-DD vs YYYY-MM, "Yes (sites)" vs "Yes - sites", single-word vs sentence. DEDUCT for: (a) factually wrong values, (b) hallucinations (entities not in the EHR), (c) missed decision-relevant findings actually documented in the EHR, (d) inverted clinical meaning (Full Code reported as DNR, "No" when EHR confirms "Yes", etc.), (e) made-up dates/genes/drugs. === VARIABLE-SPECIFIC SCOPE CLARIFICATIONS === - Metastasis / Lymph Node Involvement: "Suspected" is correct when EHR evidence is suggestive but not definitive; "No" is correct if initial concern was resolved on follow-up imaging. - Previous Surgery — SCOPE (Yes/No, oncologic resection for CURRENT diagnosis only): determine from EHR whether the patient underwent an oncologic resection for the current cancer (lobectomy/pneumonectomy for NSCLC, thymectomy for thymoma, etc.). Score 10 when extracted value correctly reflects this; 5–6 when wrong direction; 1–3 when list contains a fabricated current-cancer surgery. Do NOT deduct for prior-cancer or non-oncologic surgeries appearing in the list (informational, not in scope). - Date of Last CT — SCOPE: PET-CT counts as CT. Most recent in-house or outside imaging in chart, whichever is more recent, is acceptable. Month-level (YYYY-MM) acceptable when day uncertain. - Genetic Testing Panel — SCOPE: do not penalize molecular results that are clinically correct but not visible in the provided XML chunk (the chunk may be truncated). Only penalize factually wrong or hallucinated results. Be concise but accurate.

User prompt format: the system prompt above is paired with a user prompt that supplies the numbered variable list (with expected format strings), the extracted-values JSON, and the XML ground-truth chunk (demographics block, surgery dates, and the final 120k-character chunk of the TB-date-truncated record). The model returns a single JSON object with one entry per variable: {correctness, score, explanation, xml_value}. Decoding settings (see §4.2 “Decoding and determinism”): GPT-5 is invoked with vendor defaults (temperature 1.0; no explicit seed; no reasoning_effort override). All accuracy numbers in this paper are 42

single-run point estimates with Wilson 95% confidence intervals.

MTB Variable Dictionary

VISTA Architect: A graph database-oriented health AI system demonstrated in multidisciplinary tumor boards

Related documents