PathView-Bench: Can Multimodal Large Language Models Achieve Fine-grained Multiscale Understanding of Pathology Images?
arXiv:2607.28318v1 [cs.AI] 30 Jul 2026
Zongyi Chen National Institute for Data Science in Health and Medicine, Xiamen University Xiamen, China Jie Lin Department of Computer Science School of Informatics Xiamen University, Xiamen, China
Yu Liang Institute of Artificial Intelligence Xiamen University, Xiamen, China Liansheng Wang Department of Computer Science School of Informatics Xiamen University, Xiamen, China
Abstract Multimodal large language models (MLLMs) are increasingly used to analyze pathology images. However, dominant multimodal benchmarks in pathology mainly score final diagnostic answers, captions, or reports. These evaluations provide limited insight into whether a model understands the multiscale visual content needed for pathology reasoning and decision-making. We introduce PathViewBench, a vision-anchored benchmark for fine-grained and multiscale visual understanding in computational pathology. Built from 23 public pathology imaging datasets with human-supervised labels and spatial annotations, PathView-Bench evaluates MLLM understanding in two fields of view: Region-FOV for highresolution local regions and Slide-FOV for macro whole-slide views. By converting raw annotations into deterministic task targets, PathView-Bench enables programmatic scoring of region localization, visual recognition, quantity estimation, spatial reasoning, and insufficient-context judgment. The benchmark contains 14 VQA-style tasks, 61, 673 images, and 308, 070 samples across 28 organs and 7, 253, 526 annotations. Evaluating 18 representative general-purpose, medicaldomain, and pathology-oriented MLLMs, we observe substantial limitations even in advanced models on fine-grained visual tasks across multiscale pathology images. PathView-Bench provides a reproducible basis for developing and evaluating pathology MLLMs with explicit multiscale visual understanding.
1
Introduction
Advances in digital pathology have enabled the digitization and computational analysis of histopathological slides, creating promising opportunities for AI-assisted diagnostics [1–10]. Recently, multimodal large language models (MLLMs) are increasingly being used for whole-slide analysis. By combining visual understanding with reasoning, these approaches are moving beyond single-task classifiers toward more general and fine-grained pathology assistants [11], including visual retrieval [12– 14], report-oriented interaction [15–18], multiscale exploration [19–21], navigation [22–24], and cross-FOV visual reasoning [25–28]. Existing pathology multimodal datasets and benchmarks mainly assess whether MLLMs can achieve diagnostic correctness [29–34], generate high-quality captions [35, 36], or produce plausible reports across pathology tasks [20, 19, 22]. This text-driven question-answering evaluation paradigm leaves a key question largely unexamined: whether a correct diagnostic answer is supported by finegrained visual understanding. More concerningly, models might score highly on medical multimodal benchmarks even without image input [37, 30]. This uncertainty makes it difficult to assess whether
Pathology Images Corpus Slide FOV
Region FOV
Region Classification GT malignant lesion
Object Counting GT 102
Existing Benchmark Q What was the grade of the invasive ductal carcinoma ?
Grade 3 Q What is the histological classification of the primary neoplasm observed in the provided whole slide image? Oligoastrocytoma,astrocytoma predominant Q What is the size of the carcinoma in the breast? 1.6 x 1.5 x 1.3 cm Q Draft a formal pathology report for the findings: The pathology slide describes a left adrenal gland biopsy taken for Conn's syndrome and genetic studies. The specimen showed a primarily trabecular adrenal tumor with a mitotic rate of less than 6 per 50 hpf and cytological atypia. The tumor is consistent with an adrenal cortical neoplasm of indeterminate malignant potential...
Text-driven Question-Answering
PathView-Bench
non-cancer tissue
Region Description lymphocyte,
GT fpibroblast, lasma cell
24
Lesion Grounding
Density Localization GT
GT Tumor non-cancer
Spatial Relation GT
Density Localization GT
Organ Classification GT Lung
Spatial Relation GT
Area Estimation GT 25%-50%
1
75%-100%
4
tumor cell, stroma
Colon
Object Classification GT epithelial cell inflammatory cell
Object Grounding GT
FOV Trap GT NA Stomach
Region FOV
FOV Trap GT NA 50 cells
Slide FOV
Performance Rank Better in Slide-FOV
Better in both
8 12
General MLLM Medical MLLM Pathology MLLM
16
Better in Region-FOVr
16
12
8
4
1
A vision-anchored benchmark for fine-grained and multiscale visual understanding
Figure 1: Motivating examples for PathView-Bench. The benchmark evaluates visual understanding across local-region and whole-slide fields of view through vision-anchored pathology tasks.
the diagnostic output is trustworthy. Moreover, pathology images are extremely large, and local details can differ substantially from global semantic context, yet existing benchmarks often remain limited to a single field of view and therefore provide limited evidence about whether models can understand pathology images across scales. As illustrated in Figure 1, these limitations leave a critical gap for advancing pathology MLLMs: How can we evaluate their ability to understand fine-grained visual content in multiscale pathology images during reasoning and decision-making? To address this gap, a benchmark is needed to make the visual basis of pathology reasoning measurable. We therefore introduce PathView-Bench, a vision-anchored benchmark for fine-grained and multiscale visual understanding in computational pathology. PathView-Bench is organized around two fields of view: Region-FOV for high-resolution local regions and Slide-FOV for macro whole-slide views, reflecting the fact that pathology reasoning depends on both cellular morphology and tissue organization. Across these views, PathView-Bench probes evidence-grounded visual understanding through tasks such as region localization, visual recognition, quantity estimation, spatial reasoning, and judging whether the current view contains sufficient evidence to answer. This shifts evaluation away from final-response plausibility and toward evidence grounded in the multiscale visual context. To construct the benchmark, we map 23 publicly available pathology imaging datasets with humansupervised labels and spatial annotations into a shared, auditable benchmark space, where deterministic rules convert the annotations into task targets for programmatic scoring. The resulting benchmark contains 14 VQA-style tasks, 61, 673 images, and 308, 070 samples across 28 organs and 7, 253, 526 annotations. Across 18 representative general-purpose, medical-domain, and pathology-oriented MLLMs, PathView-Bench reveals a recurring gap: models are stronger at naming visual categories than at grounding, counting, spatially organizing, or judging the sufficiency of the underlying evidence. Matched-slide analyses further show that correct slide-level diagnostic answers can mask failures in these basic visual operations, and neither larger scale nor pathology-specific training consistently closes the gap. In summary, the contributions of this work are as follows: 1. We introduce a vision-anchored benchmark for fine-grained and multiscale visual understanding in pathology, organized around Region-FOV/Slide-FOV and designed to evaluate whether model responses are grounded in observable image content. 2
Table 1: Comparison of pathology VQA-style multimodal benchmarks. All-WSI Source indicates whether all images are derived from WSIs. Annotation: LLM = language-model-assisted generation, Exp. = expert annotation or validation, Rule = fixed-rule conversion. Benchmark PathVQA [29] PathMMU [30] WSI-VQA [31] Micro-Bench [32] WSI-Bench [20] PathBench [33] SlideBench [19] PathVG [34] MultiPathQA [22] PathView (ours)
#Samples
#Images
#Task Types
Annotation Method
All-WSI Source
Vision-anchored
Region-FOV
Slide-FOV
32,799 33,428 8,672 8,485 179,569 33,235 15,835 33,500 934 308,070
4,998 24,067 977 8,385 9,850 12,060 1,792 27,610 868 61,673
7 1 1 3 4 2 2 1 1 14
Exp.+Rule LLM+Rule+Exp. LLM+Rule LLM+Exp. LLM+Exp. LLM+Exp. LLM+Rule+Exp. LLM+Exp. Exp.+Rule Exp.+Rule
× × ✓ × ✓ × ✓ ✓ ✓ ✓
× × × ✓ × × × ✓ × ✓
✓ ✓ × ✓ × ✓ × ✓ × ✓
× × ✓ × ✓ ✓ ✓ × ✓ ✓
2. We define an auditable construction and evaluation protocol that standardizes human-supervised labels and spatial annotations into deterministic task targets with programmatic scoring, without LLM-as-a-judge evaluation. 3. We conduct comprehensive evaluations of 18 representative general-purpose, medical-domain, and pathology-oriented MLLMs on PathView-Bench, characterizing their capability boundaries in fine-grained multiscale visual understanding. We further include a PathView-tuned baseline to test whether supervision provides learnable signals for grounding-sensitive behavior.
2
Related Work
Multimodal Large Language Models. Recent general-purpose MLLMs have broadly advanced multimodal perception, interaction, and reasoning [38–40]. This progress has been adapted to medicine through medical image-text pretraining [41, 42], broad biomedical supervision [43], and domain-specific instruction tuning or medical reasoning alignment [44, 45]. In pathology, the same trajectory has moved from patch-level dialogue and visual instruction tuning [11–13], to report generation [15–17], and then toward whole-slide interaction, WSI-level modeling, navigation, and evidence-seeking reasoning [19, 20, 22, 25–27]. These models motivate a stricter evaluation regime: once systems claim richer whole-slide behaviors, they should be judged not only by linguistic fluency but also by whether their outputs remain grounded in visible pathology content. Pathology MLLM Benchmarks. Pathology multimodal evaluation has progressed from patch image question answering and image-text resources to WSI-level interaction. PathVQA [29] established pathology VQA on patch images, while Quilt-1M [35] and PathGen-1.6M [36] provided large-scale histopathology image-text pairs for representation learning. Later resources broadened the evaluation scope: PathMMU [30] and Micro-Bench [32] emphasized expert-level or microscopy-oriented local understanding, WSI-VQA [31], WSI-Bench [20], SlideBench [19], and MultiPathQA [22] moved toward whole-slide QA and navigation, and PathBench [33] covered both patch and WSI settings. PathVG [34] is closer to explicit visual grounding, but remains focused on region-level localization. Overall as shown in Table 1, existing resources either emphasize local morphology without global slide context, or evaluate WSI-level answers without requiring grounded intermediate visual analysis. PathView-Bench is designed to cover both Region-FOV and Slide-FOV while making visual grounding and fine-grained visual-understanding operations central to the evaluation.
3
PathView-Bench
In this section, we describe the design and construction of PathView-Bench, including its dual-scale task organization, annotation-grounded protocol, and deterministic split and evaluation procedure. 3.1
Task Design
WSIs are gigapixel-scale images whose diagnostically relevant information appears at different visual scales [46]. In routine pathology review, pathologists assess whole-slide tissue architecture and lesion distribution, while also examining selected high-resolution regions that contain essential morphological details [23]. PathView-Bench follows this dual-scale reading process by organizing its 3
Table 2: Overview of the diverse task types, query and answer examples of PathView-Bench Task name
Query example
Answer example Region FOV
Object Grounding Object Classification Region Classification Object Counting Region Description Density Localization Spatial Relation FOV Trap Organ Classification Lesion Grounding Object Classification Area Estimation Density Localization FOV Trap
Locate all <object>. Which category best describes the predominant finding in this region <bbox>? Which category best describes the predominant finding in this image? How many <object> are present in this image? Describe what is in the image. Which quadrant contains the densest distribution of <object>? Local coordinate <bbox> marks a core region. Which category is nearest to this region? Which organ is this image most likely from? Slide FOV Which organ is this image most likely from? Locate all regions of <object>. Which category best describes the predominant finding in this region <bbox>? What percentage of the tissue area in the image is occupied by <object>? Which quadrant contains the densest distribution of <object>? How many <cell> are present in this image?
[0, 0, 8, 12], ... A. plasma cell A. tumor cell 35 eosinophil, ... Bottom-right quadrant B. tumor cell Insufficient information B. spleen [50, 138, 87, 227], ... A. malignant lesion A. 0%-25% A. Top-left quadrant Insufficient information
VQA-style tasks into two fields of view: Slide-FOV tasks for macro whole-slide understanding and Region-FOV tasks for fine-grained local-region understanding, as shown in Table 2. 3.1.1
Slide-FOV Whole-Slide Understanding
Slide-FOV tasks evaluate visual understanding from macro whole-slide views. They focus on information that can be assessed at low magnification, including tissue context, lesion location, abnormal burden, and large-scale spatial distribution. These tasks test whether a model can interpret slide-level visual patterns and recognize when the current view is sufficient for the requested inference. Organ Classification. Identifying the organ or tissue source is a basic prerequisite for interpreting a macro view with representative tissue morphology. In this task, the model is asked to infer the most likely organ or tissue source from the selected whole-slide view. Object Classification. Object Classification evaluates whether the model can interpret a specified macro-scale object or ROI. The model is provided with a region on the macro image and asked to classify the predominant pathological or tissue category inside the marked area. Candidate objects are constructed from projected standard annotations. Lesion Grounding. Whole-slide analysis often starts by locating suspicious regions before local inspection. Lesion Grounding asks the model to identify all macro-view regions matching a target lesion or abnormal structure. Multiple independent lesions on the same slide are kept separate to preserve multifocal pathology. Area Estimation. Lesion burden is routinely assessed when estimating tumour extent, necrosis, or other abnormal tissue components. Area Estimation evaluates this ability by asking the model to select the interval that best matches the fraction of visible effective tissue occupied by a target lesion or structure. We compute the fraction over the effective tissue area in the macro image and use four discretized intervals as answer options. Density Localization. Estimating the density of a lesion helps distinguish focal, clustered, and regionally concentrated abnormalities. Density Localization divides the macro image into four quadrants and asks the model to identify the quadrant with the densest target distribution. The target quadrant is determined from both the number of target regions and their occupied area. This group further includes a Slide-FOV trap, where the model receives a macro WSI view but is asked to count cell-level objects that are not visible at that scale. The correct response is the insufficiency option, not an unsupported count. 3.1.2
Region-FOV Fine-Grained Histological Understanding
Region-FOV tasks evaluate fine-grained visual understanding at the microscopic scale, where relevant visual content appears as cellular morphology, tissue microstructures, annotated object instances, and 4
local spatial relationships. These tasks test whether a model can interpret the histological entities visible in a high-resolution field, rather than relying on slide-level recognition alone. Region Classification. At high magnification, local interpretation often begins with identifying the dominant cellular or tissue component in the field. Region Classification evaluates this ability by asking the model to assign the image to its primary morphology category. Region Description. Pathology image understanding also requires concise descriptions of the components visible in a local field. Region Description asks the model to describe the main objects or structures in the image using standardized terminology. This task does not require a full diagnostic report; instead, it tests whether the model can summarize the key histological components in a local view in a reproducible form. Object Classification. A local histology field can contain multiple annotated objects, including different cell types, tissue structures, or pathological components. Object Classification evaluates whether the model can assign the correct category to a specified instance, rather than only describe the image at the field level. Object Grounding. High-magnification interpretation often depends on accurately localizing key cells, glandular structures, necrotic regions, and other morphology-defined entities. Object Grounding evaluates whether the model can localize these fine-grained pathological entities in a high-resolution local image. Object Counting. Counting cells or small structures is central to many microscopic pathology assessments, including mitotic counts, inflammatory-cell evaluation, positive-cell estimation, and grading or scoring workflows. Object Counting asks the model to report the number of visible target-category instances in a local image, testing whether it can make precise quantitative judgments in dense fields with visually similar or partially overlapping objects. Density Localization. The spatial arrangement of cells or structures in a local field can indicate clustering or heterogeneity. Density Localization asks the model to identify the quadrant with the highest density of the target category. Spatial Relation. Local interpretation can also depend on proximity relationships between histological entities at a comparable scale. Spatial Relation asks the model to identify which candidate category is closest to a specified target instance, using annotated local geometry as supervision. This group also includes a Region-FOV trap, where the model receives a high-magnification crop with organ-agnostic morphology, such as malignant cells that can appear similar across organs, but is asked to infer the source organ. Because this information is not reliably available from the local crop alone, the correct response is the insufficiency option rather than an unsupported slide-level inference. 3.2
Standardized Protocol
To keep benchmark supervision tied to reliable visual content, we treat benchmark construction as an auditable conversion problem rather than automatic data repackaging or model-generated relabeling. Our standardized protocol maps public pathology resources with human-supervised labels and annotations into task targets only when the resulting target is label-valid, observable, and answerable. The protocol consists of four stages: data collection, data standardization and quality control, task construction, and dataset splitting and evaluation. Figure 2 summarizes the full protocol, and Figure 3 reports the resulting dataset statistics. 3.2.1
Data collection
We screened public sources, including Zenodo, Kaggle, GitHub, and dataset challenge websites, and retained pathology datasets containing real images with human-supervised labels or annotations. This yielded a source corpus of 23 datasets, comprising 63, 452 image records and 6, 019, 032 raw annotation entries. Patch-level and annotated-region datasets mainly support Region-FOV tasks, whereas whole-slide datasets support Slide-FOV tasks. 5
1 Data Collection Public Pathology
Resources
2 Standardization & QC Source
corpus
Source
Corpus
Collection resources
3 Task Construction Standardized
corpus
Standardized
corpus
Normalize labels, annotations
Kaggle
Github
Manual Review question templates, answer spaces
119 raw labels -> 44 standard labels
Others
Multi-scale
Annotation conversion
Classification
26 formats -> bounding boxes
Region tasks
Multi-organ Orgin WSI-source Clinical labels
Region-FOV
Rule-based filter
Slide-FOV
remove unresolved, background labels
Spatial reasoning FOV-trap
14 VQA-style tasks
308,070 samples
61673 images | 44 labels
7,253,526 bboxes
23 datasets | 63,452 images
28 organs | 6,019,032 annotations
Grounding Counting
Quality control
Human annotations
files
Construct VQA-style tasks
Label mapping Zenodo
Task
4 Splitting & Evaluation Task
Released
Benchmark
files
Release splits and metrics Group-wise leakage control dataset split by tasks Train 70%
Val
10%
Test
20%
Input harmonization Raw images, processed features
Programmatic scoring Dice | Accuracy | F1 | MAE
Entity recall | SRR Released benchmark
Evaluation protocol
Object tion Classifica [54,693]
i 22 on-F 0,1 O 06 V
O Groubject n [56,21ding 1]
t jec on Ob ificati] ss 10 Cla [26,8 Spatial Relation [11,565]
De Loca nsity liz [17,50ation 4]
t jec ing Ob unt 27] Co 41,9 [