EasyLens: A Training-Free Plug-and-Play Subtle-Lesion Representation Amplifier for Medical Vision-Language Models

Unknown

Conceptio › Archive › arXiv CS

arXiv CSopen access

EasyLens: A Training-Free Plug-and-Play Subtle-Lesion Representation Amplifier for Medical Vision-Language Models

Unknown · 2026 · arxiv_cs

arXiv CS · Papers · License: Open Access · 2026

Open Source ↗Direct PDF ↓

artificialintelligence knowledgerepresentation reasoning

artificial intelligence, reasoning, knowledge representation

EasyLens: A Training-Free Plug-and-Play Subtle-Lesion Representation Amplifier for Medical Vision-Language Models

arXiv:2606.06379v1 [cs.CV] 4 Jun 2026

Qiwei Zeng1,∗ Hao Wang2,∗ Jinghao Lin3,∗ Shuchang Ye2 Yuezhe Yang4 Yige Peng4 Haoyuan Che1,† Jinman Kim2,† Lei Bi4,† 1 Jilin University, Changchun, China 2 School of Computer Science, The University of Sydney, Sydney, NSW, Australia 3 ByteDance 4 Institute of Translational Medicine, Shanghai Jiao Tong University, Shanghai, China ∗ Equal contribution. † Corresponding authors [email protected] [email protected] [email protected]

Abstract Medical vision-language models (VLMs) have shown increasing potential for clinical image interpretation, including lesion detection and report generation. However, their practical utility remains limited by insufficient sensitivity to subtle lesions, whose visual evidence is often sparse, low-contrast, and embedded within complex anatomical context. As local visual tokens are aggregated, these weak lesion cues can become underrepresented in global image representations, making them difficult for medical VLMs to recognize. Existing efforts to improve lesion sensitivity mainly rely on medical-domain vision-encoder pre-training, clinical-term-guided alignment, or trainable pathological representation enhancement. Although effective, these approaches usually require additional training or model-specific adaptation and may overfit to particular disease morphologies, limiting their applicability to frozen medical VLMs. To address these limitations, we propose EasyLens, a training-free plug-and-play subtle-lesion representation amplifier for medical VLMs. EasyLens first constructs EasyBank, a pathology-anatomy prototype space that provides lesion-related prototypes and anatomy-aware normal references for comparing suspicious patches against both pathological and normal anatomical patterns. To avoid blindly amplifying normal tissues, EasyTag selects lesion-relevant patches through counterfactual prototype reasoning. To counteract the dilution of subtle lesion cues in global image representations, EasyAmplifier strengthens the selected lesion-relevant patch representations through morphology-guided residual enhancement, thereby increasing their contribution to the global image embedding. Experiments on multiple medical image datasets and frozen medical VLM backbones show that EasyLens consistently improves subtle-lesion detection and outperforms existing encoder-enhancement baselines without model fine-tuning. Code is available at: https://anonymous.4open.science/r/easylens-BEC2

Introduction

Medical vision-language models (VLMs) are increasingly explored for clinical image interpretation, including lesion detection and report generation [10, 20]. As diagnostic decision-support tools, their practical value depends on reliable recognition across a broad spectrum of lesion appearances, rather than only on highly visible abnormalities [5]. Here, salient lesions refer to abnormalities with large Preprint.

Figure 1: EasyLens brings weak subtle-lesion cues into focus in frozen medical VLMs. spatial extent, high contrast, or pronounced morphological signatures, whereas subtle lesions exhibit sparse, low-contrast, or weakly distinguishable visual cues embedded within complex anatomical context [12, 18]. Although salient lesions can often be captured by strong visual patterns, subtle lesions provide weak local evidence that can be easily confused with normal anatomical variation. As local visual tokens are aggregated into global image representations, these weak lesion cues may become underrepresented, causing medical VLMs to miss or under-recognize subtle abnormalities [22]. Therefore, improving the sensitivity of medical VLMs to subtle lesions is critical for making them more reliable in clinical image interpretation [29]. To improve the image interpretation capabilities of medical VLMs, existing approaches have primarily focused on enhancing their visual encoders to extract richer semantic features from medical images and better align image representations with clinical text [30, 1]. Initially, many approaches pre-trained vision encoders to bridge the domain gap on large-scale paired medical images and reports [30], enabling them to capture pathological relevant semantic information when encoding medical images [24]. Although these pre-trained encoders effectively improve the interpretation capability of medical VLMs in the disease diagnosis, these coarse-grained building methods for pre-trained datasets make vision encoders lack the effective guidance to detect subtle lesions [18, 12]. To provide clinically relevant guidance and enhance the pathological semantics of medical image embeddings, subsequent studies introduced domain-specific modules into pretrained vision encoders in medical VLMs. To guide visual encoding with clinical semantics, several studies explicitly inject clinical semantic entities into the image encoding process. MedKLIP [25] extracts diseaserelated clinical entities and their spatial attributes from radiology reports to establish entity-patch correspondence, while KAD [27] constructs clinical entities and relations from reports and uses knowledge-guided disease queries to condition visual representation learning. Complementary to clinical-entity guidance, other studies enhance pathological semantics by refining patch-level visual representations. MLIP [15] improves local image-text alignment through masked local representation learning, while AdaMatch [2] uses adaptive patch matching to capture abnormalities with varying sizes and locations. PLACE [23] further enriches fine-grained visual details through pathological-level alignment and patch correlation modeling. Although these encoder-enhancement methods improve the detection of subtle lesions, their general applicability to current medical VLMs remains limited. First, clinical-semantic guidance is insufficient to cover the complex pathological variations in real clinical scenarios. Since lesions often present with diverse morphological patterns and imaging appearances, supervision based on predefined clinical semantics provides coarse guidance and fails to capture subtle pathological details, leading to confusion among visually similar abnormalities. Specifically, subtle lesions occupy a limited portion of image patches, making fine-grained pathological detail preservation essential for accurate detection. Second, representation-level enhancement typically relies on extensive training or retraining, leading to considerable computational and time costs. In addition, adapting the encoder to specific pathological patterns reduce its sensitivity to other abnormalities, thereby limiting the applicability of medical VLMs across diverse clinical scenarios. To address these limitations, we propose EasyLens, a training-free plug-and-play subtle-lesion representation amplifier for medical VLMs. EasyLens is designed to strengthen weak lesion cues that are preserved in frozen visual representations but become underrepresented during global image aggregation. It first constructs EasyBank, a pathology-anatomy prototype space that provides 2

lesion-related prototypes and anatomy-aware normal references for patch-level comparison. By contrasting suspicious patches with both pathological prototypes and normal anatomical references, EasyBank supports fine-grained discrimination between subtle lesion evidence and normal anatomical variation. Built upon EasyBank, EasyTag selects lesion-relevant patches through counterfactual prototype reasoning, thereby avoiding blind amplification of normal tissues. EasyAmplifier then strengthens the selected lesion-relevant patch representations through morphology-guided residual enhancement, increasing their contribution to the global image embedding while preserving the original visual context. Both modules operate without updating model parameters or requiring lesion annotations at inference time, making EasyLens applicable to frozen medical VLMs. We validate EasyLens on a unified subtle-lesion benchmark built from ReXGroundingCT, LIDC-IDRI, and AbdomenAtlas 3.0 Mini. Experiments across multiple frozen medical VLM backbones show that EasyLens consistently improves subtle-lesion detection and report generation, and outperforms existing encoder-enhancement baselines without model fine-tuning. Our contributions are summarized as follows: (1) We propose EasyLens, a training-free plug-and-play amplifier that improves subtle-lesion recognition in frozen medical VLMs by exploiting latent pathological evidence through prototype-based reasoning. (2) We construct EasyBank, a pathology-anatomy prototype space that organizes lesion-related prototypes and anatomy-aware normal references, and design EasyTag, a counterfactual prototype-guided patch selector that identifies lesion-relevant regions through fine-grained pathological comparison. (3) We introduce EasyAmplifier, a morphology-guided residual semantic amplifier that enhances disease-related morphological semantics in selected patch representations without model fine-tuning or inference-time lesion annotations.

Related Work

2.1

Medical VLMs in Radiology

Recent medical VLMs in radiology have evolved from narrow image-to-report pipelines into more general systems for multi-task interpretation, interactive querying, visual grounding, and structured reasoning. On 2D chest radiographs, recent studies have improved clinical usability by introducing agentic tool use, anatomy-centric reasoning, fine-grained vision-language alignment, and pixelgrounded interaction. For example, MedRAX integrates multimodal tools and large models for complex chest X-ray interpretation [6], AOR performs anatomy-centric region-level reasoning [13], and RadZero strengthens fine-grained alignment for zero-shot classification, grounding, and segmentation [19]. MIMO further extends medical VLMs beyond text-only responses by supporting visual referring inputs and pixel-grounded outputs [3]. These works reflect a shift from holistic report generation toward more interactive and evidence-grounded radiological interpretation. Another major trend extends radiology VLMs from 2D radiographs to volumetric understanding. Argus studies large-scale 3D CT report generation and highlights the importance of vision encoder pretraining, visual token compression, and model/data scaling for high-resolution 3D radiology VLMs [14]. BTB3D further shows that effective volumetric tokenization is more critical than simply enlarging language backbones for scalable 3D medical VLMs [9]. Recent public systems continue this direction through organ-separated CT-language modeling, variable-length 3D visual tokens, and native CT/MRI interpretation [26, 8, 21]. In parallel, clinically grounded radiology VLMs incorporate step-by-step verification, reinforcement learning, workflow-level context, or radiologist gaze to align model reasoning with expert diagnostic procedures [7, 28, 16, 11]. Despite these advances, most radiology VLMs still primarily optimize global image-report alignment, report generation, VQA, or coarse region-level grounding. Sparse subtle-lesion cues can therefore be weakened by holistic diagnostic semantics or compressed visual tokens, leaving a gap between strong global radiology understanding and robust perception of subtle lesion-carrying patches. 2.2

Subtle Lesion Detection in Medical VLMs

Subtle lesions remain challenging for medical VLMs because their diagnostic cues are weak, spatially sparse, and often overwhelmed by surrounding anatomical structures. To improve the perception of such lesions, existing studies have enhanced medical visual representations by injecting clinically 3

(a) EasyBank Construction

Location-wise Anatomy-aware Normal Prototype Space set2 set3

×k

×k anatomical patch sets

VLM Vision Encoder

subtle lesion mask

Any Medical VLM Embedding

Medical Images

anatomical patch space

Global Lesion-related Prototype Space

set1 hyper prototype space

morphological info

patch embedding

Patch-level Image Embeddings

pathological space

average pooling

(b) Training-free Plug-and-Play Inference EasyAmplifier

EasyTag Retrieve top-k from pathological prototype space

Pathological Similarity Score

×n patches

Vision Encoder

selected patch

patch location

Inference Image patch embedding

Retrive top-k from matched anatomical prototype space

morphology

enhancing weights

morphology score

Normal Consistency Score

×n patches

enhanced embedding

selected patch embedding residual enhancing

Figure 2: Overview of EasyLens. (a) EasyBank Construction builds an offline prototype space from CT images and lesion masks. (b) Medical VLM Inference with EasyLens selects lesionrelevant patches and amplifies their visual representations before feeding them into a frozen medical VLM for lesion-aware outputs. relevant guidance into pretrained vision encoders. MedKLIP [25] extracts disease-related clinical entities and their spatial attributes from radiology reports to establish entity-patch correspondence, while KAD [27] constructs clinical entities and relations from reports and uses knowledge-guided disease queries to condition visual representation learning. Beyond clinical-entity guidance, other studies refine patch-level pathological semantics to better capture local abnormalities. MLIP [15] improves local image-text alignment through masked local representation learning, AdaMatch [2] uses adaptive patch matching to capture abnormalities with varying sizes and locations, and PLACE [23] further enriches fine-grained visual details through pathological-level alignment and patch correlation modeling. Another related direction strengthens subtle lesion perception through region-aware grounding and abnormality-sensitive tuning. VividMed [17] and MIMO [3] extend medical VLMs with segmentation, referring, and pixel-grounded outputs, enabling pathological findings to be associated with specific image regions. Reg2RG [4] incorporates region-guided referring and grounding into CT report generation, while UMed-LVLM [31] and MMedPO[32] improve abnormal-region sensitivity through abnormal-aware fine-tuning or clinical-aware preference optimization. These studies push medical VLMs from coarse image-level diagnosis toward more localized subtle lesion understanding. However, most existing methods rely on predefined clinical semantics, explicit grounding modules, region-level supervision, or additional fine-tuning, making them costly and less directly applicable to frozen advanced medical VLMs. In contrast, our work identifies lesion-carrying patch representations and amplifies their pathological semantics at inference time, improving subtle lesion sensitivity without introducing a new detector or retraining the model.

Methodology

3.1

Overview

As shown in Appendix Fig. 4, our preliminary analysis suggests that subtle-lesion cues are not entirely absent from frozen medical VLMs. Instead, they can be partially preserved in patch-level visual representations but become underrepresented when local visual tokens are aggregated into global image representations. Since subtle lesions often occupy small regions, exhibit low contrast, or appear as weakly distinguishable cues within complex anatomical context, their representations can be easily mixed with normal anatomical patterns. This observation motivates us to strengthen lesion-relevant 4

patch representations directly in the frozen visual embedding space, rather than updating the vision encoder through additional training. To this end, we propose EasyLens, a training-free plug-and-play subtle-lesion representation amplifier for medical VLMs. As illustrated in Fig. 2, EasyLens contains three components. First, EasyBank constructs a pathology-anatomy prototype space that stores lesion-related prototypes and anatomy-aware normal references. Second, EasyTag performs counterfactual prototype reasoning to select lesion-relevant patches by contrasting pathological similarity with location-matched normal consistency. Third, EasyAmplifier applies morphology-guided residual enhancement to the selected patch representations, increasing their contribution to the global image embedding while preserving the original visual context. The enhanced visual sequence is then passed to the subsequent components of the frozen medical VLM for downstream lesion-related tasks. 3.2

EasyBank: Pathology-Anatomy Prototype Space

EasyBank provides non-parametric visual references for distinguishing subtle lesion evidence from normal anatomical variation. Given a prototype construction set D = {(xi , mi )}N i=1 , where xi denotes a medical image and mi denotes its lesion mask, we extract patch-level visual representations from the frozen vision encoder: P ×d Zi = Ev (xi ) = [z1i , z2i , . . . , zP , i ]∈R

(1)

where P is the number of image patches and d is the hidden dimension. The lesion masks are used only for constructing EasyBank and are not required during inference. We project each lesion mask onto the patch grid and divide patches into lesion-related and normal anatomical sets: 1 X mi (u), Pi = {p | rip > τ }, Ni = {1, . . . , P } \ Pi , rip = (2) |Ωp | u∈Ωp

where Ωp denotes the image region corresponding to the p-th patch, rip is the lesion occupancy ratio, and τ is the occupancy threshold. EasyBank contains two complementary reference spaces. The first is a global lesion-related prototype space C L , which summarizes recurring pathological patterns. Since directly clustering all lesion patches would bias the prototype space toward large lesions, we first aggregate lesion-related patches within each lesion-containing image: 1 X p hL zi , |Pi | > 0. (3) i = |Pi | p∈Pi

The normalized image-level lesion representations are then clustered into KL lesion-related prototypes: C L = Cluster {ĥL (4) i | |Pi | > 0}, KL . This design assigns equal weight to each lesion-containing image during prototype construction, preventing large lesions from dominating the prototype space simply because they occupy more patches. The second reference space consists of location-wise anatomy-aware normal prototypes. Normal anatomical appearances vary substantially across spatial locations; for example, normal lung parenchyma, mediastinum, pleura, and abdominal organs may have very different visual representations. Therefore, using a single global normal prototype space would mix heterogeneous normal structures and provide ambiguous counterfactual references. EasyBank instead constructs a normal prototype subspace for each patch location: CpA = Cluster ({ẑpi | p ∈ Ni , i = 1, . . . , N }, KA ) ,

p = 1, . . . , P.

(5)

Each CpA summarizes normal anatomical appearances at the same patch location and provides a location-matched reference for later counterfactual comparison. In addition to prototype centers, EasyBank stores a lesion-support memory Lk and a morphology prior Mk for each lesion-related prototype. The support memory provides lesion-related reference 5

embeddings for residual enhancement, while the morphology prior describes the spatial coherence of lesion patterns associated with the prototype. The final EasyBank is summarized as: n o KL B = C L , {CpA }P , {L , M } (6) k k k=1 . p=1 Detailed clustering objectives, support-memory construction, and morphology-prior estimation are provided in Appendix Sec. A.2. 3.3

EasyTag: Counterfactual Lesion-Relevant Patch Selection

Given an inference image, EasyTag selects patches that are likely to contain subtle lesion evidence. For each patch representation zp , EasyTag compares it with two types of references in EasyBank: the global lesion-related prototypes C L and the anatomy-aware normal prototypes CpA at the same patch location. The former measures whether the patch resembles pathological patterns, while the latter evaluates whether the patch can be explained by normal anatomy at the corresponding location. Specifically, EasyTag retrieves the top-M nearest lesion-related prototypes and top-M nearest anatomy-aware normal prototypes for each patch, and computes the lesion similarity score sL p and A normal consistency score sp . The counterfactual lesion relevance score is defined as: ! A sL p − sp ap = σ , (7) τc where σ(·) is the sigmoid function and τc is a temperature parameter. A high score indicates that the patch is close to lesion-related prototypes but poorly explained by its location-matched normal references. In this sense, EasyTag implements counterfactual reasoning: it asks whether a suspicious patch still appears abnormal after being compared with normal anatomical appearances from the same location. Patch-wise scores may be noisy when lesion evidence is weak. Moreover, subtle lesions often appear as spatially coherent local structures rather than isolated patches. To include weak but morphology-consistent lesion cues, EasyTag calibrates the initial scores using morphology priors stored in EasyBank. We first select a high-confidence seed set S0 based on {ap }P p=1 . For each seed patch p, we identify its nearest lesion-related prototype: k ∗ (p) = arg max sim(zp , cL k ). k

(8)

The morphology prior associated with this prototype is then used to propagate confidence from seed patches to spatially coherent neighboring patches: (9) ãq = aq + λ max ap Mk∗ (p) (q − p) , S C = TopKq ãq , p∈S0

where λ controls the calibration strength, Mk∗ (p) (q − p) denotes the morphology-prior value at the relative offset from seed patch p to patch q, and S C is the final candidate set. This calibration allows EasyTag to select sparse but spatially coherent lesion evidence while avoiding blind amplification of normal tissues. The detailed retrieval procedure and score computation are provided in Appendix Sec. A.3. 3.4

EasyAmplifier: Morphology-Guided Residual Enhancement

After EasyTag selects lesion-relevant candidate patches, EasyAmplifier strengthens their representations before they are passed to downstream VLM components. The goal is not to replace the original visual embeddings, but to inject lesion-related residual directions into patches that are supported by both counterfactual evidence and morphology priors. This design preserves the anatomical context encoded by the frozen vision encoder while increasing the contribution of subtle lesion cues to the global image embedding. For each selected candidate patch p ∈ S C , EasyAmplifier retrieves a lesion-related reference from the support memory Lk∗ (p) associated with its recalled lesion prototype. This reference provides a prototype-consistent pathological direction in the frozen embedding space. To extend enhancement 6

Models LLaVA-Med RadFM Lingshu Hulu-Med MedGemma MedGemma1.5 MedKLIP KAD EasyLens

ReX Sel. 1.11 7.78 25.56 27.78 24.44 23.33 17.78 13.33 31.11

Stat. 0.00 46.96 7.48 64.63 40.82 42.86 0.00 0.00 66.67

Gen. 3.93 2.96 0.33 0.88 4.55 4.41 5.15

LIDC Sel. 31.58 21.05 33.83 30.83 21.05 27.82 27.07 16.54 36.09

Stat. 0.00 0.00 20.45 28.79 25.00 20.45 0.00 0.00 30.30

Gen. 33.92 13.75 16.20 23.87 35.94 41.93 45.86

Stat. 1.16 10.47 37.21 51.16 27.91 15.12 0.00 0.00 52.33

Abdomen Sel. 14.42 11.54 53.85 52.88 45.19 49.04 28.85 5.77 55.77

Gen. 36.60 9.60 25.04 39.89 26.11 38.18 40.67

Table 1: Comparison of different medical VLMs on ReX, LIDC, and Abdomen datasets.

Models

w/ EasyLens

LLaVA-Med RadFM Lingshu MedGemma MedGemma1.5

✗ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗ ✓

Stat. 0.00 0.00 46.94 48.98 7.48 7.48 40.82 46.26 42.86 66.67

ReX Sel. Gen. 1.11 3.93 1.11 32.37 7.78 2.96 10.00 4.12 25.56 0.33 27.78 8.51 24.44 4.55 25.56 6.73 23.33 4.41 31.11 5.15

Inf.(s) 1.03 1.13 2.68 2.75 0.46 0.49 0.78 1.48 1.80 2.38

BL-1 11.49 12.22 1.60 2.28 20.06 26.69 13.99 13.81 9.56 10.30

Kvasir-SEG MTR RG-L 22.69 8.61 24.35 8.56 1.31 2.41 1.44 2.61 10.81 20.38 13.89 30.00 25.38 15.59 26.00 16.19 15.87 15.39 16.55 16.18

Inf.(s) 3.33 2.80 0.57 0.49 0.96 0.63 1.43 1.81 1.87 2.01

Table 2: Comparison of different medical VLMs with and without the proposed module on ReX and Kvasir-SEG datasets. beyond isolated high-confidence patches, EasyAmplifier computes, for every patch q, the strongest morphology-consistent support from the selected candidates: p∗ (q) = arg max ãp Mk∗ (p) (q − p), p∈S C

wq = max ãp Mk∗ (p) (q − p). p∈S C

(10)

Here, p∗ (q) identifies the selected candidate that provides the strongest morphology-supported evidence for patch q, and wq measures the strength of this support. If wq is sufficiently large, patch q is considered part of a morphology-consistent lesion region. EasyAmplifier then retrieves a lesion-related reference embedding rq from the support memory of the strongest recalled prototype and updates the patch representation through a score-weighted residual enhancement: q z + αwq (rq − zq ), wq > η, z̄q = (11) zq , wq ≤ η, where α controls the amplification strength and η prevents low-confidence patches from being modified. The residual direction (rq − zq ) moves the patch toward a lesion-related reference while retaining its original embedding as the base representation. Thus, high-confidence and morphologysupported lesion cues are enhanced more strongly, whereas unrelated anatomical regions remain unchanged. Finally, EasyAmplifier outputs the enhanced visual sequence Z̄ = [z̄1 , . . . , z̄P ], which replaces the original patch sequence before being passed to the subsequent medical VLM components. Since EasyLens relies only on prototype retrieval, counterfactual scoring, morphology-guided weighting, and residual enhancement, it requires no gradient-based optimization, no inference-time lesion annotations, and no modification of the frozen VLM. 7

Experiments

4.1

Datasets and Evaluation Tasks

We evaluate EasyLens on two dataset groups spanning the lesion spectrum described in the introduction. The first group focuses on subtle lesions, for which we construct a unified benchmark from ReXGroundingCT, LIDC-IDRI, and AbdomenAtlas 3.0 Mini, denoted as ReX, LIDC, and Abdomen, respectively. This benchmark contains three task types: regional statistics (Stat.), region selection (Sel.), and lesion-aware report generation (Gen.). The second group contains more salient or regular lesion datasets, including MIMIC-CXR, Kvasir-SEG, and BKAI-Polyp, where we evaluate standard medical report generation using BLEU-1, BLEU-4, METEOR, and ROUGE-L. These datasets verify that EasyLens does not over-specialize to subtle abnormalities and preserves general report-generation ability when lesions are visually more apparent. Detailed dataset construction, task definitions, evaluation metrics, and benchmark distributions are provided in Appendix Sec. C.1. 4.2

Implementation and Experimental Setup

Unless otherwise specified, EasyLens denotes MedGemma1.5 equipped with the proposed frozenbackbone inference-time adapter in Table 1, while Table 2 applies the same adapter to each medical VLM backbone. EasyLens keeps the visual encoder, multimodal projector, and language decoder fixed, and only calibrates four interface parameters, i.e., residual strength β, selected-token budget k, local seed budget s, and support retrieval budget m, as detailed in Appendix B.1. We report subtlelesion results in Table 1, backbone transfer in Table 2, general and salient-lesion report generation in Appendix C.2, and interface ablations in Appendix C.5. For report-generation tasks, we use raw benchmark prompts, a 160-token generation limit, and the evaluation-only lesion-aware probe described in Appendix B.2. 4.3

Quantitative Evaluation

EasyLens yields consistent improvements across all subtle-lesion evaluation settings. Table 1 reports results on ReX, LIDC, and Abdomen, covering lesion status recognition, lesion-aware region selection, and report generation. EasyLens achieves the best performance in all nine dataset–task combinations. Relative to the strongest competing baseline in each setting, EasyLens improves ReX by 2.04, 3.33, and 0.60 points on Stat., Sel., and Gen., respectively. The corresponding gains are 1.51, 2.26, and 3.93 points on LIDC, and 1.17, 1.92, and 0.78 points on Abdomen. These results indicate that EasyLens improves the recognition of small, low-contrast, and spatially sparse lesions that are difficult for existing medical VLMs to capture reliably. EasyLens transfers effectively across frozen medical VLM backbones. Table 2 further evaluates EasyLens when attached to different frozen backbones, including LLaVA-Med, RadFM, Lingshu, MedGemma, and MedGemma1.5. On ReX, EasyLens improves report-generation performance for all five backbones, with particularly large gains for LLaVA-Med and Lingshu, increasing Gen. from 3.93 to 32.37 and from 0.33 to 8.51, respectively. For the strongest backbone, MedGemma1.5, EasyLens improves Stat./Sel./Gen. from 42.86/23.33/4.41 to 66.67/31.11/5.15. These results show that the proposed module is not tied to a specific backbone architecture. EasyLens preserves general report-generation capability beyond subtle-lesion tasks. As shown in Appendix C.2, EasyLens remains comparable to MedGemma1.5 on MIMIC-CXR and improves performance on Kvasir-SEG and BKAI-Polyp across all non-zero lexical metrics. Together with the backbone-transfer results in Table 2, these findings suggest that amplifying lesion-relevant visual evidence does not substantially disrupt the original reporting behavior of the frozen VLM, while providing consistent benefits for subtle-lesion perception. 4.4

Ablation Study

EasyTag provides training-free selection competitive with supervised alternatives. As shown in Table 3, EasyTag improves MedGemma1.5 on ReX, increasing Stat. from 42.86 to 66.67, Sel. from 23.33 to 31.33, and Gen. from 4.41 to 5.15. The MLP Selector is a trainable patch-scoring module supervised to identify lesion-relevant tokens. Although it obtains a slightly higher Stat. score of 69.39, its Sel. and Gen. scores decrease to 27.78 and 4.13. This comparison shows that 8

EasyTag Selector Ablation ReX Models Stat. Sel. MedGemma1.5 42.86 23.33 w/ MLP Selector 69.39 27.78 w/ GT Masks 71.43 32.22 w/ EasyTag 66.67 31.33

EasyAmplifier Amplifier Ablation ReX Models Stat. Sel. Gen. MedGemma1.5 42.86 23.33 4.41 w/o Residual 64.63 33.33 4.89 w/o Morphology 57.14 28.89 3.17 w/ EasyAmplifier 66.67 31.11 5.15

Gen. 4.41 4.13 5.36 5.15

Table 3: Ablation studies of the EasyTag and EasyAmplifier on the ReX dataset. Original

EasyTag

EasyAmplifier

Generated Report Backbone VLM: CT scan of the chest demonstrates a normal heart size and mediastinal contours. The lungs are clear with no focal consolidation, pleural effusion, or pneumothorax. The visualized osseous structures are unremarkable. No acute findings are identified.

Patch Select

First Layer Embedding

Backbone VLM w/ EasyLens CT scan of the chest demonstrates a normal heart size and mediastinal contours. A few tiny nonspecific pulmonary nodules are noted bilaterally. No focal consolidation, pleural effusion, or pneumothorax is seen. The visualized osseous structures are unremarkable.

Last Layer Embedding

Figure 3: Case study of EasyLens on subtle-lesion perception. EasyTag achieves competitive performance without training, while providing stronger lesion-aware evidence for selection and generation. The GT-mask setting selects lesion-overlapping tokens using ground-truth masks and serves only as a mask-guided upper-reference. Its results, 71.43 on Stat., 32.22 on Sel., and 5.36 on Gen., are close to EasyTag, further indicating that EasyTag recovers most useful lesion evidence without inference-time masks. EasyAmplifier benefits from residual preservation and morphology-aware enhancement. Table 3 also evaluates the amplification design. Removing the residual formulation reduces Stat. from 66.67 to 64.63 and Gen. from 5.15 to 4.89, while Sel. increases from 31.11 to 33.33. Removing morphology modeling causes a broader drop, reducing Stat. to 57.14, Sel. to 28.89, and Gen. to 3.17. These results suggest that morphology-aware enhancement is the main source of subtle-lesion amplification, while the residual path helps preserve the original visual semantics. Additional interface-level ablations are provided in Appendix C.5. 4.5

Case Study

Qualitative evidence for lesion-relevant token selection and amplification. Figure 3 provides a qualitative example of a CT case containing a subcentimeter pulmonary nodule. The selected tokens produced by EasyTag overlap with the suspected nodule region, supporting the claim that the trainingfree selector can identify micro-lesion-related visual evidence from frozen representations. After EasyAmplifier, the response associated with the selected lesion patches becomes stronger and more spatially concentrated on the nodule region, indicating that morphology-guided residual enhancement increases the lesion-semantic contribution of the relevant patches. This visual change is consistent with the generated report, where the EasyLens-enhanced model mentions the tiny pulmonary nodule, while the baseline model omits this finding.

Conclusion

We presented EasyLens, a training-free plug-and-play amplifier that improves subtle-lesion perception in frozen medical VLMs. EasyLens builds a pathology-anatomy prototype space, selects lesionrelevant tokens through counterfactual prototype reasoning, and strengthens them with morphologyguided residual enhancement. Experiments across subtle-lesion benchmarks and multiple VLM backbones show consistent gains in lesion status recognition, region selection, and lesion-aware 9

report generation without updating model parameters. These results show that EasyLens can expose and amplify weak lesion evidence already encoded in frozen visual tokens, enabling medical VLMs to better recognize and report clinically important micro-lesions without model-specific retraining.

References [1] Benedikt Boecking, Naoto Usuyama, Shruthi Bannur, Daniel C Castro, Anton Schwaighofer, Stephanie Hyland, Maria Wetscherek, Tristan Naumann, Aditya Nori, Javier Alvarez-Valle, et al. Making the most of text semantics to improve biomedical vision–language processing. In European conference on computer vision, pages 1–21. Springer, 2022. [2] Wenting Chen, Linlin Shen, Jingyang Lin, Jiebo Luo, Xiang Li, and Yixuan Yuan. Fine-grained image-text alignment in medical imaging enables explainable cyclic image-report generation. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 9494–9509, 2024. [3] Yanyuan Chen, Dexuan Xu, Yu Huang, Songkun Zhan, Hanpin Wang, Dongxue Chen, Xueping Wang, Meikang Qiu, and Hang Li. Mimo: A medical vision language model with visual referring multimodal input and pixel grounding multimodal output. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 24732–24741, 2025. [4] Zhixuan Chen, Yequan Bie, Haibo Jin, and Hao Chen. Large language model with region-guided referring and grounding for ct report generation. IEEE Transactions on Medical Imaging, 2025. [5] Zijie Cheng, Ariel Yuhan Ong, Siegfried K Wagner, David A Merle, Lie Ju, Hanyuan Zhang, Ruinian Chen, Linze Pang, Boxuan Li, Tiantian He, et al. Understanding the robustness of vision-language models to medical image artefacts. NPJ Digital Medicine, 8(1):727, 2025. [6] Adibvafa Fallahpour, Jun Ma, Alif Munim, Hongwei Lyu, and Bo Wang. MedRAX: Medical reasoning agent for chest x-ray. In Aarti Singh, Maryam Fazel, Daniel Hsu, Simon LacosteJulien, Felix Berkenkamp, Tegan Maharaj, Kiri Wagstaff, and Jerry Zhu, editors, Proceedings of the 42nd International Conference on Machine Learning, volume 267 of Proceedings of Machine Learning Research, pages 15661–15676. PMLR, 13–19 Jul 2025. URL https: //proceedings.mlr.press/v267/fallahpour25a.html. [7] Ziqing Fan, Cheng Liang, Chaoyi Wu, Ya Zhang, Yanfeng Wang, and Weidi Xie. Chestxreasoner: Advancing radiology foundation models with reasoning through step-by-step verification. arXiv preprint arXiv:2504.20930, 2025. [8] Chengyu Fang, Heng Guo, Zheng Jiang, Chunming He, Xiu Li, and Minfeng Xu. Photon: Speedup volume understanding with efficient multimodal large language models. arXiv preprint arXiv:2603.25155, 2026. [9] Ibrahim Ethem Hamamci, Sezgin Er, Suprosanna Shit, Hadrien Reynaud, Dong Yang, Pengfei Guo, Marc Edgar, Daguang Xu, Bernhard Kainz, and Bjoern Menze. Better tokens for better 3d: Advancing vision-language modeling in 3d medical imaging. arXiv preprint arXiv:2510.20639, 2025. [10] Iryna Hartsock and Ghulam Rasool. Vision-language models for medical report generation and visual question answering: A review. Frontiers in artificial intelligence, 7:1430984, 2024. [11] Kinhei Lee, Peiyuan Jing, Zhenxuan Zhang, Yue Yang, Tao Wang, Dominic C Marshall, Yingying Fang, and Guang Yang. Seeing through experts eyes a foundational vision language model trained on radiologists gaze and reasoning. arXiv preprint arXiv:2604.14316, 2026. [12] Qingqiu Li, Xiaohan Yan, Jilan Xu, Runtian Yuan, Yuejie Zhang, Rui Feng, Quanli Shen, Xiaobo Zhang, and Shujun Wang. Anatomical structure-guided medical vision-language pretraining. In International Conference on Medical Image Computing and Computer-Assisted Intervention, pages 80–90. Springer, 2024. [13] Qingqiu Li, Zihang Cui, Seongsu Bae, Jilan Xu, Runtian Yuan, Yuejie Zhang, Rui Feng, Quanli Shen, Xiaobo Zhang, Junjun He, et al. Aor: Anatomical ontology-guided reasoning for medical large multimodal model in chest x-ray interpretation. arXiv preprint arXiv:2505.02830, 2025. [14] Che Liu, Zhongwei Wan, Yuqi Wang, Hui Shen, Haozhe Wang, Kangyu Zheng, Mi Zhang, and Rossella Arcucci. Argus: benchmarking and enhancing vision-language models for 3d radiology report generation. In Findings of the Association for Computational Linguistics: ACL 2025, pages 16448–16460, 2025. 11

[15] Jiarun Liu, Hong-Yu Zhou, Cheng Li, Weijian Huang, Hao Yang, Yong Liang, Guangming Shi, Hairong Zheng, and Shanshan Wang. Mlip: medical language-image pre-training with masked local representation learning. In 2024 IEEE International Symposium on Biomedical Imaging (ISBI), pages 1–5. IEEE, 2024. [16] Kang Liu, Zhuoqi Ma, Siyu Liang, Yunan Li, Xiyue Gao, Chao Liang, Kun Xie, and Qiguang Miao. Seeing like radiologists: Context-and gaze-guided vision-language pretraining for chest x-rays. arXiv preprint arXiv:2603.26049, 2026. [17] Lingxiao Luo, Bingda Tang, Xuanzhong Chen, Rong Han, and Ting Chen. Vividmed: Vision language model with versatile visual grounding for medicine. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 1800–1821, 2025. [18] Yifan Mao, Fei-Fei Xue, Ruixuan Wang, Jianguo Zhang, Wei-Shi Zheng, and Hongmei Liu. Abnormality detection in chest x-ray images using uncertainty prediction autoencoders. In International Conference on Medical Image Computing and Computer-Assisted Intervention, pages 529–538. Springer, 2020. [19] Jonggwon Park, Soobum Kim, Byungmu Yoon, and Kyoyun Choi. Radzero: Similarity-based cross-attention for explainable vision-language alignment in radiology with zero-shot multi-task capability. arXiv e-prints, pages arXiv–2504, 2025. [20] Corentin Royer, Bjoern Menze, and Anjany Sekuboyina. Multimedeval: A benchmark and a toolkit for evaluating medical vision-language models. arXiv preprint arXiv:2402.09262, 2024. [21] Andrew Sellergren, Sahar Kazemzadeh, Tiam Jaroensri, Atilla Kiraly, Madeleine Traverse, Timo Kohlberger, Shawn Xu, Fayaz Jamil, Cían Hughes, Charles Lau, et al. Medgemma technical report. arXiv preprint arXiv:2507.05201, 2025. [22] Matthew Tivnan, Tzu-Cheng Lee, Ruoqiao Zhang, Kirsten Boedeker, Liang Cai, Jeremias Sulam, and J Webster Stayman. Task-driven ct image quality optimization for low-contrast lesion detectability with tunable neural networks. In Medical Imaging 2023: Physics of Medical Imaging, volume 12463, pages 338–343. SPIE, 2023. [23] Jun Wang, Lixing Zhu, Xiaohan Yu, Abhir Bhalerao, and Yulan He. Improving medical visual representation learning with pathological-level cross-modal alignment and correlation exploration. IEEE Journal of Biomedical and Health Informatics, 2025. [24] Zifeng Wang, Zhenbang Wu, Dinesh Agarwal, and Jimeng Sun. Medclip: Contrastive learning from unpaired medical images and text. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 3876–3887, 2022. [25] Chaoyi Wu, Xiaoman Zhang, Ya Zhang, Yanfeng Wang, and Weidi Xie. Medklip: Medical knowledge enhanced language-image pre-training for x-ray diagnosis. In Proceedings of the IEEE/CVF international conference on computer vision, pages 21372–21383, 2023. [26] Kohei Yamamoto and Tomohiro Kikuchi. Totalfm: An organ-separated framework for 3d-ct vision foundation models. arXiv preprint arXiv:2601.00260, 2026. [27] Xiaoman Zhang, Chaoyi Wu, Ya Zhang, Weidi Xie, and Yanfeng Wang. Knowledge-enhanced visual-language pre-training on chest radiology images. Nature Communications, 14(1):4542, 2023. [28] Yabin Zhang, Chong Wang, Yunhe Gao, Jiaming Liu, Maya Varma, Justin Xu, Sophie Ostmeier, Jin Long, Sergios Gatidis, Seena Dehkharghani, et al. A reasoning-enabled vision-language foundation model for chest x-ray interpretation. arXiv preprint arXiv:2604.00493, 2026. [29] Yi Zhang, Yiji Mao, Xuanyu Lu, Xingyu Zou, Hao Huang, Xinyang Li, Jiayue Li, and Haixian Zhang. From single to universal: tiny lesion detection in medical imaging. Artificial Intelligence Review, 57(8):192, 2024. 12

[30] Yuhao Zhang, Hang Jiang, Yasuhide Miura, Christopher D Manning, and Curtis P Langlotz. Contrastive learning of medical visual representations from paired images and text. In Machine learning for healthcare conference, pages 2–25. PMLR, 2022. [31] Yucheng Zhou, Lingran Song, and Jianbing Shen. Improving medical large vision-language models with abnormal-aware feedback. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 12994–13011, 2025. [32] Kangyu Zhu, Peng Xia, Yun Li, Hongtu Zhu, Sheng Wang, and Huaxiu Yao. Mmedpo: Aligning medical vision-language models with clinical-aware multimodal preference optimization. arXiv preprint arXiv:2412.06141, 2024.

Additional Method Details

A.1

Preliminary Analysis of Representation Dilution

Appendix Fig. 4 provides the empirical motivation for EasyLens. The analysis shows that subtle-lesion cues are not completely absent in frozen medical VLM representations. Instead, their patch-level evidence can be weakly encoded but becomes less separable from normal anatomical regions during visual-token aggregation. This observation motivates a training-free representation enhancement strategy that operates directly on patch-level visual embeddings. Lesion Mask

First Layer

Last Layer

Figure 4: Subtle-lesion representation dilution in medical VLMs. A.2

Detailed Construction of EasyBank

EasyBank is a non-parametric pathology-anatomy prototype space constructed from patch-level representations extracted by a frozen medical VLM. Given a prototype construction set D = H×W {(xi , mi )}N is its lesion mask, we extract i=1 , where xi is a medical image and mi ∈ {0, 1} visual patch representations: P ×d Zi = Ev (xi ) = [z1i , z2i , . . . , zP . i ]∈R

(12)

Patch-level lesion assignment. Let Ωp denote the image region covered by the p-th visual patch. We project the lesion mask onto the patch grid by computing the lesion occupancy ratio: 1 X rip = mi (u). (13) |Ωp | u∈Ωp

Patches are divided into lesion-related and normal anatomical regions: Pi = {p | rip > τ },

Ni = {1, . . . , P } \ Pi ,

(14)

where τ is the lesion occupancy threshold. Lesion-related prototype construction. Directly clustering all lesion patches would bias prototype construction toward salient lesions, since they contribute more patch embeddings. To reduce this 14

imbalance, we represent each lesion-containing image using the mean embedding of its lesion-related patches: 1 X p hL zi , |Pi | > 0. (15) i = |Pi | p∈Pi

The normalized lesion representations are collected into: B L = {ĥL i | |Pi | > 0, i = 1, . . . , N }.

(16)

We cluster this buffer into KL lesion-related prototypes: L L C L = {cL 1 , c2 , . . . , cKL }.

(17)

The clustering objective is: min

KL {cL k }k=1 h∈B L

min h − ĉL k 2. k

(18)

Each lesion-related prototype summarizes a recurring pathological representation pattern discovered from frozen VLM embeddings. Lesion-support memory. For each lesion-related prototype, we retain a support memory containing patch-level lesion embeddings from construction samples assigned to that prototype. Let L π(i) = arg min ĥL i − ĉk k

2 2

(19)

denote the prototype assignment of image i. The support memory for prototype k is: Lk = {ẑpi | π(i) = k, p ∈ Pi } .

(20)

This memory provides lesion-related reference embeddings for EasyAmplifier without adding trainable parameters. Morphology prior estimation. Subtle lesions are often spatially coherent rather than isolated at a single patch. For each lesion-related prototype, we estimate a morphology prior from the patchlevel lesion supports of its assigned construction samples. For a relative patch offset δ, the prior is computed as: 1 X X Mk (δ) = 1[p + δ ∈ Pi ], (21) Zk π(i)=k p∈Pi

where 1[·] is the indicator function and Zk is a normalization factor. We normalize Mk to [0, 1]. This prior estimates how likely neighboring patches are to belong to the same lesion pattern when a patch recalls prototype k. Anatomy-aware normal prototype construction. Normal anatomical appearances vary substantially across spatial locations. Therefore, EasyBank constructs location-wise normal prototype subspaces rather than using a single global normal prototype set. For each patch location p, we collect normal embeddings from the same position: BpA = {ẑpi | p ∈ Ni , i = 1, . . . , N }.

(22)

Each location-specific buffer is clustered into KA anatomy-aware normal prototypes: A A CpA = {cA p,1 , cp,2 , . . . , cp,KA }.

(23)

The clustering objective is: min

KA {cA p,k }k=1 z∈B A p

min z − ĉA p,k 2 . k

(24)

The final EasyBank is: n o KL B = C L , {CpA }P p=1 , {Lk , Mk }k=1 .

(25)

It provides lesion-related prototypes, anatomy-aware normal references, lesion-support memories, and morphology priors for EasyTag and EasyAmplifier. 15

A.3

Detailed Counterfactual Prototype Reasoning in EasyTag

Given an inference image, the frozen vision encoder produces patch-level representations: Z = [z1 , z2 , . . . , zP ]. (26) EasyTag evaluates each patch from two complementary perspectives. A lesion-relevant patch should be close to lesion-related prototypes in C L , while being poorly explained by anatomy-aware normal prototypes CpA from the same spatial location. For patch zp , EasyTag retrieves its top-M nearest lesion-related prototypes and top-M nearest anatomy-aware normal prototypes: L p A p RL RA (27) p = TopM(C , z ), p = TopM(Cp , z ). The lesion similarity score and normal consistency score are: 1 X 1 X sim(zp , c), sA sim(zp , c). sL (28) p = p = M M L A c∈Rp

c∈Rp

The initial counterfactual lesion relevance score is: ! A sL p − sp ap = σ . τc

(29)

A normal patch is expected to have high normal consistency with CpA , whereas a lesion-relevant patch should exhibit higher pathological similarity and lower consistency with its normal anatomical reference. Morphology-guided score calibration. Patch-wise counterfactual scores may be noisy when lesion evidence is weak. To include spatially coherent but low-confidence lesion evidence, EasyTag calibrates scores using morphology priors. We first select high-confidence seed patches: S0 = TopK0 {ap }P (30) p=1 . For each seed patch p, its nearest lesion-related prototype is: k ∗ (p) = arg max sim(zp , cL (31) k ). k

The calibrated score for patch q is: ãq = aq + λ max ap Mk∗ (p) (q − p) , p∈S0

(32)

where λ controls the strength of morphology-guided calibration. The final candidate set is: S C = TopKq∈{1,...,P } ãq . (33) This procedure encourages EasyTag to select spatially coherent lesion regions rather than isolated high-scoring patches. A.4

Detailed Morphology-Guided Residual Enhancement in EasyAmplifier

EasyAmplifier strengthens lesion-relevant representations selected by EasyTag. Although the selected patches may already contain lesion evidence, their original embeddings can still be weak due to representation dilution during visual-token aggregation. EasyAmplifier therefore enhances them through residual updates in the visual embedding space, rather than replacing the original features. Candidate-level residual direction. For each selected candidate patch p ∈ S C , EasyTag provides its calibrated score ãp and recalled lesion prototype k ∗ (p). EasyAmplifier retrieves the closest lesion-related reference from the corresponding support memory: rp = arg max sim(zp , u). (34) u∈Lk∗ (p)

The residual direction is:

∆p = rp − zp .

(35) The candidate patch can be enhanced as: zp,+ = zp + αãp ∆p , p ∈ S C , (36) where α controls the residual amplification strength. This update moves the candidate patch toward a lesion-related reference while preserving its original anatomical context. 16

Morphology-guided propagation. Enhancing only selected candidates may miss nearby patches that contain weaker but spatially consistent lesion evidence. EasyAmplifier therefore propagates residual enhancement according to the morphology prior recalled by each selected candidate. For every patch q, we compute: p∗ (q) = arg max ãp Mk∗ (p) (q − p), p∈S C

wq = max ãp Mk∗ (p) (q − p). p∈S C

(37)

When wq > η, patch q is treated as morphologically supported by the selected lesion evidence. Its lesion-related reference is retrieved from the support memory of the strongest recalled prototype: rq = arg

max

u∈Lk∗ (p∗ (q))

sim(zq , u).

(38)

The enhanced representation is: z̄q = zq + αwq (rq − zq ),

wq > η.

(39)

For patches with insufficient morphology support, EasyAmplifier keeps the original representation unchanged: z̄q = zq , wq ≤ η. (40) Enhanced visual sequence.

The final enhanced patch sequence is: Z̄ = [z̄1 , z̄2 , . . . , z̄P ].

(41)

This sequence replaces the original visual embedding sequence before being passed to the subsequent medical VLM components. Since the enhancement uses only prototype retrieval, morphology-guided propagation, and residual injection, it requires no gradient-based optimization, no inference-time lesion annotations, and no modification of the pretrained model.

Additional Implementation Details

B.1

Hyperparameter and Interface Calibration

EasyLens is designed as a plug-and-play adapter for frozen medical VLMs. Its portability comes from separating the shared lesion-enhancement mechanism from lightweight interface calibration. The shared mechanism is unchanged across datasets and backbones: EasyTag selects lesion-relevant visual tokens, and EasyAmplifier applies residual lesion-aware enhancement to the selected tokens. The calibrated parameters only determine the operating point of this fixed mechanism. Interface parameters. The main interface parameters are β, k, s, and m. The residual strength β controls the magnitude of the EasyAmplifier update. The selected-token budget k controls how many candidate visual tokens are retained by EasyTag. The local seed budget s controls how many local visual peaks are used to initialize candidate lesion regions. The retrieval budget m controls how many support tokens are retrieved from EasyBank. Let zi denote the original visual embedding of token i, and let Sk be the selected token set. EasyAmplifier updates selected tokens through a residual form: zi + βγi (ri − zi ), i ∈ Sk , z̄i = zi , i∈ / Sk , where ri is the retrieved lesion-support reference and γi is the calibrated lesion-support score from EasyTag and morphology-guided propagation. This residual formulation preserves the original visual representation while amplifying lesion-relevant evidence. In the method section, the residual coefficient is denoted by α; in experiments, we use β to denote its implementation value. Operating regimes. We use three task-conditioned operating regimes. Counting tasks use a highrecall setting because missing a subtle lesion directly changes the answer. Selection tasks use a balanced precision–recall setting because excessive token amplification may introduce distractor regions. Report-generation tasks use a conservative setting because the decoder must preserve global context and language fluency. 17

For regular lesion report generation, the operating point is further calibrated according to lesion scale and morphology. Datasets with broader or more heterogeneous lesion regions require a larger token budget. For backbone adaptation, the same discrete grid is reused across VLMs, but the final operating point is calibrated to the visual-token geometry of each backbone. This calibration does not update any backbone parameters and does not change the EasyLens architecture. Common settings. Unless otherwise specified, we use the common settings in Table 4. All selected configurations are fixed before evaluation and shared by all test samples in the corresponding dataset–task or backbone–evaluation cell. No per-instance hyperparameter selection is used. Table 4: Common EasyLens settings used in the main experiments.

B.2

Setting

Value

Vision injection layer Support mode Seed mode Seed peak kernel Suppression radius Score threshold Default support score mode Margin weight Shape weight Raw score weight Gap score weight Retrieval temperature Max new tokens for count/select Max new tokens for report generation Report prompt mode Report probe window

1 global top-k local peak top-k 5 1 0.0 shape_only 0.35 1.0 1.0 0.0 0.1 96 160 benchmark_raw first 3 decoding steps

Lesion-aware Report-generation Probe

For subtle-lesion report generation, lexical-overlap metrics alone may not fully capture whether the model recognizes the target lesion. A generated report can be fluent and globally plausible while still omitting the subtle abnormality. Therefore, in addition to generating free-form descriptions with the raw benchmark prompt, we use a fixed lesion-aware decoding probe to measure whether lesion-relevant tokens receive probability mass at the beginning of generation. The probe is evaluation-only. It does not modify the prompt, decoder, visual encoder, EasyLens module, or generation procedure. For all report-generation experiments, we use the raw benchmark prompt, set the maximum generation length to 160 tokens, and compute lesion-token statistics over the first T = 3 decoding steps. Let xi be the input image and qi be the report-generation prompt for sample i. Given the enhanced visual representation Z̄i , the frozen VLM defines a next-token distribution: pi,t (v) = pθ (v | qi , Z̄i , yi,<t ), where v is a vocabulary token and t is the decoding step. For each sample, we define a strict lesiontoken set Li from the benchmark target description. This set contains tokens corresponding to the lesion category, morphology, or lesion-relevant clinical description. At each decoding step, we compute the lesion-token probability mass: X ℓi,t = pi,t (v). v∈Li

To avoid rewarding diffuse probability mass over a large vocabulary, we also use a strict top-K version: X ℓstrict = pi,t (v), i,t v∈Li ∩TopK(pi,t )

where TopK(pi,t ) denotes the set of highest-probability vocabulary tokens at step t. The final lesion-aware report-generation probe score is computed over the first three decoding steps: T

Probe(i) =

100 X strict ℓ , T t=1 i,t

T = 3.

The dataset-level score is the average over all report-generation samples: N

Probe =

1 X Probe(i). N i=1

We focus on the first three decoding steps because early tokens usually determine the main clinical content of a generated report. If the model fails to assign probability mass to lesion-relevant tokens at this stage, later fluent continuation often cannot recover the missed subtle abnormality. This probe therefore measures lesion-awareness at the point where the model commits to the report content, while remaining independent of any training or inference-time modification.

Additional Experimental Details

C.1

Dataset Details and Benchmark Construction

We organize the evaluation datasets into two groups. The first group targets subtle lesions and is built from ReXGroundingCT, LIDC-IDRI, and AbdomenAtlas 3.0 Mini, denoted as ReX, LIDC, and Abdomen, respectively. Following the terminology in the introduction, these cases focus on abnormalities whose visual evidence is spatially sparse, low-contrast, or weakly distinguishable from surrounding anatomical structures. The second group contains more salient or regular lesion datasets, including MIMIC-CXR, Kvasir-SEG, and BKAI-Polyp. These datasets are used to evaluate whether EasyLens preserves standard medical report-generation ability when pathological findings are visually more apparent.

'LVWULEXWLRQRIWKH6XEWOH/HVLRQ%HQFKPDUN *HQ 6HO $EGRPHQ

6WDW

5H;

6WDW

6HO

*HQ

/,'& 6HO

6WDW

'DWDVHWV 5H; /,'& $EGRPHQ 7DVN7\SHV 6WDWLVWLFV 6HOHFWLRQ *HQHUDWLRQ

For the subtle-lesion group, we construct a uni- Figure 5: Dataset and task distribution of the subtlefied benchmark with three task types: regional lesion benchmark. statistics (Stat.), region selection (Sel.), and lesion-aware report generation (Gen.). Fig. 5 summarizes the distribution of the subtle-lesion benchmark across datasets and task types. For Stat. and Sel., each image is divided into a 3 × 3 grid. Grid regions containing pathological patches are treated as positive regions. Stat. evaluates whether the model can correctly recover the number of positive regions, while Sel. evaluates whether it can identify which regions contain pathological evidence. These two tasks test complementary aspects of subtle-lesion perception: Stat. emphasizes regional evidence aggregation, whereas Sel. emphasizes spatial discrimination between lesion cues and visually similar anatomical background. For subtle-lesion Gen., we evaluate whether subtle pathological evidence becomes more accessible during report generation. Instead of relying only on the final generated report, we measure the probability assigned to lesion-related tokens during decoding. This design is important because the enhanced visual embeddings may still be ignored by the decoder if the corresponding subtle-lesion tokens do not enter the final top-ranked generation path. Therefore, lesion-token probability provides a more direct signal of whether encoder-side enhancement actually increases the decoder’s awareness of subtle pathological evidence. 19

Dataset

Metric BL-1 MIMIC- BL-4 CXR MTR RG-L BL-1 Kvasir- BL-4 SEG MTR RG-L BL-1 BKAI- BL-4 Polyp MTR RG-L

LLaVA-Med 0.74 0.04 4.68 8.37 11.49 0.00 22.69 8.61 11.89 0.00 24.21 8.30

RadFM 0.11 0.01 2.24 4.37 1.60 0.00 1.31 2.41 1.86 0.00 1.43 2.03

Hulu-Med 6.97 1.11 11.63 14.45 16.14 0.00 12.49 21.40 15.01 0.00 14.20 21.26

MedGemma 14.39 1.73 13.67 16.89 13.99 0.00 25.38 15.59 14.25 0.00 26.04 16.32

MedGemma1.5 EasyLens 21.24 21.38 3.64 3.63 17.72 17.63 18.98 18.91 9.56 10.30 0.00 0.00 15.87 16.55 15.39 16.18 8.12 8.31 0.00 0.00 17.05 17.54 13.56 13.62

Table 5: Comparison of report generation performance across different medical VLMs on MIMICCXR, Kvasir-SEG, and BKAI-Polyp datasets. For the salient or regular lesion group, we evaluate standard report generation on MIMIC-CXR, Kvasir-SEG, and BKAI-Polyp. Unlike the subtle-lesion Gen. task, this evaluation uses the final generated reports rather than lesion-token probabilities, because the goal is to test general report quality when the target findings are more visually apparent. We report BLEU-1, BLEU-4, METEOR, and ROUGE-L, which evaluate lexical overlap and sentence-level similarity between generated reports and reference reports. These results complement the subtle-lesion benchmark by testing whether EasyLens preserves general medical image interpretation and report-generation ability beyond subtle-lesion scenarios. C.2

Additional Evaluation on General and Salient-Lesion Report Generation

This section complements the main quantitative results by evaluating the benignness of EasyLens beyond subtle-lesion benchmarks. The main paper focuses on subtle-lesion perception, where EasyLens brings consistent gains across ReX, LIDC, and Abdomen. Here, we further evaluate whether the same enhancement mechanism affects standard report generation on general or more salient lesion cases. Table 5 reports results on MIMIC-CXR, Kvasir-SEG, and BKAI-Polyp. EasyLens preserves general chest X-ray reporting ability. On MIMIC-CXR, EasyLens remains highly comparable to its MedGemma1.5 backbone. It slightly improves BL-1 from 21.24 to 21.38, while BL-4, MTR, and RG-L remain nearly unchanged. This shows that enhancing subtle visual evidence does not disturb the global thoracic context or the original language-generation behavior of the frozen backbone. EasyLens also remains effective on salient lesion report generation. On Kvasir-SEG, EasyLens improves MedGemma1.5 from 9.56/15.87/15.39 to 10.30/16.55/16.18 on BL-1/MTR/RG-L. On BKAI-Polyp, it further improves the same backbone from 8.12/17.05/13.56 to 8.31/17.54/13.62. Since BL-4 is zero for all methods on the two polyp datasets, the comparison mainly relies on BL-1, MTR, and RG-L. These results support the conclusion in Sec. 4.3: EasyLens improves subtle-lesion perception without sacrificing broader medical report-generation ability. C.3

Plug-and-Play Interface Calibration

Plug-and-play deployment allows lightweight interface calibration without modifying the host VLM. EasyLens is attached to frozen medical VLMs as an inference-time adapter. Across all backbones, the VLM parameters are frozen, the decoding procedure is unchanged, and the EasyTag– EasyAmplifier architecture remains identical. The only model-dependent step is interface calibration, where a small number of scalar operating parameters are selected from a shared discrete grid. Interface calibration is necessary because different VLMs have different visual token geometries. Medical VLMs vary in embedding scale, lesion-background separability, and the spatial distribution of lesion evidence. A residual strength that is suitable for one encoder may be too weak for another encoder with lower lesion contrast, or too strong for an encoder whose lesion tokens are already sharply localized. Similarly, some backbones concentrate lesion evidence in a few patches, while 20

Layer 1 6 11 16 21 26

Stat. 66.67 48.98 46.94 44.22 46.26 57.14

ReX Sel. 31.11 24.44 25.56 24.44 23.33 25.56

Gen. 5.15 4.74 5.04 4.51 4.41 5.02

Stat. 30.30 24.24 22.73 18.18 17.42 21.21

LIDC Sel. 36.09 30.08 30.08 30.83 31.58 30.83

Gen. 45.86 44.28 41.36 41.51 43.09 40.75

Stat. 52.33 45.35 45.35 45.35 48.84 46.51

Abdomen Sel. 55.77 50.00 49.04 51.92 50.96 47.12

Gen. 40.67 38.77 37.98 37.34 37.89 38.56

Macro 40.44 34.54 33.79 33.14 33.75 34.74

Table 6: Ablation of the vision layer used by EasyLens on ReX, LIDC, and Abdomen. Early injection achieves the best overall performance. others distribute it across more visual tokens. Therefore, these calibrated parameters should be interpreted as adapter-interface parameters, not as model-specific learned components. The calibrated parameters control only the strength and coverage of the same shared mechanism. The main parameters are β, k, s, and m. The residual strength β controls how strongly selected visual tokens are moved toward lesion-support directions. The selector budget k controls how many candidate tokens are retained. The seed budget s determines how many local visual peaks initialize candidate lesion regions. The retrieval budget m controls how many support tokens are retrieved from the EasyBank. These parameters are low-dimensional, interpretable, and selected from a small discrete grid. EasyLens does not tune parameters per test example. For each dataset–task or backbone–task setting, the operating point is fixed before evaluation and then applied to all test samples. Thus, the calibration is an interface-level deployment choice rather than instance-level optimization. No backbone parameters are updated, no model-specific EasyLens architecture is introduced, and no task-specific instruction tuning is performed. Task-conditioned regimes reflect different precision–recall requirements. For status or countingoriented tasks, EasyLens uses a high-recall regime because missing a subtle lesion may directly change the answer. For lesion selection, EasyLens uses a balanced regime that preserves enough candidate regions while avoiding excessive distractor amplification. For report generation, EasyLens uses a conservative-generation regime with stable token coverage and moderate residual strength, so that lesion awareness is improved without destabilizing global context or language fluency. Regular lesion report generation uses the same calibration principle. MIMIC-CXR adopts a conservative setting because reports require broad thoracic context. Kvasir-SEG uses a moderate setting because the lesion is localized and visually salient. BKAI-Polyp uses a larger coverage setting because polyp appearance and boundary morphology are more heterogeneous. These choices follow the same interface-calibration principle used for subtle-lesion tasks. C.4

Common Implementation Settings

Unless otherwise specified, all experiments share the same EasyLens implementation settings. EasyLens is applied to the first vision layer. We use global top-k support retrieval, local-peak seed initialization, a seed peak kernel size of 5, suppression radius 1, margin weight 0.35, shape weight 1.0, score threshold 0.0, and retrieval temperature 0.1. The default support scoring mode is shape_only. For VQA-style tasks, the maximum number of generated tokens is 96. For report-generation tasks, the maximum number of generated tokens is 160, and the lesion-token scoring window uses three decoding steps. C.5

Additional Interface Ablations

These ablations complement the main component ablation in Sec. 4.4. The main paper verifies the effectiveness of EasyTag and EasyAmplifier. Here, we further analyze how the same EasyTag– EasyAmplifier mechanism should be interfaced with frozen VLMs, including the visual injection position and the candidate token budget. These experiments do not introduce additional task-specific modules; they only study the operating interface of the fixed EasyLens design. 21

Setting topk_32_seed4 topk_64_seed8 topk_128_seed16 topk_256_seed32 topk_512_seed48

Stat. 46.26 45.58 54.42 66.67 71.43

ReX Sel. 26.67 26.67 31.11 28.89 32.22

Gen. 5.17 3.93 5.15 4.04 3.86

Stat. 26.52 21.21 26.52 25.76 25.00

LIDC Sel. 30.08 30.08 29.32 31.58 29.32

Gen. 42.40 42.52 43.16 45.46 44.76

Stat. 47.67 52.33 48.84 52.33 48.84

Abdomen Sel. 49.04 48.08 52.88 52.88 51.92

Gen. 39.79 37.96 39.76 38.18 38.87

Table 7: Ablation of the EasyTag candidate token budget k and seed budget s on ReX, LIDC, and Abdomen. Different tasks prefer different operating points due to distinct precision–recall requirements. Top32

Top64

Top128

Top256

Top512

Figure 6: Case study for patch selection in different topk settings C.5.1

Visual Injection Position

Early visual injection is the most effective for preserving subtle lesion evidence. Table 6 compares injecting EasyLens into different vision layers. Injecting at Layer 1 achieves the best results across all nine subtle-lesion metrics and the highest macro score of 40.44. Moving the injection to deeper layers consistently reduces the macro score, e.g., 34.54 at Layer 6, 33.79 at Layer 11, and 33.14 at Layer 16. Although Layer 26 partially recovers the ReX Stat. score, it remains clearly weaker than Layer 1 on the overall macro score and on LIDC/Abdomen. This supports using EasyLens as an early visual-interface adapter. Subtle lesions are low-contrast and spatially localized. If enhancement is applied too late, patch-level lesion evidence may already be compressed by the vision encoder and multimodal alignment layers. Early residual injection amplifies lesion-relevant visual tokens before this compression, while still allowing the frozen VLM to perform its original downstream reasoning. The candidate token budget reflects a precision–recall trade-off. Table 7 varies the EasyTag candidate budget k together with the seed budget s. No single budget dominates all datasets and metrics. On ReX, a larger budget improves Stat. and Sel., while Gen. is better with a compact budget. On LIDC, a larger budget benefits selection and generation, with k = 256 achieving the best Sel. and Gen. On Abdomen, selection prefers a moderate-to-large budget, while generation again benefits from a smaller candidate set. 22

This behavior is consistent with task-conditioned calibration. Status and selection tasks often require high recall over candidate lesion regions, especially when multiple subtle abnormalities may be present. Report generation, however, must preserve lesion evidence without introducing excessive background distractors. When k is too small, the selector may miss weak lesion evidence; when k is too large, additional candidates may dilute the lesion signal. The top-k ablation therefore supports the use of a fixed EasyLens mechanism with lightweight interface calibration rather than a one-size-fits-all operating point. C.5.2

Qualitative Analysis of Candidate Token Budget

The topk case study provides a visual explanation for the budget ablation. Figure 6 visualizes how different candidate budgets affect lesion evidence selection. A small budget tends to focus on the most confident local responses, which can reduce distractors but may miss weak or spatially scattered lesions. A moderate budget preserves more lesion-related alternatives and is therefore more suitable for selection-oriented tasks. An excessively large budget may introduce background structures with similar local appearance, diluting the lesion signal during generation. This qualitative behavior matches the quantitative trend in Table 7. The visualization shows that EasyLens does not rely on arbitrary token amplification. Instead, the candidate budget controls the amount of visual evidence exposed to the frozen VLM. Proper calibration balances lesion recall and background suppression, which is crucial for subtle-lesion grounding and faithful report generation.

Limitations

EasyLens has several limitations. First, although it is training-free at inference time, it relies on an offline prototype bank constructed from reference images and lesion masks. Its effectiveness may therefore depend on the coverage of the prototype bank, especially when target lesions differ in anatomy, scale, or imaging appearance from the reference set. Second, the current implementation uses fixed interface parameters for each dataset–task setting, including the token budget and residual strength. This simple calibration strategy may not be optimal for all cases, particularly when lesion size or visual contrast varies substantially. Third, our experiments are conducted on a limited set of medical VLM backbones and lesion-oriented benchmarks. Broader validation across additional imaging modalities, institutions, and clinical tasks is still needed to assess the generality of the proposed approach.

Licenses and Terms for Existing Assets

This work uses existing public or credentialed medical datasets and publicly released model checkpoints only for research evaluation. We do not claim ownership of any third-party assets. We cite the original creators of the datasets, models, and codebases used in our experiments, and we use each asset according to its corresponding license, data-use agreement, or access policy. We do not redistribute third-party medical images, reports, masks, or model weights. Users who reproduce our experiments should obtain the original assets from their official sources and comply with the corresponding licenses and access requirements. Any supplementary material released with this submission contains only our implementation, configuration files, and evaluation scripts. It does not include protected health information, third-party medical images, clinical reports, segmentation masks, or third-party model weights. Reproduction of the reported experiments requires users to independently obtain the relevant datasets and model checkpoints from their original providers and to comply with the corresponding licenses, data-use agreements, and citation requirements.

Table 8: Existing datasets and model assets used in this work. For assets with restricted or gated access, users must obtain access from the original provider. Asset

Role in this work

License or access terms

ReXGroundingCT

Subtle-lesion CT evaluation and prototype construction

LIDC-IDRI

Lung CT subtle-lesion evaluation and prototype construction

Released under CC BY-NC-SA 4.0 with gated access requiring acceptance of dataset conditions. We use it only for non-commercial research evaluation and do not redistribute the data. Distributed through TCIA. We follow the TCIA Data Usage Policy and the Creative Commons Attribution 3.0 Unported license terms associated with the collection. We cite the original dataset and do not redistribute the data. Released under CC BY-NC-SA 4.0. We use it for non-commercial research evaluation and do not redistribute the data. Accessed through PhysioNet under the PhysioNet Credentialed Health Data License 1.5.0 and the corresponding Data Use Agreement. Access requires credentialed-user approval and required training. We do not share the data, attempt re-identification, or redistribute any images or reports. The official terms restrict use to research and educational purposes, and commercial use requires prior written permission. We cite the dataset paper and do not redistribute the dataset. Accessed as Kaggle competition data subject to the original competition rules. We use it only for research evaluation and do not redistribute the dataset. The official release states that the data, code, and model checkpoints are intended for research use only, with non-commercial restrictions and additional terms inherited from the underlying LLaMA, Vicuna, and GPT-4 resources. It is not intended for clinical care or clinical decision making. The official code repository is released under the MIT License. The public checkpoint page used in our experiments does not specify a separate modelcard license; therefore, we use the checkpoint only for research evaluation and do not redistribute the weights. The official Hugging Face model card lists the MIT License. We use the released checkpoint only for research evaluation and not for clinical deployment. The official repository is released under the Apache License 2.0. We use the released checkpoint and code only for research evaluation and not for clinical deployment. The model weights are governed by the Health AI Developer Foundations terms of use. The accompanying repository/tutorial code is released under Apache License 2.0. We use MedGemma models only as frozen research backbones and do not treat them as clinical-grade diagnostic systems. We cite the official paper and repository. We did not identify a separate explicit license for the released repository or checkpoint at the time of writing; therefore, we use it only for research comparison and do not redistribute its code or weights. The official code repository is released under the MIT License. Any associated model or data resources are used according to their original access terms, and we do not redistribute third-party assets.

AbdomenAtlas Mini

3.0

MIMIC-CXR

Abdominal CT subtle-lesion evaluation and prototype construction Chest X-ray report-generation evaluation

Kvasir-SEG

Polyp report-generation evaluation

BKAI-Polyp / BKAIIGH NeoPolyp

Polyp report-generation evaluation

LLaVA-Med