Knowledge-guided brain tumor segmentation via synchronized visual-semantic-topological prior fusion.

Knowledge-guided brain tumor segmentation via synchronized visual-semantic-topological prior fusion - PMC Skip to main content An official website of the United States government Here's how you know Here's how you know Search Log in Primary site navigation Search Log in Search PMC Full-Text Archive Search in PMC Journal List User Guide As a library, NLM provides access to scientific literature. Inclusion in an NLM database does not imply endorsement of, or agreement with, the contents by NLM or the National Institutes of Health. Learn more: PMC Disclaimer | PMC Copyright Notice BMC Med Imaging . 2026 Jun 3;26:286. doi: 10.1186/s12880-026-02189-3 Knowledge-guided brain tumor segmentation via synchronized visual-semantic-topological prior fusion Ming-Da Zhang 1, ✉ , Kai-Wen Pan 1 Author information Article notes Copyright and License information 1 School of Software, Yunnan University, Chenggong Campus, Kunming, Yunnan Province 650500 China ✉ Corresponding author. Received 2025 Jul 13; Accepted 2026 Jan 22; Collection date 2026. © The Author(s) 2026 Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/ . PMC Copyright notice PMCID: PMC13235202  PMID: 42237259 Abstract Brain tumor segmentation requires precise delineation of hierarchical structures from multi-sequence MRI. However, existing deep learning methods primarily rely on visual features, showing insufficient discriminative power in ambiguous boundary regions. Moreover, they lack explicit integration of medical domain knowledge such as anatomical semantics and geometric topology. We propose a knowledge-guided framework, Synchronized Tri-modal Prior Fusion (STPF), that explicitly integrates three heterogeneous knowledge priors: pathology-driven differential features (T1ce-T1, T2-FLAIR, T1/T2) encoding contrast patterns, unsupervised semantic descriptions transformed into voxel-level guidance via spatialization operators, and geometric constraints extracted through persistent homology analysis. A dual-level fusion architecture dynamically allocates prior weights at the voxel level based on confidence and at the sample level through hypernetwork-generated conditional vectors. Furthermore, nested output heads structurally ensure the hierarchical constraint ET TC WT. STPF achieves a mean Dice coefficient of 0.868 on the BraTS 2020 dataset, surpassing the best baseline by 2.6% points (3.09% relative improvement). Notably, five-fold cross-validation yields coefficients of variation between 0.23% and 0.33%, demonstrating stable performance. Additionally, ablation experiments show that removing topological and semantic priors leads to performance degradation of 2.8% and 3.5%, respectively. By explicitly integrating medical knowledge priors—anatomical semantics and geometric constraints—STPF improves segmentation accuracy in ambiguous boundary regions while demonstrating generalization capability and clinical deployment potential. Supplementary information The online version contains supplementary material available at 10.1186/s12880-026-02189-3. Keywords: Glioma, Multi-modal MRI, Semantic guidance, Topological constraints, Persistent homology, Knowledge-guided learning, Prior fusion Introduction Gliomas account for 80% of primary adult brain tumors [ 1 , 2 ]. Notably, high-grade glioma patients face a poor prognosis, with median survival of less than two years and five-year survival rates below 10%. Consequently, precise tumor segmentation is important for personalized treatment, directly impacting critical clinical decisions such as surgical planning, radiation therapy target delineation, and treatment efficacy assessment [ 3 , 4 ]. To achieve this precision, multi-sequence MRI provides complementary tissue contrast information, thereby establishing a multi-modal data foundation for fine delineation of tumor sub-regions [ 5 ]. However, the tissue heterogeneity, irregular morphology, and ambiguous boundaries of brain tumors, combined with the mandatory nested hierarchical relationship between enhancing tumor (ET), tumor core (TC), and whole tumor (WT) (ET TC WT), make automatic segmentation a major challenge in computational neuro-oncology. In response to this challenge, the Brain Tumor Segmentation (BraTS) challenge series [ 6 – 8 ] has continuously advanced this field, expanding from initial glioma segmentation to meningioma [ 9 ], brain metastases [ 10 ], and pathology image analysis [ 11 ]. Deep learning methods have achieved progress in brain tumor segmentation. In particular, advanced architectures represented by nnU-Net [ 12 ] and Swin UNETR [ 13 ] have reached expert-level performance on BraTS challenges through adaptive configuration and global context modeling. Recent methods include Mamba-based long-range sequence modeling [ 14 , 15 ], diffusion model enhancement [ 16 , 17 ], hybrid masked modeling [ 18 ], and hierarchical context interaction [ 19 ]. Nevertheless, these methods focus primarily on architectural innovations in visual feature space and represent essentially data-driven visual learning. Consequently, they exhibit three systematic limitations: First, insufficient discriminative capability in regions with ambiguous boundaries and unreliable signal intensity, as highlighted by HD distance evaluation studies [ 20 ] and the Metrics Reloaded framework [ 21 ]. Second, network outputs may contain topologically unreasonable structures, thereby producing fatal defects in downstream clinical applications. Third, insufficient utilization of medical domain knowledge—clinicians comprehensively consider semantic information such as anatomical location, morphological features, and growth patterns during diagnosis, as well as geometric constraints that tumors spread along specific pathways while maintaining particular connectivity. These clinical reasoning processes embody knowledge priors that remain underexploited in current data-driven approaches. The development of vision-language models (VLMs) provides new pathways for integrating semantic knowledge priors. For instance, foundation models like BiomedCLIP [ 22 ] demonstrate zero-shot and few-shot learning capabilities through large-scale image-text pair pretraining. Similarly, BioViL-T [ 23 ] proves that even limited medical text can improve visual representation learning. However, integrating semantic knowledge into dense prediction tasks still faces three obstacles: First, medical datasets generally lack high-quality text annotations, thus requiring unsupervised semantic generation pipelines. Second, medical semantics require precise anatomical and pathological terminology rather than vague descriptions. Third, existing methods treat semantic embeddings as global conditional signals, thereby lacking mechanisms to map global descriptions to voxel-level spatial distributions. Attempts by ConTEXTual Net [ 24 ] and SEG-SAM [ 25 ] indicate that semantic guidance can improve segmentation performance; nevertheless, more comprehensive multi-modal prior fusion is still needed. Similarly, while persistent homology theory [ 26 , 27 ] provides tools for capturing geometric structure priors, how to transform abstract topological features into collaboratively optimizable spatial representations remains an open question. To address these challenges, we propose a knowledge-guided framework, Synchronized Tri-modal Prior Fusion (STPF), which achieves synchronized interaction of three heterogeneous knowledge priors—visual, semantic, and topological—at each decoder layer. Note that the “tri-modal prior” in this paper refers to three types of heterogeneous knowledge priors (visual pathology patterns, semantic anatomical descriptions, topological geometric constraints), while the input data consists of four-sequence MRI (T1, T1ce, T2, FLAIR). Beyond improving accuracy on challenging glioma segmentation benchmarks, STPF is designed with robustness in mind: by grounding prior information in anatomical location, coarse morphology, and topological structure rather than scanner-specific intensity statistics, it introduces an inductive bias that is largely invariant to changes in contrast or vendor. Core contributions include: Knowledge-driven multi-modal prior generation and spatialization: We explicitly construct pathology-driven differential features as visual knowledge priors. Additionally, we generate standardized clinical descriptions through unsupervised anomaly detection as semantic knowledge priors. Moreover, we extract geometric constraints using persistent homology as topological knowledge priors, and transform all three-way priors into voxel-level distributions through spatialization operators. Dual-level adaptive prior fusion architecture: At the voxel level, we dynamically allocate tri-modal prior weights based on confidence. Furthermore, at the sample level, we uniformly adjust decoder features through hypernetwork-generated modulation parameters. Unified prior fusion and structured output: We transform heterogeneous knowledge priors into logit space for energy superposition decision-making. Simultaneously, we structurally guarantee the hierarchical relationship ET TC WT through nested output heads. Related work Deep learning for brain tumor segmentation Since its launch in 2012, the BraTS challenge has established standardized evaluation protocols for brain tumor segmentation, defining three hierarchical regions: whole tumor (WT), tumor core (TC), and enhancing tumor (ET) [ 6 ]. Early methods, represented by 3D U-Net [ 28 ], capture multi-scale context through encoder-decoder structures. Subsequently, the nnU-Net [ 12 ] proposed by Isensee et al. achieves a mean Dice coefficient of 0.850 on BraTS 2020 through adaptive preprocessing and dynamic configuration. Following this, Myronenko [ 29 ] introduces autoencoder regularization, while SegResNet [ 30 ] enhances gradient flow through residual connections. Furthermore, inspired by Transformer success in natural language processing, Hatamizadeh et al. propose UNETR [ 31 ], introducing pure Transformer encoders to 3D segmentation. Building upon this, Tang et al.’s Swin UNETR [ 13 ] achieves 0.856 on BraTS 2021 through hierarchical Shifted Window mechanisms. Meanwhile, TransBTS [ 32 ] and TransUNet [ 33 ] explore hybrid CNN-Transformer architectures, and SwinBTS [ 34 ] further optimizes window attention mechanisms. Recent work exhibits more diverse innovation paths. For example, CKD-TransBTS [ 35 ] introduces cross-knowledge distillation, while FDiff-Fusion [ 17 ] combines diffusion models and fuzzy learning. Moreover, the SegMamba series [ 15 ] achieves efficient long-range modeling through state space models, and Diff-UNet [ 16 ] embeds diffusion processes into segmentation frameworks. Although these methods achieve progress in performance metrics, they mainly focus on architectural-level innovations with insufficient integration of medical domain knowledge priors. Consequently, these pure data-driven methods show decreased discriminative capability in ambiguous boundary regions, optimize through soft loss functions, and lack explicit constraints on topological rationality and hierarchical consistency. Semi-supervised and weakly-supervised methods [ 36 – 40 ] demonstrate potential in data-limited scenarios but still require stronger prior constraints. Semantic knowledge integration The development of vision-language models introduces a semantic understanding dimension to medical image analysis. Specifically, CLIP [ 41 ] demonstrates zero-shot classification capabilities through contrastive learning on 400 million image-text pairs. Building on this foundation, BiomedCLIP [ 22 ] is pretrained on 15 million biomedical image-text pairs from PubMed Central, specifically optimized for medical terminology. As a result, it outperforms general models in report generation and disease classification tasks. Similarly, BioViL-T [ 23 ] proves that even limited medical text can improve visual representations. Regarding practical applications, ConTEXTual Net [ 24 ] is pioneering work introducing textual information into segmentation tasks, combining U-Net and T5-Large language models, fusing text and visual features through cross-attention, and achieving a Dice coefficient of 0.716 in pneumothorax segmentation. Likewise, SEG-SAM [ 25 ] integrates LLM-generated descriptions into the SAM framework. However, integrating semantic knowledge priors into dense prediction tasks still faces three challenges: First, mainstream datasets lack text annotations and lack end-to-end automation pipelines. Second, medical semantics require precise anatomical and pathological terminology rather than vague descriptions. Third, existing methods treat semantic embeddings as global conditional signals, thereby lacking mapping mechanisms for voxel-wise spatial distributions. Attempts by ConTEXTual Net and SEG-SAM indicate that semantic guidance can improve segmentation performance; however, the granularity gap between global semantic descriptions and dense prediction tasks remains unresolved. The spatialization operator proposed in this paper transforms global semantic tokens into spatial representations at the same resolution through probability maps and distance field transforms, thereby enabling semantic knowledge priors to function at each voxel location. Topological and geometric constraints Topological data analysis provides geometric constraint tools for medical image segmentation. Specifically, persistent homology theory [ 26 ] captures multi-scale topological features of data by analyzing connectivity changes at different thresholds. The topological loss function proposed by Clough et al. [ 26 ] constrains network output connectivity through Betti numbers, reducing unreasonable isolated regions in cardiac segmentation tasks. Furthermore, a review of topological descriptors in medical imaging [ 27 ] indicates that geometric knowledge priors are important for improving anatomical rationality of segmentation. In addition, graph neural networks [ 42 ] provide another approach to modeling spatial relationships, with the multi-class graph reasoning method at MICCAI 2024 [ 43 ] demonstrating GNN potential in brain tumor segmentation. However, existing methods mainly apply topological constraints as post-processing steps or loss function terms, thereby lacking deep fusion with visual features. Transforming abstract topological knowledge priors into collaboratively optimizable spatial representations remains an open question. This paper processes skeleton graphs extracted by persistent homology through graph attention networks [ 44 ] and designs spatialization operators to map abstract graph structures to dense spatial representations, thus achieving voxel-wise interaction between topological priors and visual features. Meanwhile, unsupervised anomaly detection methods provide a technical foundation for the semantic generation module in this paper. ASC-Net [ 45 ] achieves unsupervised anomaly segmentation through adversarial training, Mahalanobis distance methods [ 46 ] detect brain anomalies using statistical distributions, and diffusion model-based contrastive analysis [ 47 ] demonstrates generative model potential in anomaly detection, thereby enabling the system to generate standardized clinical descriptions without manual annotation. Multi-modal MRI feature fusion Effective utilization of multi-sequence MRI is key to precise brain tumor segmentation. Different sequences exhibit differential sensitivity to pathological tissues: T1-weighted provides anatomical structure, T1 contrast-enhanced highlights blood-brain barrier disruption regions, T2-weighted detects edema and cystic changes, and FLAIR sequence separates peritumoral edema by suppressing cerebrospinal fluid signals. Early fusion strategies concatenate multi-sequences as multi-channel inputs, implicitly assuming networks automatically learn cross-modal relationships; however, simultaneously learning intra-modal features and inter-modal relationships increases optimization difficulty. To address this, MAMC [ 48 ] dynamically weights different sequences through channel attention, NestedFormer [ 49 ] introduces modality-sensitive gating to implement cross-modal feature propagation at different scales, and CMAF-Net [ 50 ] designs cross-modal attention fusion mechanisms. However, these methods treat each sequence as an independent information source, insufficiently considering the pathological knowledge embedded in inter-sequence differential features. Medical imaging indicates that inter-sequence contrast patterns directly correspond to specific pathological features: T1ce-T1 difference highlights contrast agent leakage regions, T2-FLAIR difference separates free water and bound water to delineate whole tumor boundaries, and T1/T2 ratio enhances necrotic tissue contrast. Furthermore, existing fusion strategies treat different modalities as equal partners, ignoring essential differences in prior representations: visual modality is dense spatial features, semantic modality is sparse global descriptions, and topological modality is abstract graph structures. The dual-level fusion architecture proposed in this paper dynamically allocates fusion weights through confidence estimation, thereby adaptively relying on the most reliable knowledge prior in different spatial locations and pathological scenarios. Method This paper proposes a knowledge-guided brain tumor segmentation framework based on synchronized tri-modal prior fusion (STPF), which performs synchronized collaborative decision-making on image-enhanced features, semantic descriptions, and topological constraints at each decoder layer. As shown in Fig. 1 , the method starts from four-sequence MRI data and extracts knowledge priors through three parallel paths. The visual path constructs a multi-scale feature pyramid based on multi-modal difference maps encoding pathological knowledge. Simultaneously, the semantic path generates structured descriptions through anomaly detection, representing anatomical knowledge. In parallel, the topological path extracts geometric constraints using persistent homology analysis, capturing morphological knowledge. After fusing three-way priors into sample-level conditional vectors, they are input to a hypernetwork to generate modulation parameters. The decoder implements dual-level prior fusion: at the global level through sample-level modulation to unify feature representations, and at the local level dynamically allocating fusion weights based on confidence at each voxel location. Finally, the unified prior fusion module transforms all knowledge priors into the same logit space for negotiation, coordinated with nested output heads ensuring hierarchical inclusion relationships, employing stick-breaking parameterization [ 51 ]. Fig. 1. Open in a new tab STPF framework diagram. The left encoder extracts visual features from multi-sequence MRI, obtaining z_vis at the bottom. The purple topological path generates topological tokens via GAT and obtains z_topo through spatialization. The pink semantic path includes candidate region generation and attribute extraction, obtaining semantic tokens through encoder and z_sem through spatialization. Three-way knowledge priors are fused at the bottom through gate and sent to global level–sample-level modulation. The yellow-green section performs attention on , , , then completes voxel-level fusion through spatialization and voxel-level adaptive fusion (estimating weights via ), progressively restoring on the right side and outputting segmentation results Multi-modal prior generation Visual enhancement modality: multi-modal differential feature construction as pathological knowledge prior In medical image analysis, contrast information between different MRI sequences contains lesion features representing pathological knowledge. Based on standard four-sequence MRI (T1, T1ce, T2, FLAIR), this paper explicitly constructs inter-sequence differential features to encode domain-specific pathological patterns. Unless otherwise stated, all experiments assume the availability of these four sequences at both training and inference time, which is consistent with the standard BraTS protocol and allows us to isolate and analyse the effect of prior-guided fusion without conflating it with missing-modality robustness. To capture features from different regions, we define three differential channels as follows: 1 2 3 Specifically, the enhancement difference captures contrast-enhanced regions corresponding to enhancing tumor (ET). Meanwhile, the edema difference uses T2 and FLAIR signal differences to separate edema regions, thereby locating whole tumor (WT) extent. Furthermore, the necrosis contrast utilizes T1 and T2 complementarity to highlight necrotic regions, thus assisting tumor core (TC) segmentation. Collectively, these differential features explicitly encode radiological knowledge about tumor pathology. Following logarithmic compression and quantile clipping, difference maps are normalized to the interval, thereby expanding the original four channels to a seven-channel enhanced representation. Subsequently, a dual-branch encoder processes the original four-sequence channels and differential three channels separately: the original branch captures basic anatomical structure and signal intensity information, while the differential branch focuses on pathological contrast patterns. The two branches interact and fuse through cross-attention mechanisms at each scale, progressively downsampling to extract a multi-scale feature pyramid , which provides visual feature representations for subsequent decoder layers. Topological modality: geometric structure prior extraction as morphological knowledge Visual and semantic modalities provide texture features and region attributes respectively; however, they lack explicit constraints on target geometric morphology. To address this limitation, the topological modality analyzes regional geometric structures through persistent homology [ 26 , 27 ], extracting geometric knowledge priors that encode how tumors grow and spread. The system defines field functions based on the three differential channels: takes high values in enhanced non-necrotic regions, captures both enhanced and necrotic signals, and is primarily dominated by edema signals. Subsequently, topological analysis is performed for each field, retaining stable connected regions while suppressing unstable isolated points and noise. In practice, we first remove tiny isolated components and apply morphological closing to obtain cleaned reference masks that preserve the global structure while reducing spurious fragments, and then compute persistent homology on these cleaned fields. Following this analysis, skeletons are extracted for stable regions and graph structures are constructed, where nodes contain key points and their attributes (field values, stability, local features), and edges record inter-region connections and geometric attributes. Consequently, graph attention networks (GAT) [ 42 – 44 ] jointly process three graph structures to extract node embeddings: 4 obtaining topological tokens. These abstract graph embeddings need to be transformed into spatial representations to interact with visual features. Accordingly, the spatialization operator maps graph structures back to voxel space: 5 For each graph edge , a Gaussian response region is constructed centered on its skeleton with radius as the scale. For each node, a radial basis function is constructed. The distance from each voxel to edge determines the influence strength of that edge on that voxel, thereby transforming the abstract graph structure into a spatial field that can directly interact with visual features. Consequently, this enables geometric knowledge constraints of tumor growth along the skeleton to function at each voxel location. Semantic modality: structured region description and spatialization as anatomical knowledge While visual features can capture texture and intensity patterns, they lack explicit representation of region location and attributes. To address this deficiency, the semantic modality transforms imaging space into structured descriptions that encode anatomical knowledge priors. These semantic priors do not model full radiology reports or diagnostic labels; instead, they summarise coarse spatial, morphological, and intensity attributes of automatically detected abnormal regions into weak, machine-generated descriptors that serve as soft auxiliary cues. Since the BraTS dataset does not include text annotations, this paper designs an automated generation pipeline from images to text, inspired by unsupervised anomaly detection methods [ 47 ]. This pipeline includes three key stages, as shown in Algorithm 1 : first detecting candidate regions through unsupervised methods, then extracting multi-dimensional attributes, and finally assembling standardized text descriptions. Open in a new tab To ensure medical standardization and semantic consistency of generated text, Stage 3 of the algorithm employs a controlled vocabulary mechanism. Table 1 shows standardized vocabularies for each attribute type, with location terms referenced from AAL3 atlas [ 52 ] definitions ensuring anatomical accuracy. Moreover, volume, morphological, and signal features are all discretized into fixed levels, thereby avoiding synonym confusion. The anomaly autoencoders used in Stage 1 are trained solely on MRI intensities, without access to any segmentation labels, and are frozen when generating semantic attributes. They thus act as a deterministic feature transformation of the same input volumes seen by the backbone, rather than an additional source of supervision, and in the cross-validation protocol do not leak label information from held-out folds. Table 1. Controlled vocabulary examples (encoding anatomical knowledge) Attribute Type Vocabulary Laterality Left, Right Location Frontal lobe, Temporal lobe, Parietal lobe, Occipital lobe, Basal ganglia, Thalamus, ... (ref. AAL3 atlas) Volume level Small, Medium, Large Morphology Regular spherical, Ellipsoidal, Irregular Enhancement pattern Prominent, Moderate, Mild Edema extent Extensive, Moderate, Localized Open in a new tab The algorithm and vocabulary jointly achieve unsupervised semantic generation: locating anomalous regions through reconstruction error and symmetry analysis, extracting spatial, morphological, and signal attributes then mapping to discrete labels, and finally assembling standardized descriptions. Notably, this pipeline requires no manual annotation while ensuring both medical accuracy and facilitating subsequent network encoding. Generated natural language descriptions need to be further transformed into spatial distributions that can interact with visual features. This transformation process consists of two steps: text encoding and spatial mapping. In the encoding stage, the system first parses descriptions into JSON structured data, where discrete fields (such as laterality, location, morphology) are mapped to dense vectors through embedding layers, continuous fields (such as precise volume values, compactness) undergo z-score normalization, and after concatenation are input to a multi-layer perceptron (MLP) to obtain semantic token sequences of fixed dimension: 6 After obtaining global semantic tokens, the spatialization operator maps them to spatial distributions at the same resolution as visual features. This operator calculates weights at each voxel location based on region probability map and distance field , thereby achieving spatial localization of semantic attributes: 7 Weight enables strong activation of corresponding semantic tokens for voxels within specific regions, while weights decay for voxels far from that region. Consequently, the semantic encoder is initialized based on Transformers pretrained on medical text corpora, with hierarchical learning rate strategies during fine-tuning and data augmentation (random dropping of non-critical attributes, synonym replacement, etc.) to enhance robustness. In the current design, these semantic priors should be regarded as weak, machine-generated descriptors rather than definitive clinical diagnoses; their influence is modulated by the learned fusion weights, and a formal neuroradiologist reader study to assess the plausibility of the derived attributes and their alignment with radiological reasoning is an important direction for future work. Semantic priors thus transform from global anatomical knowledge vectors into distributional representations that can dynamically interact with visual features at each spatial location. The three paths complete transformation from raw MRI to structured knowledge priors: the visual modality generates multi-scale feature pyramid encoding pathological patterns, the semantic modality generates tokens and their spatialization encoding anatomical knowledge, and the topological modality generates graph tokens and their spatialization encoding geometric constraints. To achieve sample-level global guidance, the system fuses global representations of three-way priors into a unified conditional vector through adaptive gating. First, global pooling is performed on three-way priors separately, then contribution degrees of each modality are dynamically allocated through attention mechanisms: 8 where represent global vector representations of visual, semantic, and topological paths respectively. This conditional vector encodes global characteristics of the current sample and is input to a lightweight hypernetwork to generate modulation parameters: channel-level affine coefficients for feature modulation, and depthwise separable convolution kernels for adaptive filtering. Consequently, this enables the network to adjust feature extraction and fusion strategies based on sample characteristics (such as location, morphology, size). Dual-level fusion architecture The prior generation stage provides multi-modal knowledge information for subsequent fusion. To effectively utilize these priors, this method employs a dual-level fusion architecture: at the local level, three knowledge priors undergo confidence-weighted synchronized fusion at each voxel location, thereby achieving dynamic balance of visual features, semantic constraints, and topological priors. Simultaneously, at the global level, sample-level conditional vectors generate modulation parameters for unified adaptive adjustment of fused features. Decoder layer-wise fusion Each upsampling layer of the decoder performs dual-level fusion. Inputs include visual features from the encoder, semantic tokens and their spatialization , topological tokens and their spatialization , and sample-level conditional vector . The fusion process first establishes information exchange channels among the three knowledge priors. Visual features as Query separately query semantic tokens and topological tokens , obtaining guidance from region attributes and geometric constraints. Simultaneously, semantic and topological tokens as Query query visual features, absorbing visual evidence from the current layer and updating their own representations: 9 This bidirectional interaction enables knowledge priors to be refined layer by layer during decoding—shallow layers based on coarse candidate regions, deep layers incorporating more visual details. Updated generate distributions for the current layer through spatialization operators, thereby providing same-resolution modal representations for subsequent fusion. After obtaining same-resolution tri-modal representations, the system calculates fusion weights at each voxel location . Confidence of three-way features is estimated through lightweight convolution and normalized via softmax: 10 11 where combines feature content with data quality (signal-to-noise ratio, reconstruction error) to output confidence. The three-way weights satisfy , thereby achieving adaptive fusion based on confidence of each knowledge prior at that location. For example, semantic weight is higher when regional structure is clear, topological weight is higher when boundaries are ambiguous but skeleton is stable, and visual weight is higher when signal quality is good. Conversely, when a prior branch becomes inconsistent with the local visual evidence (e.g., noisy semantics or unstable topology), its confidence score decreases and the corresponding weight is reduced, preventing unreliable priors from dominating the decision. After obtaining fused features , the system further applies sample-level adaptive adjustment. Conditional vector generates two types of modulation parameters through a lightweight hypernetwork: channel-level affine coefficients apply uniform transformation to all channels; for the differential branch, additional depthwise separable convolution kernels perform adaptive filtering, with this filter acting on subspaces of differential-related channels through channel grouping mechanisms: 12 This global guidance ensures that all spatial locations of the same sample share a unified adjustment tendency. After skip connections and residual block transmission, all decoder layers progressively restore resolution, finally obtaining high-resolution features . Unified prior fusion module The high-resolution features at the decoder top layer have fused visual representations from all levels; however, semantic and topological priors still need to play a role in final classification decisions. To address this, this paper designs a Unified Prior Fusion (UPF) module to transform all knowledge priors into a unified logit space. UPF has clear division of responsibilities: base logits carry visual evidence accumulated from all fusion layers, while prior energy terms specifically handle semantic and topological constraints, with both achieving decision negotiation through energy addition. For each class , UPF generates prior energy through MLP: 13 14 where is the base logit obtained from final features through a classification head, encoding visual evidence accumulated from all fusion layers. is spatial coordinates, is sample-level conditional vector, and are semantic and topological spatialization features of the final layer. The MLP maps these inputs to prior energy , representing the likelihood that location belongs to class based on current knowledge priors. Final classification logits adopt energy addition for decision-making. When visual evidence is strong and priors are consistent, energy superposition produces high confidence. When visual uncertainty exists but priors are explicit, prior energy compensates for visual insufficiency. When priors conflict but visual discriminability is strong, base logits dominate. To prevent prior energy from overly dominating decisions under low confidence, magnitude regularization is applied to during training (increasing L2 penalty when prior confidence is below threshold), while introducing prior alignment terms in high-confidence regions to gently draw predictions closer to prior soft labels. Nested output head Final classification logits need to be converted to probability distributions while ensuring hierarchical inclusion relationships . Traditional methods penalize predictions violating hierarchy through loss functions, but this is only a soft constraint—networks may still output illegal results. To structurally guarantee output legality, stick-breaking parameterization [ 51 ] embeds hierarchical relationships into the network: 15 16 17 where are intermediate parameters obtained from final logits through affine projection. This parameterization ensures: regardless of values, output probabilities necessarily satisfy . Each subset’s probability is expressed as its parent set probability multiplied by conditional probability: represents the probability of belonging to TC given that it is WT, and represents the probability of belonging to ET given that it is TC. Loss function design Model training adopts a multi-objective joint optimization strategy, with the overall loss function composed of four complementary parts, respectively constraining segmentation accuracy, hierarchical consistency, spatial continuity, and topological rationality. The total loss function is defined as: 18 The basic segmentation loss combines Dice loss [ 53 ] and weighted binary cross-entropy loss (BCE), with the former focusing on region overlap and the latter optimizing voxel-level classification accuracy. Since the hierarchical relationship ET TC WT requires multi-label classification, BCE is employed rather than multi-class cross-entropy. Specifically, , where BCE uses class frequency reciprocal as weights to alleviate class imbalance. Although nested output heads structurally guarantee hierarchical relationships, soft constraint is still introduced during training to further strengthen this characteristic. This loss accelerates convergence of hierarchical constraint learning in early training and provides additional numerical stability throughout training, thereby mitigating transient violations due to gradient fluctuations. This loss penalizes any predictions violating inclusion relationships: 19 where represents voxel space. This term is zero when predictions satisfy hierarchical constraints; otherwise, it imposes linear penalties. Spatial continuity loss encourages predictions to remain smooth within homogeneous regions while maintaining sharpness at true boundaries based on 26-connected neighborhoods. This loss accumulates for each class individually and considers voxel anisotropy: 20 where represents the 26-connected neighborhood of voxel , is the prediction probability for class at location , is the actual physical distance between voxels and , is the gradient magnitude of the fused multi-modal image, and weight coefficient controls boundary preservation strength. This design uses image gradients as boundary indicators: reducing continuity constraints at locations with large gradients (possible boundaries), thereby allowing predictions to produce discontinuities; imposing strong constraints in flat regions to promote smoothness. Topological consistency loss based on geometric priors extracted from topological analysis [ 26 ], constrains consistency of prediction mask connectivity with topological references: 21 where and represent the number of connected components and ring structures (Betti numbers) respectively, and is the topological reference mask for class . Topological reference masks are generated by applying adaptive thresholds on corresponding differential channels: first performing topological analysis on differential channels, sorting all connected regions by stability, retaining the top most stable components ( adaptively determined based on validation set), then performing morphological closing operations on support regions of these components to obtain . This loss acts as a soft regulariser with a modest weight: it encourages predictions to match the global connectivity patterns of the references and prevents excessive isolated regions or unreasonable hole structures, while remaining tolerant to small local perturbations. Experiments Dataset and experimental setup This study employs the BraTS 2020 dataset for evaluation [ 6 ], which contains 369 multi-modal MRI scans, each including four sequences: T1-weighted (T1), T1 contrast-enhanced (T1ce), T2-weighted (T2), and Fluid Attenuated Inversion Recovery (FLAIR). The cohort aggregates clinical data from multiple institutions and scanners with heterogeneous acquisition protocols, providing a realistic setting with inherent distribution shifts. Notably, all images are registered to a unified anatomical template, resampled to 1 mm isotropic resolution with dimensions of 240 240 155 voxels. Furthermore, annotations are completed by professional neuroradiologists, defining three hierarchical regions: enhancing tumor (ET, label 4), tumor core (TC = ET+necrosis, labels 1 + 4), and whole tumor (WT = TC+edema, labels 1 + 2+4). All experiments in this study use all four MRI sequences during training and inference, following the standard BraTS protocol. For evaluation, the dataset employs five-fold cross-validation, using approximately 74 cases as test set, 221 cases as training set, and 74 cases as validation set each time. The preprocessing pipeline includes: (1) Z-score normalization computing brain region mean and standard deviation; (2) spatial cropping to ; (3) differential feature construction, computing three channels , , and , normalized to after logarithmic compression and quantile clipping. In addition, data augmentation during training includes random flipping, rotation ( ), scaling (0.9–1.1), elastic deformation, and intensity transformation. Experiments are conducted on NVIDIA A100 GPU (40GB) using PyTorch 1.12. Core parameters include: AdamW optimizer [ 54 ], learning rate 2e-4, batch size 2, training for 300 epochs, cosine annealing schedule. Moreover, semantic token number , topological token number , fusion module hidden dimension 256. Inference adopts 8-fold test-time augmentation, with post-processing removing isolated components smaller than 500 mm and applying morphological closing. Evaluation metrics are Dice similarity coefficient (DSC) and 95th percentile Hausdorff distance (HD95), following recommendations from the Metrics Reloaded framework [ 21 ], while noting potential misleading aspects of HD95 [ 20 ]. Statistical testing employs paired sample t-tests ( ). Comparison with state-of-the-art methods Table 2 presents quantitative comparison of STPF with state-of-the-art deep learning methods on the BraTS 2020 dataset. Comparison methods cover mainstream architectures, including attention mechanisms (Attention-UNet [ 55 ], DAUnet [ 56 ]), convolutional networks (SegResNet [ 29 ], nnU-Net [ 12 ]), Transformers (TransBTS [ 32 ], TransUNet [ 33 ], UNETR [ 31 ], Swin UNETR [ 13 ], SwinBTS [ 34 ]), and hybrid architectures (CKD-TransBTS [ 35 ], FDiff-Fusion [ 17 ]). Importantly, all methods use official code and recommended parameters, trained and evaluated on the same data to ensure fair comparison. Table 2. Performance comparison on BraTS 2020 dataset. Reports Dice coefficient (DSC, ) and 95% Hausdorff distance (HD95, , unit: mm) for three sub-regions. Best results in bold, second-best underlined. Method WT TC ET Mean DSC DSC HD95 DSC HD95 DSC HD95 Attention-UNet [ 55 ] 0.845 15.174 0.782 16.380 0.716 9.095 0.781 DAUnet [ 56 ] 0.898 5.400 0.830 9.800 0.786 27.600 0.838 SegResNet [ 29 ] 0.915 3.275 0.836 3.769 0.730 3.486 0.827 nnU-Net [ 12 ] 0.912 3.781 0.842 7.771 0.765 18.230 0.840 TransBTS [ 32 ] 0.911 3.360 0.836 2.986 0.740 3.403 0.829 TransUNet [ 33 ] 0.892 3.146 0.825 2.891 0.758 3.621 0.825 UNETR [ 31 ] 0.902 4.305 0.813 5.740 0.732 4.643 0.816 Swin UNETR [ 13 ] 0.917 2.856 0.826 4.314 0.749 4.503 0.830 SwinBTS [ 34 ] 0.891 8.560 0.804 15.780 0.774 26.840 0.823 CKD-TransBTS [ 35 ] 0.898 2.419 0.841 3.447 0.770 3.018 0.836 FDiff-Fusion [ 17 ] 0.905 2.207 0.844 3.311 0.776 2.714 0.842 STPF (Ours) 0.901 4.203 0.864 4.498 0.838 3.217 0.868 Open in a new tab As shown in Table 2 , STPF achieves a mean Dice coefficient of 0.868 on BraTS 2020, thereby surpassing all comparison methods. Compared to the best baseline FDiff-Fusion, it improves by 2.6 percentage points (3.09% relative improvement). Moreover, sub-region analysis shows STPF has advantages in TC (0.864) and ET (0.838) regions, improving by 2.0 and 6.2 percentage points respectively compared to FDiff-Fusion. This is attributed to the synergistic effect of tri-modal knowledge priors: explicitly constructed differential features directly provide lesion contrast information. Additionally, automatically generated semantic descriptions explicitly distinguish enhancement and necrosis boundaries in TC segmentation. Furthermore, topological constraints extracted through persistent homology effectively identify true small lesions in ET regions while suppressing false positives. Taken together with the MRI-only baseline, these results indicate that the visual stream carries most of the predictive power, while semantic and topological priors provide complementary corrections that are particularly beneficial for challenging sub-regions. For ET’s HD95 metric, STPF achieves 3.217 mm, which, although not the best in this column (FDiff-Fusion at 2.714 mm), still demonstrates competitive performance. However, when combining Dice coefficient and cross-fold robustness (see Table 3 ), STPF is superior in comprehensive performance. In the WT region, STPF’s DSC of 0.901 is slightly lower than SegResNet (0.915) and Swin UNETR (0.917). This is mainly because clear T2/FLAIR signals of large-scale edema provide sufficient discriminability for pure visual methods. Nevertheless, Table 3 shows STPF’s cross-case standard deviation is smaller than pure visual baselines, thereby reflecting better robustness. Notably, recently proposed methods such as SegMamba [14] and Diff-UNet [ 16 ] show potential but have not been evaluated under the same experimental settings. Consequently, future work will incorporate comparison with these methods. Table 3. Five-fold cross-validation results (Dice coefficients). Standard deviations in table are cross-fold standard deviations ( ), used to evaluate model stability under different data splits Complete Model (STPF) MRI-Only Visual Baseline Fold WT TC ET WT TC ET Fold-1 0.904 0.867 0.842 0.866 0.804 0.754 Fold-2 0.901 0.863 0.837 0.860 0.796 0.745 Fold-3 0.898 0.862 0.836 0.859 0.795 0.743 Fold-4 0.903 0.866 0.839 0.865 0.802 0.751 Fold-5 0.899 0.864 0.838 0.861 0.798 0.747 Mean 0.901 0.864 0.838 0.862 0.799 0.748 SD 0.003 0.002 0.002 0.003 0.004 0.004 CV 0.33% 0.23% 0.24% 0.35% 0.50% 0.54% 95% CI [0.898, 0.904] [0.862, 0.867] [0.836, 0.841] [0.858, 0.866] [0.794, 0.804] [0.742, 0.754] Open in a new tab Detailed statistical analysis of dice coefficient distribution To comprehensively characterize model performance across cases of varying difficulty, we conducted distributional statistical analysis on the 369-case test set. It should be noted that statistical data in Tables 4 and 5 are based on aggregated out-of-fold prediction results from five-fold cross-validation ( n = 369). Specifically, each case is predicted exactly once as a test set in some fold, and after deduplication, the complete dataset’s performance distribution is obtained. Table 4. Detailed distribution statistics of Dice coefficients ( ). Based on aggregated out-of-fold prediction results from five-fold cross-validation Statistical Metric WT TC ET Central Tendency and Dispersion Mean SD 95% CI [0.890, 0.911] [0.851, 0.880] [0.819, 0.856] Median 0.919 0.895 0.870 Quartiles and Extremes 25th percentile (Q1) 0.860 0.815 0.760 75th percentile (Q3) 0.939 0.935 0.905 Interquartile range (IQR) 0.079 0.121 0.145 Minimum 0.562 0.413 0.264 Maximum 0.964 0.975 0.978 Distribution Shape Skewness Kurtosis 2.147 3.462 0.852 Open in a new tab Table 5. Percentile distribution of Dice coefficients. Based on aggregated out-of-fold prediction results from five-fold cross-validation ( ). Percentile WT TC ET Percentile WT TC ET P1 0.587 0.434 0.285 P50 (Median) 0.919 0.895 0.870 P5 0.781 0.650 0.419 P60 0.932 0.907 0.888 P10 0.820 0.739 0.580 P70 0.937 0.922 0.902 P15 0.844 0.772 0.654 P75 (Q3) 0.939 0.935 0.905 P20 0.853 0.790 0.703 P80 0.942 0.942 0.919 P25 (Q1) 0.860 0.815 0.760 P90 0.952 0.955 0.942 P30 0.870 0.832 0.790 P95 0.958 0.963 0.960 P40 0.893 0.857 0.835 P99 0.963 0.971 0.974 Open in a new tab Distribution statistics reveal STPF’s adaptive characteristics across cases of varying difficulty. Medians ( , , ) are higher than means, combined with negative skewness (−0.601 to −1.199), thereby confirming most cases achieve good results. This is attributed to differential features providing discriminative signals and dual-level fusion automatically elevating reliable modality weights. Interquartile ranges show gradient growth ( , , ), reflecting increasing task difficulty: small IQR for WT benefits from clear edema signals enabling visual modality dominance, while large IQR for ET stems from topological constraint effectiveness depending on initial detection quality. Meanwhile, TC’s high kurtosis (3.462) and negative skewness (−1.199) indicate performance polarization. Extreme value ranges (minimum , , ) reveal that in difficult cases, low confidence across all three priors leads to increased fusion uncertainty. Nevertheless, narrow 95% confidence intervals ( 0.010–0.019) and minority left tail in distribution prove method stability in the majority of scenarios. Dice coefficient percentile distribution analysis To characterize model performance distribution at finer granularity, we computed complete percentile statistics from P1 to P99. Percentile distribution characterizes dual-level fusion’s adaptive behavior across difficulty gradients (Fig. 2 ). In the difficult range ( ), ET presents lowest values ( , ), reflecting small lesion detection challenges, where all three priors are unreliable due to low SNR. In contrast, WT maintains 0.820 at P10, benefiting from clear signals of large-scale edema enabling visual modality dominance. In the typical case range ( ), performance stabilizes at and , where differential features clearly delineate boundaries, semantic descriptions accurately match, and topological structures meet expectations. Consequently, dual-level fusion flexibly allocates weights based on voxel location locally and maintains consistency through sample-level conditional vectors globally. In the high-performance range ( ), approaching human expert levels ( : , , ), these simple cases have typical imaging features. Therefore, the unified prior fusion module transforms tri-modal knowledge predictions into the same logit space with consistent predictions, coordinated with nested output heads ensuring hierarchical relationships, thereby making outputs both accurate and compliant with medical definitions. Fig. 2. Open in a new tab Dice coefficient percentile curves for three tumor sub-regions (WT, TC, ET). Steeper curves indicate performance concentration in most cases, flatter curves indicate broader performance distribution. The steepest WT curve indicates stable edema segmentation, while the flattest ET curve reflects increased difficulty of small lesion detection Five-fold cross-validation stability analysis To evaluate model training stability and result reproducibility, we conducted comparison of complete STPF model and MRI-only visual baseline using five-fold cross-validation. Five-fold cross-validation confirms the stability advantage of the knowledge-guided tri-modal prior fusion architecture (Fig. 3 ). Cross-fold coefficients of variation (CV) for the complete model (0.23% to 0.33%) are lower than the visual baseline (0.35% to 0.54%). This is attributed to prior redundancy and adaptive fusion—when one prior fluctuates due to data distribution differences, local confidence weighting automatically reduces its weight while elevating contributions from other priors for compensation. Five-fold mean comparison shows the complete model consistently surpasses the baseline across all folds, with minimum improvement margins of +3.9 for WT, +6.5 for TC, and +9.0% points for ET. Stability difference is most pronounced in ET region (complete model cross-fold CV = 0.24% vs baseline 0.54%), because topological knowledge constraints extracted through persistent homology provide geometric constraints orthogonal to visual features. Consequently, when visual misjudgment occurs due to noise, topological constraints can identify and suppress unreasonable predictions. Narrow 95% confidence intervals (complete model ) and all five folds achieving p 0.001 significance verify STPF’s generalization capability and clinical deployment reliability. Fig. 3. Open in a new tab Five-fold cross-validation Dice coefficient heatmap, comparing complete STPF model (exp-1) with MRI-only visual baseline (exp-7). Darker colors indicate greater performance differences. Complete model outperforms baseline across all folds and sub-regions, showing higher cross-fold stability Loss function ablation experiments To verify contributions of each component, we progressively removed loss function terms and prior branches to observe performance changes (Table 6 ). Importantly, all ablation configurations use the same training set and hyperparameters. Table 6. Loss function and prior ablation results (with standard deviations). DSC and HD95 show means, Std shows cross-case standard deviations ( ). HD95 unit: mm Configuration WT TC ET Mean DSC Std HD95 Std DSC Std HD95 Std DSC Std HD95 Std DSC HD95 Complete model 0.901 0.101 4.203 8.514 0.864 0.141 4.498 10.027 0.838 0.179 3.217 8.973 0.868 3.973 w/o 0.893 0.106 4.584 8.917 0.857 0.146 4.937 10.483 0.826 0.185 3.648 9.421 0.859 4.390 w/o 0.883 0.108 4.817 9.281 0.854 0.149 5.164 10.748 0.822 0.190 3.782 9.684 0.853 4.588 w/o 0.886 0.110 4.918 9.463 0.848 0.153 5.376 11.082 0.811 0.196 4.127 10.139 0.848 4.807 w/o Topological prior 0.881 0.112 5.127 9.741 0.842 0.156 5.594 11.368 0.808 0.201 4.283 10.447 0.844 5.001 w/o Semantic prior 0.876 0.114 5.318 10.024 0.838 0.158 5.759 11.637 0.801 0.206 4.541 10.829 0.838 5.206 MRI-only visual 0.862 0.119 5.841 10.627 0.799 0.170 6.518 12.483 0.748 0.226 5.387 11.942 0.803 5.915 Open in a new tab Ablation experiments verify contributions of knowledge-guided components (Fig. 4 ). At the loss function level, removing has the greatest impact (mean DSC from 0.868 to 0.848, HD95 deteriorating from 3.973 to 4.807 mm). In contrast, removing reduces mean DSC by 1.5% (to 0.853). Meanwhile, removing causes ET’s HD95 to deteriorate by 13.4% (from 3.217 to 3.648 mm). Fig. 4. Open in a new tab Ablation experiment Dice coefficient change boxplots ( ). Each configuration shows performance degradation relative to complete model ( Dice) and its 95% confidence interval. Removal of semantic prior (w/o SemPrior) and pure visual baseline (only MRI) cause substantial performance loss, thereby proving the important role of knowledge-guided multi-modal prior fusion At the prior level, ablation more directly quantifies knowledge fusion value: removing topological prior increases ET’s HD95 to 4.283 mm (+33.1%). Similarly, removing semantic prior causes ET’s HD95 to deteriorate to 4.541 mm (+41.2%). The MRI-only visual baseline declines substantially (mean DSC to 0.803, −6.5%; ET to 0.748, −9.0%), thereby proving that explicitly introduced differential features, semantic knowledge, and geometric constraints bring improvements. Furthermore, cross-case standard deviation analysis shows complete model ( , , ) smaller than baseline ( , , ), thus indicating multi-modal knowledge prior redundancy enhances robustness. Taken together, these ablations confirm that the visual stream forms the main backbone of the prediction, while semantic and topological priors act as complementary sources of information that stabilise performance and are especially useful in difficult ET and TC cases. Recent semi-supervised and weakly-supervised methods [ 36 – 38 ] may provide additional improvements in data-limited scenarios, worth exploring in combination with STPF in future work. Statistical significance testing To verify statistical significance of performance differences in ablation experiments, we employed paired sample t-tests on 369 test cases (Table 7 ). Table 7. Paired sample t-tests for ablation experiments ( ). Comparison t-statistic p-value 95% CI Cohen’s d w/o vs Complete [ , ] 0.383 w/o vs Complete [ , ] 0.639 w/o vs Complete [ , ] 0.851 w/o Topological prior vs Complete [ , ] 1.022 w/o Semantic prior vs Complete [ , ] 1.278 MRI-only visual vs Complete [ , ] 2.768 Open in a new tab Paired sample t-tests confirm high statistical significance (all p-values ) for differences between all ablation configurations and the complete model, thereby ruling out random chance. Cohen’s d effect size hierarchical distribution quantifies component importance: MRI-only visual baseline’s large effect size ( ) and its confidence interval ( [−0.067, −0.063] completely below zero) indicate knowledge-guided multi-modal prior fusion brings improvements, thus validating the core hypothesis that explicitly introducing medical knowledge and geometric constraints can improve pure visual learning. Large effect sizes at the prior level (semantic , topological ) prove importance of both knowledge priors. Loss function effect sizes show gradient distribution, with spatial continuity loss ( ) verifying the important role of explicit boundary preservation. Meanwhile, medium effect sizes of hierarchical constraints ( ) and topological loss ( ) indicate these regularization terms primarily play reinforcement roles. Robustness analysis To evaluate model tolerance to incomplete or noisy semantic priors, we designed gradient perturbation experiments covering from mild perturbations to completely random (Table 8 ). Table 8. Semantic condition perturbation experiments Perturbation Type WT TC ET Mean Performance Drop Complete semantic prior 0.901 0.864 0.838 0.868 – Attribute loss scenarios Drop 30% attributes 0.896 0.862 0.834 0.864 0.46% Drop 50% attributes 0.891 0.859 0.829 0.860 0.92% Drop 70% attributes 0.884 0.855 0.824 0.854 1.61% Noise injection scenarios Random noise 10% 0.898 0.863 0.836 0.866 0.23% Random noise 30% 0.893 0.859 0.832 0.861 0.81% Completely random semantic 0.877 0.849 0.818 0.848 2.30% Open in a new tab Perturbation experiments verify adaptive tolerance capability of dual-level fusion mechanism (Fig. 5 ). Under attribute loss scenarios, performance shows gradual degradation curve (30% loss −0.46%, 50% −0.92%, 70% −1.61%). This is attributed to local confidence weighting detecting semantic incompleteness and automatically elevating visual and topological weights for compensation. Noise injection scenarios show model suppression capability for erroneous information (30% noise only −0.81%), because confidence estimation detects inconsistencies through cross-modal attention and reduces noise weights. Completely random semantic scenario (performance drop 2.30% to 0.848) verifies automatic degradation capability: when semantics are unreliable, confidence weighting makes semantic weights approach zero, with performance approaching the configuration without semantic prior (0.838) but still higher than pure visual baseline (0.803). Consequently, this proves topological knowledge prior continues providing geometric constraints to maintain basic performance, reflecting multi-modal knowledge prior redundancy value. Taken together, these perturbation experiments show that the network does not blindly trust the semantic stream, but instead learns to discount inconsistent or noisy semantic cues and fall back towards the visual–topological configuration when necessary. The gaze-guided weakly-supervised method proposed by Chen et al. [ 40 ] provides new insights for further enhancing semantic robustness. Fig. 5. Open in a new tab Robustness analysis dumbbell plot under different semantic perturbation scenarios. Each line segment connects the complete model with corresponding perturbation scenario’s mean Dice coefficient, with segment length indicating performance degradation magnitude. Model shows good tolerance to attribute loss and noise injection, maintaining stable performance even under 70% attribute loss or completely random semantics Qualitative visualization analysis of representative cases Analyzing the actual performance of the STPF framework through qualitative visualization of three representative cases in Fig. 6 : Case 1 presents a large mass with ring enhancement and central necrosis structure, Case 2 shows incomplete ring enhancement with unclear boundaries, while Case 3 demonstrates irregular morphology and extensive necrotic regions. Under these three representative scenarios, STPF’s unified prior fusion module and nested output head consistently ensure the structural legality of ET TC WT, with WT and TC regions maintaining stable high scores (Dice 0.88–0.95). Meanwhile, the ET region achieves further improvement to 0.9129 in Case 3 with clear ring enhancement, and maintains 0.8812 performance in Case 2 with discontinuous ring through the completion and denoising effects of topological and semantic knowledge priors. Error pixels (210–388) are primarily concentrated at the ET-TC rim zone and the peripheral FLAIR gradient margins of WT, which is consistent with the discussion in the main text regarding HD95’s susceptibility to boundary thresholds and intensity gradients. Attention heatmaps show a progressive focusing trajectory from broad to precise: encoder layers demonstrate wide-area perception, while decoder layers progressively converge to rim walls and interfaces. This aligns with the voxel-level confidence weighting mechanism , indicating that pathological contrasts from differential channels (T1ce-T1, T2-FLAIR, T1/T2), semantically spatialized weights, and spatial mappings of topological skeletons form effective complementarity at ambiguous boundaries. Consequently, this achieves the Dice improvements reported in Table 2 and the cross-fold stability in Table 3 , while effectively preventing topological errors such as ET fragmentation or detachment from TC. These results validate the advantages of knowledge-guided STPF in ambiguous boundary scenarios. Fig. 6. Open in a new tab Qualitative visualization analysis of representative cases. Each row displays one case, from left to right including: original MRI four sequences (T1, T1ce, T2, FLAIR), ground truth and prediction overlay (GT/Pred overlay, green = WT, red = TC, blue = ET), independent segmentation results and Dice coefficients for three tumor sub-regions, error distribution map (red regions mark error pixel counts), encoder-decoder layer attention heatmaps (jet colormap, red = high activation), Dice bar chart comparison. Case 1 shows large glioma (210 error pixels), case 2 shows ambiguous boundary tumor (342 error pixels), case 3 shows irregular morphology and necrotic regions (388 error pixels). Progressive focusing pattern in attention heatmaps validates dual-level fusion mechanism effectiveness Model scale and computational efficiency analysis To explore tradeoffs among model scale, computational efficiency, and segmentation accuracy, we trained four versions of STPF with different base filter number configurations (See Table 9 ). Table 9. Complete metrics for model scale and computational efficiency. Includes parameters, FLOPs, latency quantiles, throughput, memory, etc Base Params FLOPs Avg Infer SD P95 P99 Sample thru Filters (M) (GFLOPs) (ms) (ms) (ms) (ms) (samples/s) 16 42.63 16,003.79 1,199.18 41.95 1,253.16 1,330.81 1.646 24 79.10 34,490.50 1,214.79 30.30 1,241.65 1,317.61 1.651 32 129.53 60,174.04 1,240.20 34.54 1,248.18 1,361.45 1.617 40 193.90 93,054.40 1,307.50 91.38 1,548.14 1,554.08 1.592 Base Batch thru Avg latency Infer mem Train mem FLOPs/ms Params/mem eff Filters (batches/s) (ms) (GB) (GB) (M/GB) 16 0.823 1,214.84 2.84 5.72 13.35 7.45 24 0.825 1,211.46 3.57 6.79 28.39 11.65 32 0.808 1,236.93 4.34 8.35 48.52 15.51 40 0.796 1,256.38 5.15 9.97 71.17 19.45 Open in a new tab Model scale evaluation reveals Base Filters 32 achieves good balance across performance, speed, resources, and stability. From parameter-performance tradeoff perspective, parameters increase from 42.63 M to 193.90 M (355% growth) across four configurations; however, performance gains show diminishing marginal returns. Specifically, Base Filters 32 achieves 0.868 DSC with 129.53 M parameters, saving 33.2% parameters compared to configuration 40 while losing only 0.3% accuracy. Regarding computational efficiency, configuration 32’s FLOPs/ms ratio of 48.52 is higher than 16 (13.35) and 24 (28.39), thereby indicating full GPU utilization without memory access bottlenecks. Stability metrics show configuration 32’s P99 latency (1361.45 ms) differs from mean by only 9.8%, with moderate SD of 34.54 ms. In contrast, configuration 40 differs by 18.9% with SD surging to 91.38 ms. For resource footprint, configuration 32’s inference memory of 4.34GB and training memory of 8.35GB leave ample margin under A100 (40GB) constraints, with parameter efficiency of 15.51 M/GB balancing expressive capability and memory overhead. Although STPF introduces additional prior branches, this configuration keeps the overall computational footprint comparable to typical 3D transformer-based glioma segmentation models and well within the capabilities of research-grade GPUs; further reducing runtime on standard hospital hardware is mainly an engineering problem (e.g., thinner backbones, mixed-precision inference, pruning or distillation) that is orthogonal to the proposed prior-guided fusion mechanism. Overall, Base Filters 32 achieves an attractive configuration for practical deployment. Discussion While STPF achieves a mean Dice coefficient of 0.868 in fully-supervised scenarios on BraTS 2020 by integrating differential features, semantic descriptions, and topological constraints, its adaptation capability in data-limited environments still requires verification. Specifically, the current framework’s semantic generation module employs unsupervised anomaly detection strategies, which can theoretically utilize unlabeled data to generate pseudo-semantic labels in semi-supervised scenarios [ 47 ]. Moreover, when combined with consistency regularization [ 36 ] and dynamic contrastive learning [ 37 ], this approach can further improve model robustness under annotation scarcity. In few-shot learning scenarios, topological knowledge priors extracted through persistent homology analysis [ 26 , 27 ] rely on stable geometric pattern recognition. However, limited training samples may lead to unstable skeleton extraction. Additionally, confidence estimation in dual-level fusion requires sufficient samples to calibrate voxel-level weight allocation. Fortunately, these issues can be mitigated through meta-learning or prototypical networks [ 38 ]. Regarding weakly-supervised scenarios, they provide opportunities for STPF: region-level attribute descriptions in semantic modality (such as “large irregular frontal lobe tumor”) naturally correspond to coarse-grained annotations. Furthermore, topological constraints can infer complete morphology from scribble annotations [ 38 ] or gaze/attention signals [ 40 ]. Consequently, dual-level fusion can adaptively rely on more reliable priors in low-confidence regions. Evaluating STPF in these scenarios can not only quantify multi-modal knowledge prior redundancy’s compensatory capability when supervision signals weaken but also reveal complementary mechanisms of different priors under sparse annotation conditions. From a generalisation perspective, the proposed STPF framework is conceptually well suited to heterogeneous clinical data such as the multi-institutional, multi-scanner cohorts used in the BraTS challenges [ 6 , 7 ]. First, the anatomical and semantic priors are expressed in a normalised anatomical space (e.g., MNI space with AAL3 parcellation [ 52 ]) and in terms of relative region-level attributes (location, laterality, volume category, morphology), which are expected to be more stable across scanners and acquisition protocols than raw intensities. Second, the topological priors operate on connectivity and nesting relationships (for example, whether tumour components form additional holes or isolated fragments) rather than on absolute signal values, consistent with recommendations from topological data analysis for medical imaging [ 26 , 27 ]. Third, the voxel-level fusion weights and the sample-level gate are learned jointly with the segmentation objective and can reweight visual, semantic, and topological streams depending on their local consistency. Together, these design choices encourage the model to rely on structural cues that transfer across acquisition settings, while allowing the fusion module to adaptively compensate for moderate domain shifts within a heterogeneous clinical cohort. While the current framework integrates MRI multi-sequences, natural language semantics, and geometric topology, it remains limited to specific configurations for brain gliomas. Notably, the BraTS challenge has expanded to diverse tasks including meningioma radiotherapy planning [ 9 ], brain metastases [ 10 ], and pathology image analysis [ 11 ], each with different clinical requirements and technical challenges. For instance, meningiomas require precise delineation of dural invasion, brain metastases involve detecting multiple small lesions, and pathology analysis demands cell-level segmentation. Looking forward, broader knowledge prior integration includes metabolic activity captured by functional imaging (fMRI, PET), white matter connections revealed by diffusion tensor imaging, and molecular subtypes provided by genomics. Importantly, these heterogeneous data sources can collaboratively make decisions through STPF’s unified prior fusion module in logit space. Another practical extension is to relax the assumption that all four MRI sequences are always available and combine STPF with modality-dropout training or cross-modal completion modules to handle missing-sequence scenarios commonly encountered in clinical practice [ 57 , 58 ]. Furthermore, STPF is naturally compatible with federated learning for multi-institutional neuroimaging, since anomaly detection, semantic attribute extraction, and topological skeletonisation can be performed locally at each site and only the segmentation and fusion parameters need to be shared, as discussed in recent work on federated learning in medicine and in reviews of brain tumour segmentation with federated learning techniques [ 59 – 61 ]. More promising directions involve deep integration with medical foundation models: pretrained models like BiomedCLIP [ 22 ] can replace the current lightweight semantic encoder, thereby providing domain knowledge. Additionally, embedding segmentation tasks into LLM-driven clinical reasoning workflows can achieve end-to-end parsing from imaging reports to structured attributes. Ultimately, achieving zero-shot cross-task transfer through instruction fine-tuning becomes possible. This requires transitioning from current task-specific explicit prior construction toward general modality alignment and cross-modal reasoning. Recent works like SegMamba [ 15 ], Diff-UNet [ 16 ], and HybridMIM [ 18 ] provide technical foundations for this transition. Conclusion This paper proposes a knowledge-guided brain tumor segmentation framework based on synchronized tri-modal prior fusion (STPF), achieving accurate and structurally reasonable tumor region segmentation through explicit integration of differential features, semantic knowledge, and geometric constraints. Core innovations include knowledge-driven multi-modal prior generation modules, dual-level adaptive fusion architecture, and unified prior fusion with nested output heads. Experimentally, STPF achieves a mean Dice coefficient of 0.868 on the BraTS 2020 dataset, surpassing the best baseline by 2.6% points. Moreover, five-fold cross-validation coefficients of variation of 0.23% to 0.33% prove robustness. Additionally, ablation experiments verify effectiveness of each knowledge prior module. More importantly, STPF’s success demonstrates the potential of knowledge-guided paradigms—through explicit modeling of medical knowledge priors rather than pure data-driven approaches, discriminative capability can be improved in ambiguous boundary regions. Looking ahead, future work will focus on three directions: first, verifying generalization capability in data-limited scenarios including semi-supervised [ 36 ], few-shot [ 38 ], and weakly-supervised [ 40 ]; second, extending to multi-source heterogeneous modalities and exploring deep integration with medical foundation models [ 22 ]; third, evaluating adaptability on BraTS 2025 new tasks [ 8 ] and integrating with methods like SegMamba [ 15 ] and Diff-UNet [ 16 ]. In conclusion, this research provides a knowledge-guided paradigm for constructing more intelligent and reliable medical image analysis systems. Electronic supplementary material Below is the link to the electronic supplementary material. Supplementary Material 1 (175.6KB, pdf) Acknowledgements Not applicable. Abbreviations 3D Three-Dimensional AAL3 Automated Anatomical Labeling 3 BCE Binary Cross-Entropy BraTS Brain Tumor Segmentation CI Confidence Interval CLIP Contrastive Language-Image Pre-training CNN Convolutional Neural Network CV Coefficient of Variation DSC Dice Similarity Coefficient ET Enhancing Tumor FLAIR Fluid Attenuated Inversion Recovery FLOPs Floating Point Operations fMRI functional Magnetic Resonance Imaging GAT Graph Attention Network GNN Graph Neural Network GPU Graphics Processing Unit HD95 95th percentile Hausdorff Distance IQR Interquartile Range LLM Large Language Model MLP Multi-Layer Perceptron MRI Magnetic Resonance Imaging PET Positron Emission Tomography SAM Segment Anything Model SD Standard Deviation SNR Signalto-Noise Ratio STPF Synchronized Tri-modal Prior Fusion T1 T1-weighted T1ce T1 contrast-enhanced T2 T2-weighted TC Tumor Core UPF Unified Prior Fusion VLM Vision-Language Model WT Whole Tumor Author contributions M-DZ: Conceptualization, Methodology, Software, Formal analysis, Investigation, Writing – original draft, Writing – review & editing, Visualization. K-WP: Data curation, Initial experiments. All authors read and approved the final manuscript. Funding This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors. Data availability The Brain Tumor Segmentation (BraTS) 2020 dataset used in this study is publicly available through registration at the CBICA Image Processing Portal ( https://ipp.cbica.upenn.edu ). The dataset can be accessed following the data request procedure described at https://www.med.upenn.edu/cbica/brats2020/registration.html . Declarations Ethics approval and consent to participate This study used publicly available anonymized neuroimaging data from the BraTS 2020 challenge dataset. No additional ethics approval was required as the dataset was previously approved by the institutional review boards of the contributing institutions and all patient identifiers were removed prior to public release. Consent to participate was obtained by the original data collectors at the contributing institutions. Consent for publication Not applicable. This study uses publicly available anonymized data with no identifiable patient information. Competing interests The authors declare that they have no competing interests. Footnotes Publisher’s Note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. References 1. Voigtlaender S, Nelson TA, Karschnia P, Vaios EJ, Kim MM, Lohmann P, et al. Value of artificial intelligence in neuro-oncology. Lancet Digit Health. 2025. Published online Aug 2025, 10.1016/j.landig.2025.100876 Art. No. 100876. [ DOI ] [ PMC free article ] [ PubMed ] 2. Ostrom QT, Price M, Neff C, Cioffi G, Waite KA, Kruchko C, et al. CBTRUS statistical report: primary brain and other Central nervous system tumors diagnosed in the United States in 2015–2019. Neuro-Oncology. 2022;24(Suppl 5):1–95. 10.1093/neuonc/noac202 [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ] 3. Wan B, Hu B, Zhao M, Li K, Ye X. Deep learning-based magnetic resonance image segmentation technique for application to glioma. Front Med. 2023;10:1172767. 10.3389/fmed.2023.1172767 [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ] 4. Bianconi A, Rossi LF, Bonada M, Zeppa P, Nico E, De Marco R, et al. Deep learning-based algorithm for postoperative glioblastoma MRI segmentation: a promising new tool for tumor burden assessment. Brain Inf. 2023;10(26). 10.1186/s40708-023-00207-6 [ DOI ] [ PMC free article ] [ PubMed ] 5. Abbad Andaloussi M, Maser R, Hertel F, Lamoline F, Husch AD. Exploring adult glioma through MRI: a review of publicly available datasets to guide efficient image analysis. Neuro-Oncol Adv. 2025;7(1):197. 10.1093/noajnl/vdae197. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ] 6. Baid U, Ghodasara S, Mohan S, Bilello M, Colak E, Calabrese E, et al. The RSNA-ASNR-MICCAI BraTS 2021 benchmark on brain tumor segmentation and radiogenomic classification. arXiv preprint arXiv:2107.02314. 2021. 7. Bonato B, Nanni L, Bertoldo A. Advancing precision: a comprehensive review of MRI segmentation datasets from BraTS challenges. Sensors. 2025;25(6):1838. 10.3390/s25061838. [ DOI ] [ PMC free article ] [ PubMed ] 8. Bakas S, Pati S, Menze B, Reyes M: B. “Lighthouse” challenge: brain metastases, meningiomas, and beyond. BraTS challenge announcement (synapse repository). 2025 (2025. https://www.synapse.org/#!Synapse:syn64153130 . Accessed 2025. 9. LaBella D, Baid U, Khanna O, et al. Analysis of the BraTS 2023 intracranial meningioma segmentation challenge. Machine learning in biomedical imaging (MELBA). 2025;2(3). 10.59275/j.melba.2025-003 10. Moawad AW, Janas A, Baid U, et al. The brain tumor segmentation (BraTS-METS) challenge 2023: brain metastasis segmentation on pre-treatment MRI. arXiv preprint arXiv:2306.00838 (2023). https://arxiv.org/abs/2306.00838 . 11. Myronenko A, Siddiquee MMR, He Y, et al. Auto3DSeg for Brain Tumor Segmentation from 3D MRI in BraTS 2023 Challenge. arXiv preprint arXiv:2510.25058. 2025. https://arxiv.org/abs/2510.25058 . 12. Isensee F, Jaeger PF, Kohl SAA, Petersen J, Maier-Hein KH. nnUnet: a self-configuring method for deep learning-based biomedical image segmentation. Nat Methods. 2021;18(2):203–11. 10.1038/s41592-020-01008-z. [ DOI ] [ PubMed ] 13. Hatamizadeh A, Nath V, Tang Y, Yang D, Roth HR, Xu D. Swin UNETR: Swin Transformers for semantic segmentation of brain tumors in MRI images. arXiv preprint arXiv:2201.01266. 2022. 14. Xing Z, Ye T, Yang Y, Liu G, Zhu L. SegMamba: long-range sequential modeling Mamba for 3D medical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention – MICCAI 2024. Lecture Notes in Computer Science, vol. 15009. Springer; 2024. p. 578–588. 10.1007/978-3-031-72111-3_54. 15. Xing Z, Ye T, Yang Y, Cai D, Gai B, Wu XJ, Gao F, Zhu L. SegMamba-V2: long-range sequential modeling mamba for general 3-D medical image segmentation. IEEE Trans Med Imaging. 2026. 10.1109/TMI.2025.3589797. [ DOI ] [ PubMed ] 16. Xing Z, Wan L, Fu H, Yang G, Zhu L. Diff-UNet: a diffusion embedded network for robust 3D medical image segmentation. Med Image Anal. 2025;105:103654. 10.1016/j.media.2025.103654. [ DOI ] [ PubMed ] [ Google Scholar ] 17. Ding W, Fan C, Wang G, Shi Y, Liu F. Fdiff-fusion: denoising diffusion fusion network based on fuzzy learning for 3D medical image segmentation. Inf Fusion. 2024;112:102540. 10.1016/j.inffus.2023.102540. [ Google Scholar ] 18. Xing Z, Wan L, Fu H, Yang G, Zhu L. HybridMIM: a hybrid masked image modeling framework for 3D medical image segmentation. IEEE J Biomed Health Inf. 2024;28(4):2115–25. 10.1109/JBHI.2023.3290285. [ DOI ] [ PubMed ] [ Google Scholar ] 19. Cheng Z, Yuan D, Zhang W, Lukasiewicz T. Effective and efficient medical image segmentation with hierarchical context interaction. Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV). 2025, pp. 9396–406. 10.1109/WACV61041.2025.00910. 20. Podobnik G, Vrtovec T. Understanding implementation pitfalls of distancebased metrics for image segmentation. arXiv preprint arXiv:2410.02630. 2024. 21. Jaeger PF, Zweig KA, Burke E, Schöffski O, Koenig M, Zimmer VA, et al. Metrics reloaded: recommendations for image analysis validation. Nat Methods. 2024;21(2):195–212. 10.1038/s41592-023-02151-z. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ] 22. Zhang S, Xu Y, Usuyama N, Xu H, Bagga J, Tinn R, et al. BiomedCLIP: a multimodal biomedical foundation model pretrained from 15 million image–text pairs. arXiv preprint arXiv:2303.00915 (2023). version 3 updated Jan. 2025. 23. Boecking J, Chico L, Camacho J, Syeda-Mahmood T, Pfohl S, Saleh E, et al. Making the most of text semantics to improve biomedical Vision–language processing. Advances in Neural Information Processing Systems (NeurIPS) – Workshop on Medical Imaging Meets NeurIPS. arXiv:2204.09817. 2022. 24. Huemann Z, Tie X, Hu J, Bradshaw TJ. ConTEXTual Net: a multimodal Vision–language model for segmentation of pneumothorax. arXiv preprint arXiv:2303.01615. 2023. [ DOI ] [ PMC free article ] [ PubMed ] 25. Huang S, Liang H, Wang Q, Zhong C, Zhou Z, Shi M. SEG-SAM: semantic-guided SAM for unified medical image segmentation. arXiv preprint arXiv:2412.12660. 2024. 26. Clough JR, Byrne N, Oksuz I, Zimmer VA, Schnabel JA, King AP. A topological loss function for deep-learning based image segmentation using persistent homology. IEEE Trans Pattern Anal Mach Intel. 2022;44(12):8766–78. 10.1109/TPAMI.2020.3013679. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ] 27. Andlauer TK, Gaudeau-Bosma C, Yger P, Gori P, Colliot O. Topological data analysis for medical image processing: a literature review. medRxiv Prepr. 2025. 10.1101/2025.02.21.25322669. [ Google Scholar ] 28. Cicek O, Abdulkadir A, Lienkamp SS, Brox T, Ronneberger O. 3d unet: learning dense volumetric segmentation from sparse annotation. In: Medical image computing and computer-assisted intervention – MICCAI 2016. Springer; 2016. p. 424–32. [ Google Scholar ] 29. Myronenko A. 3d mri brain tumor segmentation using autoencoder regularization. In: Brainlesion: glioma, multiple sclerosis, stroke and traumatic brain injuries. Lecture notes in computer science. Vol. 11384. Springer; 2019. p. 311–20. 10.1007/978-3-030-11726-928. [ Google Scholar ] 30. Cardoso MJ, Li W, Brown R, et al. MONAI: an open-source framework for deep learning in healthcare. arXiv preprint arXiv:2211.02701. 2022. 10.48550/arXiv.2211.02701. 31. Hatamizadeh A, Tang Y, Nath V, Yang D, Myronenko A. UNETR: transformers for 3D medical image segmentation. Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV). 2022, pp. 574–84. 32. Wang W, Chen C, Ding M, Yu H, Zha S, Li J. TransBTS: multimodal brain tumor segmentation using Transformer. In: Medical image computing and computer assisted intervention – MICCAI 2021. Lecture notes in computer science. Vol. 12901. Springer; 2021. p. 109–19. 10.1007/978-3-030-87193-2_11. [ Google Scholar ] 33. Chen J, Lu Y, Yu Q, Luo X, Adeli E, Wang Y, et al. TransUNet: rethinking the U-Net architecture design for medical image segmentation through the lens of transformers. Med Image Anal. 2024;97:103280. 10.1016/j.media.2024.103280. [ DOI ] [ PubMed ] [ Google Scholar ] 34. Jiang Y, Zhang Y, Lin X, Dong J, Cheng T, Liang J. SwinBTS: a method for 3D multimodal brain tumor segmentation using Swin Transformer. Brain Sci. 2022;12(6):797. 10.3390/brainsci12060797. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ] 35. Lin J, Lin J, Lu C, Chen H, Lin H, Zhao B, et al. CKD-TransBTS: clinical knowledge-driven hybrid Transformer with modality-correlated cross-attention for brain tumor segmentation. IEEE Trans Med Imag. 2023;42(8):2451–61. 10.1109/TMI.2023.3250474. [ DOI ] [ PubMed ] [ Google Scholar ] 36. Chen J, Zhang J, Debattista K, Han J. Semi-supervised unpaired medical image segmentation through task-affinity consistency. IEEE Trans Med Imag. 2023;42:594–605. 10.1109/TMI.2022.3213372. [ DOI ] [ PubMed ] [ Google Scholar ] 37. Chen J, Chen C, Huang W, Zhang J, Debattista K, Han J. Dynamic contrastive learning guided by class confidence and confusion degree for medical image segmentation. Pattern Recognit. 2024;145:109881. 10.1016/j.patcog.2023.109881. [ Google Scholar ] 38. Chen J, Huang W, Zhang J, Debattista K, Han J. Addressing inconsistent labeling with cross image matching for scribble-based medical image segmentation. IEEE Trans Image Process. 2025;34:842–53. 10.1109/TIP.2025.3530787. [ DOI ] [ PubMed ] [ Google Scholar ] 39. Chen J, Li W, Li H, Zhang J. Deep class-specific affinity-guided convolutional network for multimodal unpaired image segmentation. In: Medical Image Computing and Computer Assisted Intervention – MICCAI 2020. Lecture Notes in Computer Science. Vol 12264. Springer; 2020:187-196. 10.1007/978-3-030-59719-1_19 40. Chen J, Duan H, Zhang X, Gao B, Grau V, Han J. From gaze to insight: bridging human visual attention and Vision-language model explanation for weakly-supervised medical image segmentation. arXiv preprint arXiv:2504.11368. 2025. https://arxiv.org/abs/2504.11368 . [ DOI ] [ PubMed ] 41. Radford A, Kim JW, Xu T, Hallacy C, Ramesh A, Goh G, et al. Learning transferable visual models from natural language supervision. Proceedings of the 38th International Conference on Machine Learning (ICML). Proceedings of Machine Learning Research, vol. 139. 2021. pp. 8748–63. 42. Zhang W, Yu PS. Graph neural networks in biomedical image analysis: a comprehensive survey. arXiv preprint arXiv:2301.04124. 2023. https://arxiv.org/abs/2301.04124 . 43. Cao X, Wang L, Chen Z, Shi F, Fishman EK, Yuille A, et al. Graph-CNN hybrid inference for multi-class brain tumor segmentation. In: Medical image computing and computer-assisted intervention – MICCAI 2024. Lecture notes in computer science. Vol. 14233. Springer; 2024. p. 42–52. [ Google Scholar ] 44. Veličković P, Cucurull G, Casanova A, Romero A, Liò P, Bengio Y. Graph attention networks. International Conference on Learning Representations (ICLR). 2018. 45. Chen H, Qi X, Heng PA, Dou Q. ASC-Net: adversarial-based selective cutting for unsupervised anomaly segmentation. In: Medical Image Computing and Computer Assisted Intervention – MICCAI 2021. Lecture Notes in Computer Science. Vol 12902. Springer; 2021:729-740. 10.1007/978-3-030-87234-2_69. 46. Behrendt F, Zimmerer D, Hering A, Wiesenfarth M, Kleesiek J. Leveraging the Mahalanobis distance to enhance unsupervised brain MRI anomaly detection. In: Medical Image Computing and Computer Assisted Intervention – MICCAI 2024. Lecture Notes in Computer Science. Vol 14349. Springer; 2024:377-387. 10.1007/978-3-031-43887-0_36. 47. Pinaya WHL, Graham MS, Gray R, et al. Fast unsupervised brain anomaly detection and segmentation with diffusion models. In: Medical Image Computing and Computer Assisted Intervention – MICCAI 2022. Lecture Notes in Computer Science. Vol 13438. Springer; 2022:705-714. 10.1007/978-3-031-16452-1_67. 48. Zhou H-Y, et al. Relative mamc: multi-attention multi-contrast mri tumor segmentation. arXiv:2104.03309. 2021. 49. Xing Z, Yu L, Wan L, Han T, Zhu L. NestedFormer: nested modality-aware transformer for brain tumor segmentation. In: Medical Image Computing and Computer Assisted Intervention – MICCAI 2022. Lecture Notes in Computer Science. Vol 13435. Springer; 2022:140-150. 10.1007/978-3-031-16443-9_14. 50. Sun K, Ding J, Li Q, Chen W, Zhang H, Sun J, Jiao Z, Ni X. CMAF-Net: a cross-modal attention fusion-based deep neural network for incomplete multi-modal brain tumor segmentation. Quant Imaging Med Surg. 2024;14(7):4579-4604. 10.21037/qims-24-9. [ DOI ] [ PMC free article ] [ PubMed ] 51. Lee J, Zhu H. Learning from heterogeneous data with stick-breaking variational autoencoder. Presented at: The 7th International Conference on Econometrics and Statistics (EcoSta2024); 2024; Tokyo, Japan. Abstract A0594. https://www.cmstatistics.org/EcoSta2024 . 52. Rolls ET, Huang C, Lin C, Feng J. Automated anatomical labeling atlas 3. Hum Brain Mapp. 2020;41(8):2145–57. 10.1002/hbm.24915. [ Google Scholar ] 53. Milletari F, Navab N, Ahmadi S-A V-net: fully convolutional neural networks for volumetric medical image segmentation. Proceedings of the 4th International Conference on 3D Vision (3DV). 2016, pp. 565–71. 10.1109/3DV.2016.79. 54. Loshchilov I, Hutter F. Decoupled weight decay regularization. International Conference on Learning Representations (ICLR). 2019. 55. Oktay O, Schlemper J, Le Folgoc L, Lee M, Heinrich M, Misawa K, et al. Attention u-net: learning where to look for the pancreas. Medical Imaging with Deep Learning (MIDL) 2018. 2018. https://arxiv.org/abs/1804.03999 . 56. Feng Y, Cao Y, An D, Liu P, Liao X, Yu B. Daunet: a u-shaped network combining deep supervision and attention for brain tumor segmentation. Knowl-Based Syst. 2024;285:111348. 10.1016/j.knosys.2023.111348. [ Google Scholar ] 57. Havaei M, Guizard N, Chapados N, Bengio Y. HeMIS: hetero-modal image segmentation. In: Medical Image Computing and Computer Assisted Intervention – MICCAI 2016. Lecture Notes in Computer Science. Vol 9901. Springer; 2016:469-477. 10.1007/978-3-319-46723-8_54. 58. Zhou Y, Feng Z, Lv X, Zhang Z, Jia X, Wu G. A literature survey of MR-based brain tumor segmentation with missing modalities. Computerized Med Imag Graphics. 2023;104:102055. 10.1016/j.compmedimag.2022.102055. [ DOI ] [ PubMed ] [ Google Scholar ] 59. Sheller MJ, Edwards B, Reina GA, Martin J, Pati S, Kotrotsou A, et al. Federated learning in medicine: facilitating multi-institutional collaborations without sharing patient data. Sci Rep. 2020;10(1):12598. 10.1038/s41598-020-69250-1. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ] 60. Pati S, Baid U, Edwards B, Sheller MJ, Bakas S, et al. Towards fair decentralized benchmarking of healthcare AI algorithms with the federated tumor segmentation (FeTS) challenge. Nat Commun. 2024;15(1):4775. 10.1038/s41467-024-47437-5. Large multi-institutional benchmark for federated tumour segmentation. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ] 61. Ahamed MF, Hossain MM, Nahiduzzaman M, Islam MR, Noman MAA, Rahman MA, et al. A review on brain tumor segmentation based on deep learning methods with federated learning techniques. Computerized Med Imag Graphics. 2024;109:102313. 10.1016/j.compmedimag.2023.102313. [ DOI ] [ PubMed ] [ Google Scholar ] Associated Data This section collects any data citations, data availability statements, or supplementary materials included in this article. Supplementary Materials Supplementary Material 1 (175.6KB, pdf) Data Availability Statement The Brain Tumor Segmentation (BraTS) 2020 dataset used in this study is publicly available through registration at the CBICA Image Processing Portal ( https://ipp.cbica.upenn.edu ). The dataset can be accessed following the data request procedure described at https://www.med.upenn.edu/cbica/brats2020/registration.html . Articles from BMC Medical Imaging are provided here courtesy of BMC ACTIONS View on publisher site PDF (4.8 MB) Cite Collections Permalink RESOURCES Similar articles Cited by other articles Links to NCBI Databases Follow NCBI NCBI on X (formerly known as Twitter) NCBI on Facebook NCBI on LinkedIn NCBI on GitHub NCBI RSS feed Connect with NLM NLM on X (formerly known as Twitter) NLM on Facebook NLM on YouTube National Library of Medicine 8600 Rockville Pike Bethesda, MD 20894 Web Policies FOIA HHS Vulnerability Disclosure Help Accessibility Careers NLM NIH HHS USA.gov Back to Top

Related documents

• UTriGate-Net : Uncertainty-aware brain tumor segmentation via triaxial context encoding and gated modality fusion#7860
• Pooling multimodal cancer data across unaligned embedding spaces maintains tumor of origin signal.#418381
• FRMF-Net: Feature rectification and adaptive modality fusion guided multi-modal brain tumor segmentation network#211126
• Semi-supervised Medical Image Segmentation via Perturbation-Aware Mutual Learning and Edge-Aware Uncertainty Loss for Accurate Anatomical Delineation#445028
• Localization-Infused Vision-Language Semantic Fusion for Text-Guided Medical Image Segmentation#665858