SPK: Eliciting Structured Prior Knowledge for Interpretable Out-of-Distribution Detection in Real-Time Object Detection

SPK: Eliciting Structured Prior Knowledge for Interpretable Out-of-Distribution Detection in Real-Time Object Detection Changshun Wu1,2∗ , Weicheng He2 , Xiaowei Huang1 , Saddek Bensalem3 1 University of Liverpool, UK Université Grenoble Aples, France 3 CSX-AI, France Changshun.Wu@liverpool.ac.uk, Weicheng.He@univ-grenoble-alpes.fr, Xiaowei.Huang@liverpool.ac.uk, Saddek.Bensalem@csx-ai.com

arXiv:2608.19080v1 [cs.CV] 19 Aug 2026

2

Abstract Object detectors often produce over-confident predictions for objects outside their training categories, leading to so-called out-of-distribution (OoD) hallucinations. Existing approaches for detecting or mitigating such hallucinations typically either construct scoring functions directly over learned object detector representations or modify the object detector itself to suppress hallucination emergence. However, the latent priors implicitly encoded in these representations remain largely unexplored and have not been explicitly decoded for OoD detection. To uncover and exploit these latent priors, we propose Structured Prior Knowledge (SPK), a hallucination-oriented framework that explicitly elicits OoD-relevant priors from pretrained object detectors. Specifically, SPK leverages indistribution data and hallucination-inducing samples as diagnostic supervision to elicit part-level semantic concepts underlying object detector decision-making, rather than using them merely for rejection or object detector adaptation. The elicited semantic priors are further integrated with geometric and contextual priors to form a compact five-dimensional SPK representation for OoD detection. Extensive experiments across diverse object detector architectures and multiple OoD benchmarks demonstrate that SPK achieves state-of-the-art OoD detection. Our findings reveal that pretrained object detectors already encode substantially richer latent knowledge than is typically exploited for OoD detection. More importantly, this knowledge can be explicitly elicited and organized into a compact, structured, and interpretable knowledge space for prediction reliability analysis. This suggests a promising proactive route for improving object detector reliability by explicitly uncovering and leveraging latent priors. Code and data are available at: https://gricad-gitlab.univ-grenoblealpes.fr/dnn-safety/spk.

Introduction Object detection has become a fundamental capability of modern vision systems, enabling numerous real-world applications such as autonomous driving, robotics, and intelligent perception. Despite remarkable progress, modern object detectors still suffer from a critical reliability issue: they often produce confident predictions for objects belonging to categories outside their predefined training categories. These erroneous predictions, referred to as out-of-distribution (OoD) hallucinations (He et al. 2025; Wu et al. 2026), arise when the closed-world assumption underlying detector training is violated in open-world environments. Such hallucinations ∗

Corresponding author.

may severely compromise the reliability and safety of downstream vision systems, making effective OoD hallucination mitigation an increasingly important problem. Existing approaches mitigate OoD-induced hallucinations through two complementary routes. Reactive approaches detect hallucinations after predictions are generated by introducing external OoD detectors based on confidence estimation (Peng, Dong, and Ong 2026; Li et al. 2026), uncertainty modeling (Dai, Chen, and Yu 2026), or intermediate feature representations (Du et al. 2022a; Wilson et al. 2023; Wu, Falcone, and Bensalem 2023; Wu et al. 2024; He, Wu, and Bensalem 2024). Proactive approaches, in contrast, seek to reduce hallucination occurrence by adapting the object detector itself, for example through lightweight fine-tuning that exploits detector-intrinsic properties (Wu et al. 2026). Although these two routes differ in their mitigation strategies, they ultimately operate on the same learned high-dimensional representation space of object detectors: existing methods either design increasingly advanced OoD scoring functions over these representations or modify the object detector to improve their discriminability. This naturally raises a fundamental question: What latent priors have object detectors already learned within these representations that determine whether a prediction becomes a valid detection or an OoD hallucination? We argue that answering this question provides a fundamentally different proactive perspective for OoD hallucination mitigation. Rather than designing increasingly sophisticated algorithms over opaque high-dimensional representations or modifying the object detector itself, we seek to explicitly elicit the latent priors already encoded within pretrained object detectors and organize them into a compact and interpretable representation space for OoD detection. To answer this question, we propose Structured Prior Knowledge (SPK), a hallucination-oriented framework that explicitly elicits hallucination-relevant priors from pretrained object detectors. Our key observation is that OoD hallucinations are not arbitrary prediction failures, but are consistently associated with two representative sources. The first consists of proximal OoD objects, whose visual appearances resemble known categories and therefore induce semantic confusion. The second consists of background-only samples, where nonobject regions unexpectedly activate detector objectness and produce false detections. Rather than using these samples for

Figure 1: The proposed SPK framework, a proactive OoD hallucination mitigation framework, further reduces OoD-induced hallucinations beyond previous state-of-the-art methods (Wu et al. 2026), achieving additional improvements in challenging high-performance regimes. outlier exposure or detector adaptation, SPK treats them as diagnostic supervision for revealing the part-level semantic concepts underlying object detector decision-making. Building upon this observation, we develop an automated data construction pipeline that discovers informative proximal OoD and background-only samples while generating high-quality part-level semantic supervision. The elicited semantic priors are further integrated with geometric and contextual priors to form a compact five-dimensional SPK representation. The resulting representation consists of three semantic priors capturing part-level object concepts, a geometric prior measuring the relative spatial extent of predictions, and a contextual prior reflecting image-level contextual evidence. Importantly, this structured representation enables stronger OoD detection performance despite its substantially lower dimensionality. As shown in Table 1, when applying the same downstream OoD detection algorithms, SPK consistently outperforms the original object detector representations. More importantly, as shown in Fig. 1, a lightweight Isolation Forest (iForest) anomaly detector operating on the SPK representation surpasses the strongest existing mitigation strategy (Wu et al. 2026), referred to as Proximal-OoD, which combines object detector fine-tuning for suppressing hallucination emergence with KNN-based OoD detection. This advantage is consistently observed across diverse detector architectures, with clear performance margins over the existing mitigation strategy. These results highlight a new perspective for improving object detector reliability by uncovering and leveraging structured knowledge embedded within pretrained detectors. Our contributions are summarized as follows:

tectures and multiple OoD benchmarks, achieving stateof-the-art (SoTA) performance in OoD detection while enabling lightweight and interpretable deployment. • Our findings reveal that pretrained object detectors encode substantially rich latent knowledge, which can be explicitly organized into a compact, structured, and interpretable knowledge space, opening a promising new direction for improving object detector reliability.

Related Work OoD Hallucination Detection and Understanding in Object Detectors Object detectors are typically trained under a closed-world assumption and may produce over-confident predictions for objects outside their training categories, resulting in the so-called OoD hallucinations. Existing approaches can be broadly categorized into reactive and proactive paradigms. Reactive approaches formulate OoD hallucination as a post-hoc detection task (OoD detection). They detect hallucinated predictions after object detector inference by confidence estimation (Peng, Dong, and Ong 2026; Li et al. 2026), uncertainty modeling (Dai, Chen, and Yu 2026), or intermediate feature representations (Du et al. 2022a; Wilson et al. 2023; Wu et al. 2024). Although effective in improving the separation between ID and OoD predictions, these methods largely treat hallucinations as prediction-level anomalies and provide limited insight into why the detector makes such decisions. Proactive approaches instead seek to understand the underlying causes of hallucination by actively diagnosing detector behavior. Recent proactive intervention methods (He et al. 2025; Wu et al. 2026) reveal that conventional objectness estimation may respond to generic objectlike patterns rather than ID objects, and leverage proximal OoD data to calibrate objectness toward distribution-aware prediction. In contrast, our work follows a proactive elicitation perspective. Rather than intervening in the detector’s decision mechanism, we aim to uncover the latent seman-

• We propose a proactive framework for eliciting and organizing latent priors from pretrained object detectors into a structured knowledge space for OoD detection. • We demonstrate the effectiveness of SPK through extensive experiments across diverse object detector archi2

Structured Prior Knowledge Framework

tic knowledge encoded in pretrained detectors by eliciting class-wise semantic prototypes responsible for their decisions. Through the SPK space, our approach provides an interpretable view of what evidence the detector observes when generating predictions and enables OoD hallucination detection based on these elicited representations.

To answer the question of what knowledge object detectors have already acquired, we propose SPK, a proactive framework that explicitly elicits detector priors for OoD hallucination detection. Unlike existing approaches that directly operate on object detector outputs or high-dimensional feature representations, SPK aims to uncover the underlying factors that contribute to detector decisions. As illustrated in Fig. 2, SPK elicits three complementary types of detector priors, namely semantic, geometric, and contextual priors, through hallucination-oriented supervision. These priors provide an explicit characterization of detector behavior and are organized into a compact five-dimensional SPK representation. Finally, the SPK representation is used by a lightweight OoD detector to determine whether each prediction should be retained or rejected. The following subsections describe each component of the proposed framework.

Concept-based Knowledge Elicitation and Structured Priors Understanding and extracting interpretable knowledge from deep networks has been widely studied through feature decoding and concept-based representation learning. Linear probing (Alain and Bengio 2017), Network Dissection (Bau et al. 2017), and Net2Vec (Fong and Vedaldi 2018) demonstrate that intermediate representations contain human-interpretable semantic information. Concept bottleneck models further introduce explicit concept variables for improving model transparency, with recent works exploring label-free and spatial concept representations (Oikarinen et al. 2023; Benou and Riklin Raviv 2025; Schrodi et al. 2025). Beyond interpretability, semantic concepts and structured priors have also been investigated for improving model robustness. Part-level supervision has been shown to encourage more robust object representations beyond brittle appearance cues (Sitawarin et al. 2023; Li et al. 2024), while recent detector-based concept decomposition methods explore interpretable semantic evidence for unknown-object discovery (Lv et al. 2026). However, existing concept-based methods are rarely designed for analyzing OoD hallucinations in object detectors. In contrast, our work leverages part-level concepts and object detector behaviors to elicit hallucination-relevant priors, integrating semantic, geometric, and contextual knowledge into a compact SPK representation for interpretable OoD detection.

Hallucination-oriented Data Construction In this section, we first detail the construction of the two types of OoD data yielding hallucinations, used along with ID data to elicit the semantic priors within our SPK framework. Then, we present the tool we built for annotating these three types data at the part-level. Hallucination-Induced Data Preparation We construct hallucination-inducing training data from two complementary sources: proximal OoD objects and background-only images. For each ID category, we follow the prompt design of (Wu et al. 2026) and query GPT-5 to identify semantically or visually similar non-ID categories, from which annotated images are retrieved from Objects365 (Shao et al. 2019). We additionally collect background-only samples from the DTD dataset (Cimpoi et al. 2014) and filter out images containing ID objects using a pretrained object detector. We then run the target object detector on both sources and retain only the samples that induce hallucinated ID predictions. These hallucination-induced samples are used to train the semantic elicitation heads. More details on data construction and filtering are provided in Appendix A.1.

Problem Formulation We follow the standard formulation of OoD detection in object detection (Du et al. 2022b; Wu et al. 2026). Let DID denote the ID dataset used to train an object detector, where objects belong to a predefined category set CID . Given an input image I, an object detector f trained on DID produces a set of predictions P = f (I) = {pi }N i=1 , where each prediction is represented as pi = (bi , ŷi ). Here, bi denotes the predicted bounding box and ŷi ∈ CID denotes the predicted category. For each prediction p, we denote its corresponding contextual image by Ictx (p), which represents the image from which the prediction is generated. Given an OoD dataset DOoD whose object categories are disjoint from CID , OoD detection aims to identify predictions generated on OoD inputs that should be rejected. Formally, an OoD detector g is a binary decision function: g : P → {0, 1}, where an output of 1 denotes that pi is an OoD-induced hallucination and should be rejected, while an output of 0 denotes that pi is retained. Such OoD-induced hallucinations may originate from different sources, including objects from categories outside CID and background regions that spuriously activate detector responses.

Part-Level Automated Annotation Training the semantic elicitation module requires spatially grounded part-level semantic supervision that captures the visual evidence associated with object detector decisions. To eliminate part-level annotation as a practical bottleneck, we develop an automated annotation tool that generates localizable part concepts, grounds them within object regions, and produces the corresponding concept masks for training the semantic elicitation heads. The resulting annotations require only lightweight manual verification. Further implementation details are provided in Appendix A.1.

Prior Elicitation Rather than directly constructing OoD decision functions on high-dimensional object detector representations, SPK investigates whether object detector predictions can be characterized through a small set of explicit latent priors. To this end, 3

SPK Framework for OoD detection Prior Elicitation Module ROI Features & label:

Data Preparation

ID Data

Target

Proximal OoD Data

Object

Background only Data

Detector

Bounding Box: bi

Semantic Prior sid, sprox, sbg

Geometric Prior rgeo

SPK Space zSPK := [s

id, sprox, sbg, rgeo, dctx]

🔥 Trainable Part-level Semantic

Image-level encoding

Contextual Prior dctx

Elicitation Head

OoD Detection Function

(Class-wise)

ID Channels

max

sid

Proximal Channels

max

sprox

Background Channel

sbg

Part-level Semantic Aactivation Maps: Three Types

Figure 2: Overview of the proposed SPK framework. SPK elicits semantic, geometric, and contextual priors from a pretrained object detector and organizes them into a compact five-dimensional representation for OoD hallucination detection. we elicit three complementary types of object detector priors, namely semantic, geometric, and contextual priors, each capturing a different aspect of detector behavior. Specifically, semantic priors characterize the semantic evidence associated with a prediction, geometric priors describe object-level geometric regularities, and contextual priors capture the similarity between the associated image Ictx (p) of a prediction and ID samples in the image-level representation space. The following subsections describe each prior elicitation module.

layout within the RoI. Following part-level concept annotations, each RoI is divided into a fixed spatial grid, and the semantic head is trained to recover the corresponding celllevel concept masks. This formulation enables the detector representation to be decoded into interpretable part-level semantic responses without requiring pixel-level segmentation. To optimize the semantic elicitation head, we employ three complementary objectives: LSPK = Lconcept + λg Lsuppress + λs Lgroup .

Semantic Prior Elicitation Semantic information provides an important cue for determining whether a detector prediction corresponds to a valid object or an OoD hallucination. Inspired by recent advances in concept-based representation learning (Sun et al. 2023), we hypothesize that object detectors also implicitly encode category-specific semantic evidence within their intermediate representations. Rather than learning new semantic knowledge, our goal is to elicit and decode this latent knowledge through part-level semantic concepts. We therefore introduce a semantic prior elicitation module that translates detector representations into structured part-level semantic responses. Given a prediction pi = (bi , yi ), we first extract the Region-of-Interest (RoI) feature Fi corresponding to its predicted bounding box bi . Instead of directly using Fi as an OoD representation, we learn a category-specific semantic elicitation head Hyi to decode the latent semantic evidence encoded in the detector representation. Specifically, each semantic head predicts the activation of a set of part-level concepts associated with the prediction category. Let Cyi = {1, . . . , Nyi } denote the concept vocabulary associated with prediction category yi . To construct semantic supervision, we consider three complementary concept groups associated with category yi : (1) part-level concepts from the ID category yi , (2) concepts from proximal OoD categories related to yi , and (3) background concepts obtained from background-only data. These concept groups are constructed from ID, proximal OoD, and background-only data, respectively, and represent different semantic sources that may contribute to detector predictions. For each prediction, the semantic head outputs a concept activation tensor:

The first objective encourages accurate reconstruction of part-level semantic concepts. Specifically, we minimize a Dice loss (Sudre et al. 2017) between the predicted concept activation tensor Ai and the corresponding cell-level concept annotation Mi : Lconcept = Epi [DiceLoss(Ai , Mi )]. The Dice objective encourages each concept channel to recover the spatial distribution of its corresponding part-level concept. The second objective prevents the semantic elicitation head from activating concepts that are not supported by the observed RoI. Given the annotated concept set of a prediction, we introduce a spurious concept suppression loss that penalizes responses from absent concepts. This objective improves the specificity of elicited semantic responses and reduces spurious concept activations. For prediction pi , we define its absent concept set as Ci− . For example, an RoI for a bird typically includes part-level concepts such as eyes, beak, wings, torso, and feet; however, if the feet are occluded and not visible, the ground-truth annotation for the feet concept becomes empty, and the corresponding concept channel belongs to Ci− . Let Ai,c ∈ [0, 1]H×W denote the activation map corresponding to concept channel c, and Ui ∈ {0, 1}H×W the spatial union mask. We formulate the spatial constraint as   X 1 ⟨Ui , − log (1 − Ai,c )⟩  Lsuppress = Epi  − (1) ∥Ui ∥1 Ci − c∈Ci

Nyi ×H×W

Ai = Hyi (Fi ) ∈ [0, 1] , where each channel corresponds to a specific part-level concept in Cyi , and the spatial dimensions represent the cell-level

where ⟨·, ·⟩ and ∥Ui ∥1 denote the Frobenius inner product and the number of annotated-positive cells, respectively. The logarithm is applied element-wise. 4

Structured Prior Knowledge

Finally, we introduce a group-level semantic objective to ensure that the dominant semantic response corresponds to the actual source of the underlying visual evidence. We define the set of semantic groups as K = {id, prox, bg}, where each concept channel is assigned to one group through the mapping πyi : Cyi → K. Given the channel activation aci , the response of each semantic group is obtained by: ski =

max c∈Cyi :πyi (c)=k

aci ,

k∈K

The elicited semantic, geometric, and contextual priors provide complementary views of the latent knowledge encoded by object detectors. We organize these priors into a compact SPK representation for each prediction. Specifically, the semantic module produces three group-level responses prox bg si = [sid , si ], corresponding to the ID, proximal i , si OoD, and background concept groups. Together with the geometric prior rigeo and contextual prior dctx i , the final SPK representation is defined as

(2)

The resulting group response is optimized with a crossentropy loss: Lgroup = −Epi

X k∈K

prox bg geo ctx , si , ri , di ] ∈ R5 . zSPK = [sid i i , si Each dimension of the SPK vector corresponds to an interpretable aspect of object detector knowledge, enabling OoD detection based on structured priors rather than raw highdimensional detector features.

exp(ski ) k′ k′ ∈K exp(si )

qik log P

where qi denotes the one-hot semantic group label of prediction pi .

SPK-based OoD Detection , we learn an Given the structured prior representation zSPK i OoD detector gSPK : zSPK → {0, 1} to determine whether i a prediction should be retained or rejected. Specifically, gSPK (zSPK ) = 1 indicates that prediction pi is rejected i as an OoD-induced hallucination, while gSPK (zSPK ) = 0 i indicates that the prediction is retained as an ID prediction. In practice, gSPK can be applied with different OoD detection methods, such as KNN (Sun et al. 2022) and Isolation Forest (iForest) (Liu, Ting, and Zhou 2008). By decoupling OoD detection from raw detector features, SPK enables the final decision to be made based on explicitly elicited and structured detector priors.

Geometric Prior Elicitation Object detectors inherently rely on geometric regularities when generating bounding box predictions. For example, common object detector designs exploit object scale and spatial distributions through anchor design or bounding box regression priors. Motivated by this observation, we extract geometric priors from the predicted bounding boxes to characterize whether a prediction follows the geometric patterns of known objects. Given a prediction pi = (bi , ŷi ), we compute its relative area as the geometric descriptor: Area(bi ) ri = , Area(Ictx (pi )) where Ictx (pi ) denotes the image associated with prediction pi . The relative area provides a scale-normalized measurement of the predicted object size and is invariant to image resolution. This simple geometric statistic captures the object-scale prior implicitly used by object detectors and serves as the geometric component of the proposed SPK representation.

Experiments In this section, we comprehensively evaluate SPK from two perspectives: OoD detection with the underlying object detector unchanged, and OoD hallucination mitigation against approaches that modify object detector parameters. This evaluation is conducted across different object detection architectures and benchmark settings.

Contextual Prior Elicitation Besides object-level properties, detector predictions are also influenced by the visual context of the image in which they appear. We therefore introduce a contextual prior that measures the similarity between the associated image of a prediction and the ID training images. Given a prediction pi , we first extract its associated image representation: vi = ϕdet (Ictx (pi )) where ϕdet (·) denotes the image-level representation extracted from the detector. To focus on the priors intrinsically encoded by the detector, we directly utilize the detector representation without introducing additional external models. We construct an ID train image representation bank: VID = {ϕdet (Ij ) | Ij ∈ DID }. The contextual prior is then computed by measuring the similarity between vi and its K-nearest neighbors (KNN) in VID : dctx = KNN(vi , VID ), i

Experimental Setup OoD Detection We follow the recently introduced calibrated evaluation protocol (Wu et al. 2026), which removes potential test contamination from OoD evaluation. Moreover, this benchmark considers three representative object detection architectures, including YOLO (Wang et al. 2025a), Faster R-CNN (Ren et al. 2015), and RT-DETR (Zhao et al. 2024), covering one-stage, two-stage, and transformer-based detectors. We evaluate on two ID detection tasks, PASCALVOC (Everingham et al. 2010) and BDD-100K (Yu et al. 2020), with two categories of OoD test samples: Near-OoD and Far-OoD. Near-OoD samples share visual similarities with ID categories, whereas Far-OoD samples exhibit larger semantic and visual discrepancies. The baseline methods include MSP (Hendrycks and Gimpel 2017), EBO (Liu et al. 2020), MLS (Hendrycks et al. 2022), MDS (Lee et al. 2018b), BAM (Wu et al. 2024), SCALE (Xu et al. 2024), KNN (Sun et al. 2022), iForest (Liu, Ting, and Zhou 2008), and Proximal-OoD (Wu et al. 2026).

where dctx denotes the resulting image-level contextual i score. A lower value indicates that the associated image of the prediction is more consistent with the visual characteristics observed in the ID training data. 5

methods. We consider four representative approaches, MDS, BAM, KNN, and iForest, which can be directly applied to raw high-dimensional detector features without modifying the underlying detector. Replacing the raw detector features with the proposed SPK representation consistently yields substantial performance improvements across all detector architectures and benchmarks. Moreover, the resulting SPKbased methods outperform existing SoTA OoD detection approaches, even though the OoD detection algorithms themselves remain unchanged. These results indicate that the performance gain primarily stems from the quality of the learned representation rather than the choice of the OoD detector. This observation suggests that constructing a better representation space may be more important than designing increasingly sophisticated OoD detection algorithms. The corresponding AUROC results are provided in Appendix A.4, leading to the same conclusions. Beyond standard OoD detection metrics, we further evaluate the practical effectiveness of SPK by measuring the reduction in the number of OoD-induced hallucinations. As shown in Table 2, SPK consistently removes substantially more hallucinations than the recent ProximalOoD-based fine-tuning approach (Wu et al. 2026) across three representative detector architectures. Notably, SPK achieves these improvements without modifying or fine-tuning the underlying object detector, demonstrating that explicitly leveraging the object detector’s latent prior knowledge alone is sufficient to achieve highly effective OoD mitigation.

We adopt two complementary evaluation metrics. First, we report standard OoD detection metrics, including AUROC and FPR95, where FPR95 denotes the false positive rate at 95% true positive rate of ID samples. Second, we report the reduction in the number of OoD-induced hallucinations, which enables comparison with strong OoD mitigation approaches that modify the underlying object detector. In particular, we compare with (Wu et al. 2026), which substantially improves OoD hallucination mitigation through object detector fine-tuning and represents a strong benchmark for this evaluation setting. For comparison with OoD detection methods whose applicability to the calibrated benchmark is unclear due to architecture-specific designs or unavailable implementations, we additionally evaluate SPK under the uncalibrated benchmark adopted by these methods, despite its known annotation issues in (Wu et al. 2026). We follow the configurations in (Peng, Dong, and Ong 2026), which considers Deformable-DETR (Zhu et al. 2021) on PASCAL-VOC and BDD-100K and Faster R-CNN on PASCAL-VOC, with MSCOCO (Lin et al. 2014) and OpenImages (Kuznetsova et al. 2020) as OoD test sets. SPK achieves competitive performance by exploiting detector-intrinsic priors. The DINObased variant further improves results by replacing the contextual prior with a stronger external visual representation. The complete comparison results are provided in Appendix A.5. Prior Elicitation For semantic prior elicitation, we train a lightweight four-layer residual convolutional head that decodes each detector RoI feature to part-level semantic responses in the form of spatial concept-logit maps, one for each concept in the corresponding part-level vocabulary. Since relevant concepts differ across object categories, we train one semantic elicitation head for each ID class. Each head is trained for up to 80 epochs with early stopping. For instance, training all heads on the 10-class BDD dataset takes about one hour on a single NVIDIA A100 GPU 40GB. Additional training details and qualitative visualizations are provided in Appendix A.2. To elicit contextual priors, we treat the entire image as an RoI and aggregate each multi-scale neck feature using its channel-wise spatial mean and standard deviation to form an image-level contextual embedding. We organize these embeddings into class-specific reference banks according to the object categories present in each image. More details are provided in Appendix A.2.

Ablation on Semantic Prior Elicitation The semantic prior is the core component of the proposed SPK framework. To investigate the contribution of each learning objective, we conduct an ablation study by progressively removing individual loss terms while keeping all other components unchanged. The results are summarized in Table 3. Removing any of the three losses consistently degrades OoD detection performance, demonstrating that they play complementary roles during semantic prior elicitation. Specifically, the Dice loss provides accurate supervision for part-level concept activation, the group discrimination loss encourages concept responses to concentrate on the correct semantic group, and the spatial consistency loss further regularizes the spatial activation patterns within each concept channel. The best performance is achieved only when all three objectives are jointly optimized, confirming that accurate semantic prior elicitation requires both semantic discrimination and spatially consistent concept representations. Additional quantitative results on Faster R-CNN and RTDETR are provided in the Appendix A.4, showing consistent trends across different detector architectures.

Effectiveness of SKP Rather than introducing a new OoD detector, the objective of SPK is to construct a structured prior representation on which existing OoD detection techniques can operate more effectively. We therefore evaluate SPK from two complementary perspectives: (1) whether the learned SPK representation improves the discriminability of existing OoD detection methods, and (2) whether it translates into SoTA OoD hallucination mitigation performance.

Ablation on Prior Components We next investigate the contribution of each prior component in the proposed SPK representation. Starting from semantic priors alone, we progressively incorporate geometric and contextual priors while keeping the OoD detector unchanged. The results are reported in Table 4. Semantic priors alone already provide strong OoD discrimination, confirming that part-level concepts capture rich latent knowledge

A Better Representation Space for OoD Detection. Table 1 evaluates whether the proposed SPK representation provides a better feature space for existing OoD detection 6

Table 1: Comparison of OoD detection performance using FPR95 across different detector architectures trained on PASCALVOC and BDD-100K. Lower is better. YOLO Method

PASCAL-VOC

Faster R-CNN BDD-100K

PASCAL-VOC

RT-DETR

BDD-100K

PASCAL-VOC

BDD-100K

Near-OoD Far-OoD Near-OoD Far-OoD Near-OoD Far-OoD Near-OoD Far-OoD Near-OoD Far-OoD Near-OoD Far-OoD MSP EBO MLS SCALE MDS BAM KNN iForest

67.48 90.49 89.88 80.67 57.67 45.36 48.20 70.27

67.18 90.84 90.08 80.92 69.47 43.72 39.50 67.82

72.93 87.22 86.47 77.44 68.42 49.63 41.95 60.42

74.12 87.06 87.06 70.59 82.35 52.18 45.24 65.23

68.36 60.62 59.62 92.34 49.96 65.44 61.95 75.43

78.69 56.21 57.89 80.70 56.38 42.16 37.53 52.38

77.46 94.83 92.55 86.35 78.90 65.73 50.54 63.25

74.41 94.37 91.08 84.98 79.34 61.34 49.76 62.78

70.44 98.03 92.84 81.12 48.65 75.61 77.10 79.52

67.81 96.69 89.75 76.92 52.90 65.27 63.00 65.92

77.46 94.83 92.55 86.35 78.90 75.48 62.10 68.23

74.41 94.37 91.08 84.98 79.34 68.44 58.00 62.30

SPK-MDS SPK-BAM SPK-KNN SPK-iForest

14.99 21.96 19.64 14.25

17.28 21.73 17.99 11.84

23.35 16.75 13.47 9.86

6.46 3.07 1.17 0.70

21.03 18.98 15.50 13.92

23.50 18.17 13.69 10.52

42.42 9.07 4.70 2.31

42.73 6.18 3.09 1.52

18.43 23.12 18.26 15.48

17.83 25.19 20.43 17.32

41.77 18.76 16.97 11.42

16.48 16.84 13.74 9.25

Table 2: OoD detection counts (Near-OoD/Far-OoD) across different detector architectures. Lower is better. Model

Method

VOC (N/F)

BDD (N/F)

YOLO

Original Proximal-OoD SPK

946 / 440 134 / 60 135 / 52

701 / 666 80 / 47 69 / 5

Faster R-CNN

Original Proximal-OoD SPK

2150 / 1335 710 / 253 299 / 140

2576 / 1634 207 / 167 60 / 25

RT-DETR

Original Proximal-OoD SPK

2311 / 1589 386 / 470 358 / 275

3145 / 1220 525 / 240 359 / 113

sults demonstrate that semantic, geometric, and contextual priors capture complementary aspects of detector knowledge, and their combination provides a more comprehensive representation of the latent priors encoded by object detectors. Additional quantitative results on Faster R-CNN and RT-DETR are provided in the Appendix A.4, showing consistent trends across different detector architectures. Table 4: Ablation study of different prior components on YOLO trained on PASCAL-VOC and BDD-100K. Each dataset column reports Near-OoD / Far-OoD FPR95. The average is computed over all four results. Lower is better. Prior components

Table 3: Ablation study of the SPK loss components on YOLO trained on PASCAL-VOC and BDD-100K. Results are reported as Near-OoD / Far-OoD FPR95. The average is computed over all four results. Lower is better. Ldice Lsuppress Lgroup ✗ ✓ ✓ ✓

✓ ✓ ✗ ✓

✓ ✗ ✓ ✓

VOC

BDD

Average

Near / Far

Near / Far

FPR95 ↓

25.80 / 21.30 17.85 / 18.26 22.10 / 16.40 15.29 / 15.97 20.50 / 15.80 14.18 / 12.93 14.25 / 11.84 9.86 / 0.70

Semantic Semantic + Geometric Semantic + Geometric + Contextual

VOC

BDD

Average

Near / Far

Near / Far

FPR95 ↓

15.43 / 13.28 30.37 / 25.58 13.23 / 11.50 20.88 / 3.84 14.25 / 11.84 9.86 / 0.70

21.17 12.36 9.16

Conclusion

20.80 17.44 15.85 9.16

In this work, we propose Structured Prior Knowledge (SPK), a proactive framework that explicitly elicits latent priors from pretrained object detectors for OoD detection. By organizing these priors into a compact and interpretable representation, SPK enables effective OoD detection and hallucination mitigation without modifying the underlying detector. Extensive experiments across diverse object detector architectures and benchmark settings demonstrate the effectiveness and generality of the proposed framework. Beyond the framework itself, our results reveal that pretrained object detectors encode substantially richer knowledge than is typically exploited for OoD detection. Rather than learning new representations from raw high-dimensional detector features, SPK shows that these latent priors can be explicitly elicited into an interpretable knowledge space, where each dimension corresponds to a semantically meaningful quantity. We believe this study establishes prior elicitation as a promising proactive di-

encoded by object detectors. Incorporating geometric priors further improves the performance, suggesting that objectscale information provides complementary structural cues beyond semantic concepts. Adding contextual priors brings further improvements, particularly on the more complex BDD-100K benchmark, where image-level similarity provides additional cues for distinguishing OoD-induced hallucinations from valid ID predictions. In contrast, the improvement on PASCAL-VOC is relatively limited, potentially because its test images share similar visual characteristics with the ID distribution, reducing the additional discriminative benefit provided by image-level context. Overall, these re7

Everingham, M.; Van Gool, L.; Williams, C. K.; Winn, J.; and Zisserman, A. 2010. The pascal visual object classes (voc) challenge. International Journal of Computer Vision (IJCV), 88: 303–338. Fong, R.; and Vedaldi, A. 2018. Net2Vec: Quantifying and Explaining How Concepts Are Encoded by Filters in Deep Neural Networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 8730– 8738. Gupta, A.; Narayan, S.; Joseph, K.; Khan, S.; Khan, F. S.; and Shah, M. 2022. Ow-detr: Open-world detection transformer. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 9235–9244. He, W.; Wu, C.; and Bensalem, S. 2024. Box-based monitor approach for out-of-distribution detection in YOLO: an exploratory study. In International Conference on Runtime Verification (RV), 229–239. Springer. He, W.; Wu, C.; Cheng, C.-H.; Huang, X.; and Bensalem, S. 2025. Mitigating hallucinations in YOLO-based object detection models: A revisit to out-of-distribution detection. In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 9870–9877. IEEE. Hendrycks, D.; Basart, S.; Mazeika, M.; Zou, A.; Kwon, J.; Mostajabi, M.; Steinhardt, J.; and Song, D. 2022. Scaling Out-of-Distribution Detection for Real-World Settings. In International Conference on Machine Learning (ICML), 8759–8773. PMLR. Hendrycks, D.; and Gimpel, K. 2017. A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks. In International Conference on Learning Representations (ICLR). Kuznetsova, A.; Rom, H.; Alldrin, N.; Uijlings, J.; Krasin, I.; Pont-Tuset, J.; Kamali, S.; Popov, S.; Malloci, M.; Kolesnikov, A.; et al. 2020. The open images dataset v4: Unified image classification, object detection, and visual relationship detection at scale. International Journal of Computer Vision (IJCV), 128(7): 1956–1981. Lee, K.; Lee, H.; Lee, K.; and Shin, J. 2018a. Training confidence-calibrated classifiers for detecting out-ofdistribution samples. In 6th International Conference on Learning Representations(ICLR). Lee, K.; Lee, K.; Lee, H.; and Shin, J. 2018b. A simple unified framework for detecting out-of-distribution samples and adversarial attacks. Advances in Neural Information Processing Systems (NeurIPS), 31. Li, J.; Wu, J.; Zhao, W.; Bai, S.; and Bai, X. 2024. PartGLEE: A foundation model for recognizing and parsing any objects. In European Conference on Computer Vision (ECCV), 475– 494. Springer. Li, Y.; Motamedi, A.; Kwon, H. J.; Park, C. B.; and Kim, I.-M. 2026. UNI-OOD: Unified Object- and Imagelevel Out-of-Distribution Detection via Cross-Context Attentive Vision-Language Modeling. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 6282–6292. Lin, T.-Y.; Maire, M.; Belongie, S.; Hays, J.; Perona, P.; Ramanan, D.; Dollár, P.; and Zitnick, C. L. 2014. Microsoft

rection for OoD detection and suggests that exposing latent model knowledge may provide a general strategy for improving the reliability and interpretability of foundation models. While this work focuses on two representative sources of OoD-induced hallucinations, namely proximal OoD objects and background-induced false predictions, the proposed prior elicitation framework can naturally be extended to additional hallucination sources and broader reliability challenges. Future work will investigate how to uncover richer latent priors and how to better leverage the elicited knowledge to prevent OoD hallucination generation.

Acknowledgments Funded by the European Union. Views and opinions expressed are however those of the author(s) only and do not necessarily reflect those of the European Union or the European Health and Digital Executive Agency (HADEA). Neither the European Union nor the granting authority can be held responsible for them. RobustifAI project, ID 101212818.

References Alain, G.; and Bengio, Y. 2017. Understanding Intermediate Layers Using Linear Classifier Probes. In 5th International Conference on Learning Representations (ICLR), Workshop Track Proceedings. Bau, D.; Zhou, B.; Khosla, A.; Oliva, A.; and Torralba, A. 2017. Network Dissection: Quantifying Interpretability of Deep Visual Representations. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 6541–6549. Benenson, R.; Popov, S.; and Ferrari, V. 2019. Large-scale interactive object segmentation with human annotators. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (CVPR), 11700–11709. Benou, I.; and Riklin Raviv, T. 2025. Show and Tell: Visually Explainable Deep Neural Nets via Spatially-Aware Concept Bottleneck Models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 30063–30072. Cimpoi, M.; Maji, S.; Kokkinos, I.; Mohamed, S.; and Vedaldi, A. 2014. Describing Textures in the Wild. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Dai, B.; Chen, C.; and Yu, Y. 2026. Mitigating Simplicity Bias in OOD Detection through Object Co-occurrence Analysis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 20345– 20355. Du, X.; Gozum, G.; Ming, Y.; and Li, Y. 2022a. Siren: Shaping representations for detecting out-of-distribution objects. Advances in Neural Information Processing Systems (NeurIPS), 35: 20434–20449. Du, X.; Wang, Z.; Cai, M.; and Li, S. 2022b. Towards Unknown-aware Learning with Virtual Outlier Synthesis. In International Conference on Learning Representations (ICLR). 8

coco: Common objects in context. In European Conference on Computer Vision (ECCV), 740–755. Springer. Liu, F. T.; Ting, K. M.; and Zhou, Z.-H. 2008. Isolation forest. In IEEE International Conference on Data Mining (ICDM), 413–422. IEEE. Liu, W.; Wang, X.; Owens, J.; and Li, Y. 2020. Energy-based out-of-distribution detection. Advances in Neural Information Processing Systems (NeurIPS), 33: 21464–21475. Lv, X.; Zhang, S.; Xing, Y.; Xu, D.; Wang, P.; and Zhang, Y. 2026. Knowing the Unknown: Interpretable Open-World Object Detection via Concept Decomposition Model. In International Conference on Learning Representations (ICLR). Macêdo, D.; Zanchettin, C.; and Ludermir, T. 2022. Distinction maximization loss: Efficiently improving outof-distribution detection and uncertainty estimation by replacing the loss and calibrating. arXiv preprint arXiv:2205.05874, 40. Minderer, M.; Gritsenko, A.; and Houlsby, N. 2023. Scaling open-vocabulary object detection. Advances in Neural Information Processing Systems (NeurIPS), 36: 72983–73007. Oikarinen, T.; Das, S.; Nguyen, L. M.; and Weng, T.-W. 2023. Label-Free Concept Bottleneck Models. In International Conference on Learning Representations (ICLR). Peng, T.; Dong, J.; and Ong, Y.-S. 2026. Learning Latent Concepts for Detecting Out-of-Distribution Objects. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 28723–28733. Ravi, N.; Gabeur, V.; Hu, Y.-T.; Hu, R.; Ryali, C.; Ma, T.; Khedr, H.; Rädle, R.; Rolland, C.; Gustafson, L.; et al. 2025. Sam 2: Segment anything in images and videos. In International Conference on Learning Representations (ICLR), volume 2025, 28085–28128. Ren, S.; He, K.; Girshick, R.; and Sun, J. 2015. Faster RCNN: Towards real-time object detection with region proposal networks. Advances in Neural Information Processing Systems (NeurIPS), 28. Ren, T.; Liu, S.; Zeng, A.; Lin, J.; Li, K.; Cao, H.; Chen, J.; Huang, X.; Chen, Y.; Yan, F.; et al. 2024. Grounded sam: Assembling open-world models for diverse visual tasks. arXiv preprint arXiv:2401.14159. Sastry, C. S.; and Oore, S. 2020. Detecting out-ofdistribution examples with gram matrices. In International Conference on Machine Learning (ICML), 8491– 8501. PMLR. Schrodi, S.; Schur, J.; Argus, M.; and Brox, T. 2025. Selective Concept Bottleneck Models Without Predefined Concepts. Transactions on Machine Learning Research (TMLR). Shao, S.; Li, Z.; Zhang, T.; Peng, C.; Yu, G.; Zhang, X.; Li, J.; and Sun, J. 2019. Objects365: A large-scale, high-quality dataset for object detection. In Proceedings of the IEEE/CVF international conference on computer vision (ICCV), 8430– 8439. Sitawarin, C.; Pongmala, K.; Chen, Y.; Carlini, N.; and Wagner, D. 2023. Part-Based Models Improve Adversarial Robustness. In International Conference on Learning Representations (ICLR).

Sudre, C. H.; Li, W.; Vercauteren, T.; Ourselin, S.; and Jorge Cardoso, M. 2017. Generalised dice overlap as a deep learning loss function for highly unbalanced segmentations. In International Workshop on Deep Learning in Medical Image Analysis (DLMIA), 240–248. Springer. Sun, P.; Chen, S.; Zhu, C.; Xiao, F.; Luo, P.; Xie, S.; and Yan, Z. 2023. Going denser with open-vocabulary part segmentation. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 15453–15465. Sun, Y.; Ming, Y.; Zhu, X.; and Li, Y. 2022. Out-ofdistribution detection with deep nearest neighbors. In International Conference on Machine Learning (ICML), 20827– 20840. PMLR. Tack, J.; Mo, S.; Jeong, J.; and Shin, J. 2020. Csi: Novelty detection via contrastive learning on distributionally shifted instances. Advances in neural information processing systems (NeurIPS), 33: 11839–11852. Wang, A.; Chen, H.; Liu, L.; Chen, K.; Lin, Z.; Han, J.; et al. 2025a. Yolov10: Real-time end-to-end object detection. Advances in Neural Information Processing Systems (NeurIPS), 37: 107984–108011. Wang, A.; Liu, L.; Chen, H.; Lin, Z.; Han, J.; and Ding, G. 2025b. Yoloe: Real-time seeing anything. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 24591–24602. Wilson, S.; Fischer, T.; Dayoub, F.; Miller, D.; and Sünderhauf, N. 2023. Safe: Sensitivity-aware features for out-ofdistribution object detection. In IEEE/CVF International Conference on Computer Vision (ICCV), 23565–23576. Wu, A.; Chen, D.; and Deng, C. 2023. Deep feature deblurring diffusion for detecting out-of-distribution objects. In Proceedings of the IEEE/CVF international conference on computer vision (ICCV), 13381–13391. Wu, A.; and Deng, C. 2023. TIB: Detecting unknown objects via two-stream information bottleneck. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 46(1): 611–625. Wu, A.; and Deng, C. 2025. Percept, memory, and imagine: World feature simulating for open-domain unknown object detection. In Proceedings of the Computer Vision and Pattern Recognition Conference (CVPR), 4682–4691. Wu, C.; Falcone, Y.; and Bensalem, S. 2023. Customizable reference runtime monitoring of neural networks using resolution boxes. In International Conference on Runtime Verification (RV), 23–41. Springer. Wu, C.; He, W.; Cheng, C.-H.; Huang, X.; and Bensalem, S. 2024. BAM: box abstraction monitors for real-time OoD detection in object detection. In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2632– 2638. IEEE. Wu, C.; He, W.; Cheng, C.-H.; Huang, X.; and Bensalem, S. 2026. Revisiting Out-of-Distribution Detection in RealTime Object Detection: From Benchmark Pitfalls to a New Mitigation Paradigm. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 48(5): 5243–5260. 9

Xu, K.; Chen, R.; Franchi, G.; and Yao, A. 2024. Scaling for Training Time and Post-hoc Out-of-distribution Detection Enhancement. In International Conference on Learning Representations (ICLR), volume 2024, 16316–16329. Yu, F.; Chen, H.; Wang, X.; Xian, W.; Chen, Y.; Liu, F.; Madhavan, V.; and Darrell, T. 2020. Bdd100k: A diverse driving dataset for heterogeneous multitask learning. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2636–2645. Zhao, Y.; Lv, W.; Xu, S.; Wei, J.; Wang, G.; Dang, Q.; Liu, Y.; and Chen, J. 2024. DETRs beat YOLOs on real-time object detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 16965– 16974. Zhu, L.; Yang, Y.; Nie, Z.; Gao, Y.; Li, J.; Gu, Q.; Wang, X.; Zhou, C.; and Ye, N. 2025. InfoBound: A Provable Information-Bounds Inspired Framework for Both OoD Generalization and OoD Detection. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 47(11): 10227–10242. Zhu, X.; Su, W.; Lu, L.; Li, B.; Wang, X.; and Dai, J. 2021. Deformable {DETR}: Deformable Transformers for Endto-End Object Detection. In International Conference on Learning Representations (ICLR).

10

Appendix A A.1

matic part annotation methods, achieving the highest overall mean Intersection over Union (mIoU) and Recall@0.5. Compared with the strongest baseline, VLPart (Sun et al. 2023), it improves mIoU and Recall@0.5 by 3.8 and 1.6 percentage points, respectively. The improvement is consistent across object categories, with our tool achieving the best ConceptmIoU on six of the seven evaluated classes. These results demonstrate that the proposed tool can automatically generate accurate part-level semantic annotations, making semantic elicitation practical without requiring extensive manual annotation.

Data Preparation

Proximal mining Proximal OoD objects are one of the main sources of detector hallucinations because they share semantic or visual characteristics with ID categories. To curate such data, we follow the prompt design introduced in (Wu et al. 2026) and query GPT-5 to generate a set of proximal categories set for each ID category. The proposed proximal categories should be disjoint from the ID categories while being semantically or visually similar to the corresponding ID category. For each ID category, we use the obtained proximal category set to retrieve 1,000 annotated images from Objects365 (Shao et al. 2019), a large-scale object detection dataset covering diverse object categories. As our evaluation benchmarks are constructed from Open Images V7 (Benenson, Popov, and Ferrari 2019), using Objects365 as the source of training data helps avoid any data leakage issue. We run the target object detector on these images and retain images that induce hallucinated ID predictions.

A.2

Implementation Details of Prior Elicitation

Multi-Scale RoI Feature Extraction An important requirement of our semantic elicitation head is to reconstruct spatial concept evidence inside each detected proposal, which makes the choice of proposal-level representation crucial. Let pi = (bi , yi ) denote a detector prediction and let Fi denote the RoI feature associated with its predicted bounding box bi . For YOLO and RT-DETR, we extract features from three spatial feature levels, denoted by {Eℓ }L ℓ=1 , where

Background mining OoD hallucinations in object detection may also occur when object detectors encounter backgrounds with textures or local cues resembling those of ID objects. To capture this failure mode, we collect background-only samples from the Describable Textures Dataset (DTD) (Cimpoi et al. 2014). DTD is an imagelevel texture recognition dataset containing diverse realworld texture patterns, making it a good source for probing background-induced detector hallucinations. Since DTD is an image classification dataset without object-level annotations and may contain ID objects, we first employ YOLOE11-L (Wang et al. 2025b) to filter out images containing ID objects.

Eℓ ∈ RCℓ ×Hℓ ×Wℓ Specifically, we use the detection-neck outputs for YOLO and the multi-scale hybrid-encoder outputs for RT-DETR. For the latter, we use the hybrid-encoder features instead of the final decoder queries because decoder queries do not define a canonical spatial grid within the predicted bounding box (Zhao et al. 2024). For each prediction, RoIAlign is independently applied to every feature level:
Fℓi = RoIAlign Eℓ , bi ; 7 × 7 ∈ RCℓ ×7×7 . (3) The multi-scale RoI features are then concatenated along the channel dimension:

Part-Level Automated Annotation To train the semantic elicitation head, we require part-level semantic annotations that capture the visual evidence associated with object detector predictions. Since manually annotating such concepts is labor-intensive and difficult to scale, we develop an automated part-level annotation tool, illustrated in Fig. 4, to efficiently generate the required supervision. Given an object category, our tool first queries GPT-5 following the prompt design of (Oikarinen et al. 2023) to generate a compact vocabulary of visually localizable part-level concepts. For example, for the category bird, the generated concepts include beak, eye, wing, torso, and feet. The tool then grounds each concept within the corresponding Region of Interest (RoI) using OWLv2 (Minderer, Gritsenko, and Houlsby 2023). Each grounded part is subsequently refined into a pixel-level segmentation mask by SAM 2 (Ravi et al. 2025). To suppress background-induced annotations, the generated part mask is retained only if at least 70% of its pixels overlap with the corresponding object mask predicted by SAM 2. Finally, each retained part mask is projected into object detector-RoI coordinates and converted into the concept-level supervision used to train the semantic elicitation head. Concepts without valid annotations are assigned empty supervision. We evaluate the quality of the proposed annotation tool in Table 5. Our tool consistently outperforms existing auto-

L X
ℓ D×7×7 Cℓ . , D= Fi = ConcatL ℓ=1 Fi ∈ R

(4)

ℓ=1

This produces an RoI representation that combines finegrained spatial evidence with coarser semantic context. For Faster R-CNN, we directly use the RoI feature produced by the Detectron2 box_pooler. Following the standard FPN assignment strategy, each predicted box is assigned to an appropriate pyramid level and pooled into a 256 × 7 × 7 feature map. Table 6 summarizes the feature sources, channels, and resulting RoI feature dimensions for all detector architectures. Part-Level Semantic Concept Learning. For YOLOv10, the multi-scale RoI feature map has dimension Fi ∈ R896×7×7 . The resulting representation is processed by a lightweight semantic elicitation head comprising a 1 × 1 projection layer, two residual convolutional blocks, and a final 1 × 1 prediction layer. The projection layer reduces the channel dimension from 896 to 256, while each residual block applies a 3 × 3 convolution followed by GroupNorm, GELU, and Dropout2d. The head produces a spatial conceptlogit tensor z ∈ RC×7×7 , where C denotes the size of the corresponding part-level concept vocabulary. Because part 11

Figure 3: Automated part-level annotation pipeline. Given an RoI crop and its object category, OWLv2 grounds the corresponding GPT-5-generated concept vocabulary into part bounding boxes. Each box prompts SAM 2 to produce a refined pixel-level part mask. Masks satisfying the object-mask coverage threshold are projected into detector-RoI coordinates and rasterized as binary 7 × 7 targets. Concepts without a retained mask receive an all-zero target. Table 5: Quality of the automated part-level annotations. We report the overall concept-mIoU, Recall@0.5, number of covered concepts, and per-class concept-mIoU. All values except concept coverage are percentages. Best results are shown in bold. Overall

Method

Per-class concept-mIoU

concept-mIoU Recall@0.5 # Concepts bird Ours VLPart (Sun et al. 2023) Grounded SAM (Ren et al. 2024)

40.8 37.0 32.3

38.4 36.8 30.7

25 25 25

Table 6: RoI feature sources and dimensions for each detector. YOLO and RT-DETR concatenate RoI-aligned features from three feature scales, whereas Faster R-CNN uses the Detectron2 box_pooler. Detector

Feature source

Channels

RoI feature Fi

YOLO RT-DETR Faster R-CNN

Detect neck Hybrid-encoder neck ResNet-FPN

128 + 256 + 512 256 + 256 + 256 256

896 × 7 × 7 768 × 7 × 7 256 × 7 × 7

bus

car

cat

cow

dog

horse

46.5 37.9 43.9 40.4 37.4 41.8 35.6 32.8 32.5 38.1 36.8 44.1 32.1 31.7 31.3 33.0 29.5 38.3

38.2 36.5 29.4

catenate these statistics across scales to form an embedding vector v, which is subsequently ℓ2 -normalized. The detectorspecific feature sources and resulting embedding dimensions are summarized in Table 8. Using ID training images, we organize the normalized embeddings into class-specific refy }y∈Y . At retrieval time, neighbors are erence banks {VID ranked by cosine similarity. We retain its k nearest references and define the contextual distance as X 1 cos(v, r) , (5) dctx (v) = 1 − k ŷ r∈Nk (v,VID ))

vocabularies are inherently class-specific, e.g., wing, tail, and beak are relevant to birds, whereas body and cap are relevant to bottles, we train an independent semantic elicitation head for each ID category. To obtain an RoI-level activation for each concept, we aggregate its spatial logit map using log-sum-exp. The resulting vector captures the activation of category-specific semantic concepts within the RoI and is subsequently used as the semantic prior in SPK. Each head is trained for up to 80 epochs with early stopping. Training all heads takes approximately one hour on a single NVIDIA A100 GPU. Additional architectural and training hyperparameters of the semantic elicitation head are summarized in Table 7.

ŷ where ŷ is the predicted class, and Nk (v, VID ) contains the k nearest reference embeddings from the class-specific refŷ erence bank VID . We set k = 5 in all experiments. A small contextual distance indicates that the query occurs in a context similar to those observed for the predicted class during ID training, whereas a large distance indicates an atypical context. We use this distance as the contextual prior in SPK.

A.3

Semantic Prior Learning Analysis

We evaluate the learned semantic elicitation heads from two complementary perspectives: (1) the quality of the elicited part-level semantic responses and (2) the discriminative capability of the learned semantic groups. Fig. 5 visualizes the concept activation maps predicted by the bird-specific semantic elicitation head. The activations corresponding to concepts such as wing, torso, and foot are well aligned with their anatomical regions, indicating that the semantic head

Contextual Prior Learning. To elicit contextual priors, we treat the entire image as an RoI and construct an imagelevel representation from the detector’s multi-scale neck features. For each feature scale, we compute the channel-wise mean and standard deviation over all spatial cells and con12

Figure 4: Part-Level Concept Annotation Example. We use a bird RoI crop to illustrate the step-by-step annotation process for the concepts wing, head, beak, and torso. Step 1: OWLv2 grounds each concept within the detector RoI and generates a bounding-box proposal (a). Step 2: SAM 2 refines each proposal into a pixel-level part mask, which is retained only if at least 70% of its pixels overlap with the corresponding SAM 2 object mask (b). Step 3: Each retained mask is projected into detector-RoI coordinates and converted into a binary 7 × 7 supervision target (c). Step 4: The binary target is overlaid on the RoI crop for visualization (d). successfully decodes spatially grounded part-level semantic evidence from the detector RoI representation.

ing to their underlying semantic source. Although the accuracy decreases on unseen validation samples, the overall trend confirms that the proposed group objective successfully captures the intended semantic grouping.

We further evaluate whether the learned semantic responses are correctly summarized into the three semantic groups introduced by the group objective. Table 9 reports the group prediction accuracy on ID, proximal OoD, and background samples. The semantic head achieves high group prediction accuracy on the training data, indicating that the learned concept responses are effectively organized accord-

Fig. 6 further analyzes the learned semantic group responses on the training data. ID samples are dominated by the ID semantic group, while the responses of the proximal and background groups remain largely suppressed. Proximal OoD samples predominantly activate the proximal semantic 13

Table 7: Hyperparameters of the Semantic Elicitation Head. Semantic Elicitation Head

Architecture

Training

Hyperparameter Hidden channels Dropout Normalization Activation Residual blocks Output head Inference activation RoI spatial size Inference pooling Training epochs Batch size Optimizer Learning rate Weight decay Random seed Validation split Training sampler Suppress loss weight Group loss weight Early-stopping patience

Value 256 0.1 GroupNorm GELU 2 1 × 1 conv → num_concepts Sigmoid 7×7 LogSumExp, τ = 0.5 80 2000 AdamW 2 × 10−4 5 × 10−4 42 10% WeightedRandomSampler 0.25 0.75 10 epochs

Table 8: Image-level embedding sources and dimensions for each detector. We globally pool each selected feature map by its spatial mean and standard deviation, then concatenate the resulting statistics. YOLO and RT-DETR use their deepest selected backbone stage, whereas Faster R-CNN aggregates all four ResNet-FPN levels. Detector

Feature source

Channels

Embedding dimension

YOLO RT-DETR Faster R-CNN

Backbone L6 (stride 16) HGBlock L9 backbone (stride 32) ResNet-FPN P2–P5

256 2048 4 × 256

mean + std : 512 mean + std : 4096 mean + std : 2048

Table 9: Semantic-head group-classification accuracy. Computed by applying arg max to concept activations for YOLO on PASCAL-VOC. Data split ID training set Proximal OoD Background

A.4 Comprehensive Results on the Calibrated Benchmark Effectiveness of SPK under AUROC Table 10 presents the OoD detection results measured by AUROC. Consistent with the FPR95 results reported in the main paper, these results further validate that SPK provides a more effective representation space for OoD detection when paired with the same OoD detection methods. The improved OoD detection performance of SPK is further supported by the enhanced separability observed in the learned representation space. Fig. 7 visualizes the representation spaces of the object detector classification logits and SPK representations for detections predicted as dog, sheep, and cat by a YOLO detector trained on PASCAL-VOC. While ID and OoD samples largely overlap in the conventional classification-logit space commonly used for OoD detection, SPK yields a more structured representation space with clearer ID–OoD separation.

Accuracy (%) 96.5 81.0 77.0

group, with a small fraction also exhibiting strong ID responses due to their high semantic similarity to ID objects. Likewise, background samples are characterized by dominant background responses and consistently low activations for the ID and proximal groups. These results demonstrate that the learned semantic groups successfully capture the underlying source of the visual evidence and provide meaningful semantic priors for constructing the SPK representation.

Qualitative Examples Fig. 8 provides qualitative examples illustrating how SPK characterizes different sources of detector hallucinations despite high-confidence predic14

Table 10: AUROC comparison across detector architectures on PASCAL-VOC and BDD-100K. Higher is better. Faster R-CNN

YOLO Method

PASCAL-VOC

BDD-100K

PASCAL-VOC

RT-DETR

BDD-100K

PASCAL-VOC

BDD-100K

Near-OoD Far-OoD Near-OoD Far-OoD Near-OoD Far-OoD Near-OoD Far-OoD Near-OoD Far-OoD Near-OoD Far-OoD MSP EBO MLS SCALE MDS BAM KNN iForest

81.24 60.73 59.41 69.84 85.62 90.14 89.52 77.38

79.47 57.92 61.26 71.73 77.81 89.38 90.31 80.71

77.63 65.41 63.78 72.46 80.34 89.57 91.26 82.46

75.12 62.86 65.02 79.31 68.46 86.94 88.47 80.19

78.71 82.58 84.76 57.49 88.39 80.96 81.73 76.31

73.84 86.14 83.58 69.72 84.41 91.56 92.58 85.64

72.48 52.07 54.82 64.18 73.68 80.75 88.43 81.37

76.39 50.83 59.46 67.82 70.94 83.65 86.72 83.29

79.16 40.12 56.37 69.37 87.43 75.75 75.12 70.82

78.72 43.31 59.74 75.16 85.71 81.65 81.46 81.94

74.31 49.68 54.91 66.42 73.27 76.50 83.68 78.46

75.06 52.74 57.43 65.73 71.18 80.45 85.29 83.57

SPK-MDS SPK-BAM SPK-KNN SPK-iForest

96.12 94.55 94.91 96.31

93.55 92.50 93.31 95.43

95.80 96.90 97.40 98.10

98.65 99.35 99.71 99.81

96.10 96.45 97.07 97.38

94.20 95.40 96.46 97.25

92.10 98.30 99.06 99.50

91.80 98.85 99.36 99.65

94.35 93.75 94.40 95.23

95.54 93.90 94.87 95.67

89.10 93.45 93.94 95.78

99.21 99.18 99.36 99.60

(a) Input + ROI

(b) Prediction: wing

(c) Prediction: torso

(d) Prediction: foot

Figure 6: Distributions of the learned semantic group responses. Samples from different data sources predominantly activate their corresponding semantic groups, validating the effectiveness of the proposed group objective.

Figure 5: Qualitative visualization of part-level semantic responses. The learned concept maps are well aligned with the corresponding regions, demonstrating that the semantic elicitation head successfully decodes spatially grounded semantic evidence from detector RoI features.

SPK not only detects OoD-induced hallucinations but also provides interpretable evidence about which prior components contribute to each rejection decision. Ablation on Semantic Prior Learning Additional ablation results on Faster R-CNN and RT-DETR are presented in Table 11, further demonstrating the effectiveness of jointly optimizing the three proposed loss components.

tions. For Near-OoD examples, the hallucinated detections exhibit strong proximal-concept responses, while ID and background concept responses remain suppressed, indicating that these false positives are primarily caused by their semantic similarity to known categories. In contrast, FarOoD examples lack coherent object-level semantics and are dominated by strong background responses with negligible ID or proximal activations. The geometric and contextual priors provide additional complementary evidence by capturing object-scale and image-level inconsistencies. Based on the resulting SPK representations, the OoD detector successfully rejects these hallucinated detections despite their high confidence scores. These examples demonstrate that

Ablation on Prior Components. Additional ablation results on Faster R-CNN and RT-DETR are presented in Table 12, further confirming the effectiveness of jointly combining the three types of priors.

A.5 Experimental Results on Uncalibrated Benchmark To facilitate comparison with a broader range of existing OoD detection methods, we additionally evaluate SPK under the 15

Table 11: Ablation of SPK loss components. Evaluated across Faster R-CNN and RT-DETR on PASCAL-VOC and BDD-100K. Lower FPR95 is better. Faster R-CNN Ldice Lsuppress Lgroup

PASCAL-VOC

RT-DETR

BDD-100K

PASCAL-VOC

BDD-100K

Near-OoD Far-OoD Near-OoD Far-OoD Near-OoD Far-OoD Near-OoD Far-OoD ✗ ✓ ✓ ✓

✓ ✓ ✗ ✓

✓ ✗ ✓ ✓

27.87 24.42 22.19 13.92

19.98 15.42 14.73 10.52

18.36 14.15 11.34 2.31

18.74 16.11 13.27 1.52

32.31 27.86 24.86 15.48

27.08 22.53 21.64 17.32

25.35 20.71 18.15 11.42

29.14 24.36 20.81 9.25

Table 12: Ablation of SPK prior components. Evaluated across Faster R-CNN and RT-DETR on PASCAL-VOC and BDD100K. Lower FPR95 is better. Prior components

Faster R-CNN PASCAL-VOC

RT-DETR

BDD-100K

PASCAL-VOC

BDD-100K

Near-OoD Far-OoD Near-OoD Far-OoD Near-OoD Far-OoD Near-OoD Far-OoD Semantic Semantic + Geometric Semantic + Geometric + Contextual

18.17 15.78 13.92

18.91 16.43 10.52

23.92 14.52 2.31

conventional uncalibrated benchmark adopted by prior work. While our main experiments follow the calibrated benchmark proposed in (Wu et al. 2026), several previous methods are architecture-specific, require specialized training protocols, or lack publicly available implementations, making direct evaluation under the calibrated benchmark difficult. Evaluating under their original protocol therefore enables a broader and fairer comparison with the existing literature, despite the known annotation issues of this benchmark discussed in (Wu et al. 2026). Following (Peng, Dong, and Ong 2026), we evaluate Deformable-DETR on PASCAL-VOC and BDD-100K, and Faster R-CNN on PASCAL-VOC, with MS-COCO (Lin et al. 2014) and OpenImages (Kuznetsova et al. 2020) as OoD test sets. Tables 13 and 14 show that SPK variants consistently achieve state-of-the-art or highly competitive performance across all evaluation settings. Among the twelve evaluation cases, SPK variants achieves the best performance in ten, while ranking second in the remaining case. Some existing methods additionally exploit external visual encoders for OoD detection rather than relying solely on detectorintrinsic representations. To enable a fair comparison under this setting, we introduce SPK (DINO ViT), which replaces the detector-intrinsic contextual prior with an image-level representation extracted by DINO ViT. This variant follows the same external-encoder setting as methods such as UNOAdapter while retaining the semantic and geometric priors of SPK.

A.6

25.21 15.71 1.52

18.34 15.85 15.48

29.17 25.35 17.32

33.36 26.51 11.42

34.75 26.82 9.25

ms per image on an NVIDIA A4000-8GB GPU. By extracting detection outputs, image-level contextual embeddings, and RoI features within the same forward pass, SPK inference takes 12.65 ms. Semantic prior elicitation and Isolation Forest inference introduce only marginal additional latency. Although detector inference and RoI feature extraction costs may vary across object detector architectures, the remaining components have comparable computational costs across detectors, as they operate on compact representations with lightweight additional modules.

Inference Runtime Analysis.

Table 15 reports the per-image runtime of SPK and its individual components for a YOLO model pretrained on PASCAL-VOC. The original detector inference takes 10.15 16

Table 13: Comparison with competitive OoD detection methods for Deformable-DETR. Results are reported on PASCALVOC and BDD-100K as ID datasets, with MS-COCO and OpenImages as OoD datasets. Higher AUROC and lower FPR95 indicate better OoD detection performance. Methods marked with † employ an external DINO ViT encoder to extract additional visual representations for OoD detection, rather than relying solely on Deformable-DETR features. SPK (DINO ViT) is included to provide a fair comparison with UNO-Adapter, as both methods operate under this setting. The best results are highlighted in bold. ID: BDD-100K

ID: PASCAL-VOC Method

OoD: MS-COCO

OoD: OpenImages

OoD: MS-COCO

OoD: OpenImages

FPR95↓ AUROC↑ FPR95↓ AUROC↑ FPR95↓ AUROC↑ FPR95↓ AUROC↑ MDS (Lee et al. 2018b) Gram matrices (Sastry and Oore 2020) KNN (Sun et al. 2022) CSI (Tack et al. 2020) VOS (Du et al. 2022b) OW-DETR (Gupta et al. 2022) DisMax (Macêdo, Zanchettin, and Ludermir 2022) SIREN-vMF (Du et al. 2022a) SIREN-KNN (Du et al. 2022a) SAFE (Wilson et al. 2023) InfoBound (Zhu et al. 2025) UNO-Adapter† (Peng, Dong, and Ong 2026)

97.39 94.16 91.80 84.00 97.46 93.09 82.05 75.49 64.77 48.88 44.88 32.61

50.28 43.97 62.15 55.07 54.40 55.70 75.21 76.10 78.23 78.88 89.76 91.68

97.88 95.29 91.36 79.16 97.07 93.82 76.37 78.36 65.99 8.99 43.89 19.90

49.08 38.81 59.64 51.37 52.77 57.80 70.66 71.05 74.93 96.73 88.00 95.40

70.86 73.81 64.75 70.27 76.44 80.78 77.62 67.54 53.97 39.18 44.88 9.88

76.83 60.13 80.90 77.93 77.33 70.29 72.14 80.06 86.56 85.95 89.76 97.61

71.43 71.56 61.13 71.30 72.58 77.37 81.23 66.31 47.28 21.10 43.89 3.80

77.98 57.14 79.64 76.42 76.62 73.78 67.18 79.77 89.00 94.31 88.00 99.04

SPK SPK (DINO ViT)†

52.32 28.55

75.84 92.38

24.38 14.85

90.20 96.25

1.68 0.00

99.42 99.80

0.37 0.00

99.93 99.97

Table 14: Comparison with competitive OoD detection methods for Faster R-CNN. Results are reported on PASCAL-VOC as the ID dataset, with MS COCO and OpenImages as OoD datasets. Higher AUROC and lower FPR95 indicate better OoD detection performance. Methods marked with † employ an external DINO ViT encoder to extract additional visual representations for OoD detection, rather than relying solely on Faster R-CNN features. SPK (DINO ViT) is included to provide a fair comparison with UNO-Adapter, as both methods operate under this setting. The best results are highlighted in bold. Method CSI (Tack et al. 2020) GAN-Synthesis (Lee et al. 2018a) VOS (Du et al. 2022b) SIREN (Du et al. 2022a) TIB (Wu and Deng 2023) DFDD (Wu, Chen, and Deng 2023) WFS (Wu and Deng 2025) UNO-Adapter† (Peng, Dong, and Ong 2026) SPK SPK (DINO ViT)†

MS-COCO AUROC↑ FPR95↓ 82.95 57.41 82.67 59.97 85.23 51.33 85.36 64.68 90.36 41.55 90.79 41.34 89.01 40.05 91.25 38.73 91.58 41.20 95.48 26.28

OpenImages AUROC↑ FPR95↓ 81.83 59.91 83.67 60.93 88.70 47.53 82.78 68.53 88.09 47.19 88.65 44.52 90.35 39.17 92.40 35.74 95.50 25.61 98.16 10.97

Table 15: Per-image runtime of the complete SPK inference pipeline. Reported for a YOLO model pretrained on PASCALVOC. The complete SPK pipeline introduces an additional 2.72 ms latency per image, corresponding to a 26.8% runtime overhead over the original detector inference. * SPK inference obtains detection outputs, image-level contextual embeddings, and RoI features within the same forward pass.

Component

Cost (ms)

Original inference SPK inference* Semantic prior elicitation Isolation Forest

17

10.15 12.65 0.17 0.05

(a) dog

(b) sheep

(c) cat

Figure 7: UMAP visualization of representation spaces for detections from the dog, sheep, and cat categories. The visualizations are obtained using YOLO trained on PASCAL-VOC. We compare detector classification logits (left) with the proposed SPK representations (right), using the same ID-validation, Near-OoD, and Far-OoD samples. SPK yields a more structured representation space with clearer distributional differences between ID and OoD samples.

18

Figure 8: Qualitative SPK OoD verdicts for OoD-induced hallucination. Results are shown for YOLO on PASCAL-VOC. Near-OoD examples (left) exhibit strong proximal-concept responses, whereas Far-OoD examples (right) are dominated by background-concept responses. Each example shows the predicted bounding box, detector confidence, and five SPK prior components. Green bands represent class-specific normal ranges estimated from ID training data, red bands indicate values outside these ranges, and circular markers denote the observed component values. All examples are correctly rejected as OoD.

19

Related documents

• Attribution Gaps in Zero-Training LLM+OVOD Pipelines: A Fine-Grained Analysis of the CAAP--SNAP Discrepancy#1116898
• Exploiting Stability-Plasticity Asymmetry in Pretrained Detectors for Incremental Object Detection#820196
• Achieving Near-Zero-Overhead Multi-Model Hierarchical Classification in Real-Time Detection Pipelines#447913
• PiVoT: A Variational Solution for Real-time Large-scale Multi-object Detection and Tracking under Heavy Clutter#370333
• Albireo: Adaptive, Energy-Efficient Inference Framework for Video Object Detection on the Edge#1077576