ForestQuery: Boundary-Aware and Spatially Anchored Query Learning for Unified Forest Point Cloud Segmentation

ForestQuery: Boundary-Aware and Spatially Anchored Query Learning for Unified Forest Point Cloud Segmentation Zhihao Zhana,b , Le Taob , Yifei Tianc , Xin Liud and Jie Yuana,∗ a School of Electronic Science and Engineering, Nanjing University, Nanjing, 210023, China b TopXGun Robotics, Nanjing, 211100, China c School of Computer Science, Nanjing University of Posts and Telecommunications, Nanjing, 210023, China

arXiv:2610.03403v1 [cs.CV] 2 Oct 2026

d Department of Computer Science and Technology, Tsinghua University, Beijing, 100084, China

ARTICLE INFO

ABSTRACT

Keywords: Forest Scene Understanding Point Cloud Segmentation Query Learning Forest Point Clouds

Forest point cloud segmentation is fundamental for fine-grained 3D forest scene understanding, yet remains challenging due to irregular tree structures, severe occlusions, density variations, and ambiguous instance boundaries. Recent query-based forest segmentation methods have shown promise for unified semantic and instance prediction, but they still insufficiently exploit forest-specific spatial structure and account for boundary uncertainty. In this paper, we propose ForestQuery, a boundary-aware and spatially anchored query learning framework for unified forest point cloud segmentation. ForestQuery enhances instance and semantic query learning through two complementary designs. Specifically, boundary uncertainty is explicitly modeled to guide reliable instance query construction and modulate query optimization through adaptive loss reweighting. Meanwhile, spatially anchored semantic query enhancement (SA-SQE) introduces learnable 3D anchors encoding forest vertical stratification priors to enrich semantic queries with explicit spatial references. We evaluate ForestQuery on multiple public forest point cloud benchmarks and a self-collected annotated real-world dataset. Extensive experiments demonstrate consistent improvements in both individualtree segmentation and semantic segmentation across diverse forest scenes. Code and data are publicly available at https://zhan994.github.io/ForestQuery.

1. Introduction Understanding complex 3D natural scenes is a fundamental problem in computer vision. Recent advances in 3D sensing and scene reconstruction have enabled large-scale outdoor environments to be represented as point clouds with rich geometric and structural information (Liu et al., 2021; Wilkes et al., 2023; Zhan et al., 2025, 2026). Forest point clouds are particularly challenging due to irregular spatial layouts, heterogeneous tree structures, severe occlusions, and ambiguous object boundaries. This work focuses on unified forest point cloud segmentation, which jointly performs point-wise semantic labeling of forest components and instance-level separation of individual trees. Such finegrained forest understanding provides essential support for forest inventory, ecological monitoring, and sustainable forest management (Waser et al., 2021; Xiang et al., 2024, 2025; Tan et al., 2026). General point cloud segmentation has been studied with a variety of representations, including point-based, voxelbased, sparse-convolutional, and Transformer-based architectures (Qi et al., 2017a,b; Tchapmi et al., 2017; Choy et al., 2019; Zhao et al., 2021). For instance segmentation, existing approaches can be broadly categorized into topdown, bottom-up, and query-based paradigms. Top-down methods first generate object proposals or bounding boxes and subsequently predict instance masks within individual ∗ Jie Yuan (yuanjie@nju.edu.cn) is the corresponding author.

ORCID (s): 0009-0003-9065-5261 (Z. Zhan); 0009-0004-7566-6584 (L. Tao); 0000-0001-8884-2170 (Y. Tian); 0000-0002-7658-284X (X. Liu); 0000-0003-0597-7657 (J. Yuan)

Zhan et al.: Preprint submitted to Elsevier

proposals (Yi et al., 2019; Hou et al., 2019), whereas bottomup methods typically predict point-wise embeddings, offsets, or affinities and assemble points into instances through clustering or hierarchical grouping (Jiang et al., 2020; Chen et al., 2021; Vu et al., 2022). These approaches often rely on accurate proposal generation or carefully designed grouping and post-processing strategies. More recently, query-based Transformer methods directly decode segmentation masks through interactions between learnable queries and scene features (Schult et al., 2023; Kolodiazhnyi et al., 2024). This formulation reduces reliance on explicit proposal generation and hand-crafted grouping, while naturally supporting unified semantic and instance segmentation. The query-based paradigm has also been extended to forest point clouds for joint semantic labeling and individual-tree segmentation (Xiang et al., 2025). As illustrated by the representative scenes in Fig. 1, unified forest point cloud segmentation must cope with complex spatial organization, heterogeneous vegetation structures, and closely interacting tree crowns. Despite recent progress, existing query-based formulations remain insufficiently tailored to these forest-specific characteristics. For instance segmentation, severe crown overlap and intertwined branches create ambiguous transition regions between neighboring trees, where points may contain mixed instance information. Queries initialized from such regions can therefore yield unreliable instance representations, while uniform supervision may further amplify the influence

Page 1 of 16

ForestQuery Leaf Wood Ground

Input

Semantic

Leaf Wood Ground

Instance

(a) TUWIEN

Input

Semantic

Instance

(b) Yuchen

Figure 1: Representative examples of unified forest point cloud segmentation. Examples from the TUWIEN and Yuchen scenes of FOR-instanceV2 illustrate the complex spatial organization of forest point clouds, where heterogeneous vegetation structures and overlapping tree crowns pose challenges to semantic understanding and individual-tree separation.

of boundary uncertainty during optimization. For semantic segmentation, forest scenes exhibit pronounced vertical stratification, with different semantic components distributed across distinct yet partially overlapping spatial layers. Existing semantic queries, however, are typically initialized without explicit spatial references and must recover these structural regularities solely through interactions with scene features. Such insufficient spatial anchoring makes semantic discrimination particularly difficult for components with similar local geometry. These observations motivate forest-specific query learning that explicitly accounts for boundary uncertainty while incorporating spatial structure into semantic representations. To address these challenges, we propose ForestQuery, a boundary-aware and spatially anchored query learning framework for unified forest point cloud segmentation. For instance segmentation, boundary uncertainty guides reliable query construction under ambiguous inter-tree boundaries. In parallel, spatially anchored semantic query enhancement (SA-SQE) incorporates learnable 3D anchors encoding forest vertical stratification priors into semantic queries. The resulting queries are jointly decoded with shared scene features, with boundary uncertainty further used to adaptively reweight instance-query supervision. Extensive experiments on multiple public benchmarks and a self-collected annotated real-world dataset demonstrate the effectiveness and cross-scene generalization of ForestQuery. The main contributions are summarized as follows: • We propose ForestQuery, a unified query-based framework for joint individual-tree instance segmentation and point-wise semantic labeling in complex forest scenes. • We develop a boundary-aware mechanism that explicitly models boundary uncertainty and exploits it in both instance query construction and optimization, enabling more reliable separation of densely overlapping trees. Zhan et al.: Preprint submitted to Elsevier

• We introduce SA-SQE, which incorporates learnable 3D anchors encoding forest vertical stratification priors, enriching semantic queries with explicit spatial references. • We evaluate ForestQuery on multiple public benchmarks and a self-collected annotated real-world dataset, covering diverse forest scenes and acquisition conditions. The remainder of this paper is organized as follows. Section 2 reviews related work on general point cloud segmentation and forest scene segmentation. Section 3 presents ForestQuery, covering boundary-aware instance query construction, spatially anchored semantic query enhancement, and unified query decoding and optimization. Section 4 reports the experimental setup, comparative evaluations, ablation studies, and generalization tests. Finally, Section 5 concludes the paper and outlines future work..

2. Related Work 2.1. Point Cloud Segmentation Point cloud segmentation is a fundamental task in 3D scene understanding. Early deep networks, represented by PointNet (Qi et al., 2017a) and PointNet++ (Qi et al., 2017b), established a direct point-based paradigm by learning representations from unordered point sets. Building on this foundation, subsequent methods strengthened local geometric modeling through convolutional operators defined over points or voxelized space. KPConv (Thomas et al., 2019) performs kernel point convolution over local neighborhoods, whereas SEGCloud (Tchapmi et al., 2017) adopts a volumetric representation and projects predictions back to raw points. To improve scalability in large-scale scenes, sparse-convolutional architectures exploit the sparsity of 3D data for efficient feature extraction (Choy et al., 2019), while RandLA-Net (Hu et al., 2020) combines random sampling

Page 2 of 16

ForestQuery

with local feature aggregation for efficient point-wise prediction. More recently, Transformer-based models further extend point cloud representation learning by capturing longrange contextual dependencies. Point Transformer (Zhao et al., 2021) introduces self-attention tailored to irregular point sets, Stratified Transformer (Lai et al., 2022) enlarges the effective receptive field through stratified key sampling, and Point Transformer V3 (PTv3) (Wu et al., 2024) further improves efficiency and scalability through serialized point-cloud processing. Collectively, these advances have substantially strengthened point-wise semantic understanding, while fine-grained scene interpretation further requires distinguishing individual object instances. Instance segmentation extends point-wise recognition toward object-level scene decomposition. Existing methods can be broadly categorized into top-down, bottom-up, and query-based paradigms. Top-down approaches first localize candidate objects and subsequently predict instance masks within individual proposals. Representative methods include GSPN (Yi et al., 2019), which generates object proposals directly from point clouds, and 3D-SIS (Hou et al., 2019), which jointly performs 3D object localization and instance mask prediction. Bottom-up methods instead construct object instances from point-wise predictions. SGPN (Wang et al., 2018) learns pairwise similarities for instance grouping, while PointGroup (Jiang et al., 2020) and HAIS (Chen et al., 2021) organize points into instances through offset-based grouping and hierarchical aggregation, respectively. SoftGroup (Vu et al., 2022) further adopts soft semantic grouping followed by proposal refinement to improve instance formation. Although effective, top-down and bottom-up methods often depend on proposal generation, clustering, or carefully designed grouping procedures. More recently, query-based methods directly represent object instances with learnable queries and decode their masks through interactions with scene features. Mask3D (Schult et al., 2023) employs Transformer decoders to iteratively associate instance queries with multi-scale point features, while QueryFormer (Lu et al., 2023) further investigates query initialization and refinement for robust instance prediction. Building on this paradigm, OneFormer3D (Kolodiazhnyi et al., 2024) unifies semantic, instance, and panoptic segmentation within a shared query-based framework. These advances establish a natural foundation for extending unified query-based segmentation to complex forest scenes.

2.2. 3D Forest Scene Segmentation Forest point cloud segmentation has been extensively studied for recovering tree- and component-level structures from 3D observations. Early approaches mainly relied on geometric priors and hand-crafted procedures, including canopy height models, region growing, clustering, and rulebased tree detection (Naveed et al., 2019; Liu et al., 2021; Wilkes et al., 2023). Although effective in relatively regular forest stands, such methods often degrade under severe occlusion, heterogeneous point densities, understory vegetation, and overlapping crowns. Learning-based approaches Zhan et al.: Preprint submitted to Elsevier

alleviate these limitations by learning forest representations directly from point clouds. ForAINet (Xiang et al., 2024) jointly predicts semantic labels and individual-tree instances for automated forest inventory, while SegmentAnyTree (Wielgosz et al., 2024) adopts a PointGroupinspired architecture to improve transferability across sensing platforms and point densities. TreeLearn (Henrich et al., 2024) further develops a data-driven pipeline for individualtree segmentation from ground-based forest point clouds. Despite their effectiveness, these methods largely retain grouping- or clustering-based instance formation, making accurate tree separation difficult in scenes with complex canopy interactions and large structural variations. Building on these learning-based formulations, recent studies have increasingly incorporated semantic, structural, and interactive cues into individual-tree segmentation. 3DPSNet (Xiu et al., 2025) introduces prompt-driven mask prediction for interactive and automatic tree extraction. Ma et al. (2025) incorporate semantic priors and contrastive learning into instance clustering to improve tree separation under canopy overlap. SATree (Du et al., 2026) jointly models semantic categories, stem locations, and pointwise offsets to facilitate structure-aware instance grouping, whereas ForestSeg3D (Zhang et al., 2026) combines hierarchical semantic supervision with cross-task distillation to improve tree delineation in crowded forest scenes. These studies demonstrate the benefit of introducing semantic and structural cues into instance reasoning. Nevertheless, their primary objective remains individual tree extraction, and instance formation still commonly relies on explicit grouping, clustering, or proposal refinement. The demand for joint scene-component understanding and individual-tree delineation has further motivated unified forest segmentation frameworks. ForestFormer3D (Xiang et al., 2025) introduces an end-to-end query-based formulation that jointly predicts point-wise semantic labels and individual-tree instances, incorporating instance-aware query selection and one-to-many association for forestspecific decoding. SegmentAnyTreeV2 (Wielgosz et al., 2026) combines a PTv3 (Wu et al., 2024) backbone with cross-attention decoding and instance-aware query initialization to improve semantic and instance prediction across diverse forest conditions. ForestMamba (Nguyen et al., 2026) incorporates forest-specific geometric cues at both the representation and query levels, using vertical-priority voxel serialization for Mamba encoding and canopy-height-guided initialization for tree queries. Beyond query-based formulations, CTS (Xie et al., 2026) adopts a panoptic formulation that jointly models semantic tree components and instance identities through semantic grouping and feature clustering. Collectively, these studies highlight the potential of unified semantic-instance reasoning for fine-grained forest scene understanding. However, existing query-based formulations provide limited mechanisms for explicitly modeling boundary uncertainty during instance query construction and optimization, as well as for incorporating forest-specific spatial structure into semantic query representations. Our Page 3 of 16

ForestQuery

Figure 2: Overview of the proposed ForestQuery framework. A shared sparse 3D backbone supports boundary-aware instance query construction and joint query decoding. Boundary cues guide instance query construction and optimization, while SA-SQE injects learnable 3D anchors encoding forest vertical stratification priors into semantic queries. The resulting queries are jointly decoded with shared scene features to predict semantic and individual-tree instance masks.

work addresses these complementary limitations through a boundary-aware mechanism for instance queries and spatially anchored semantic query enhancement.

3. Methodology 3.1. Problem Formulation and Overview Given a forest point cloud 𝐏 ∈ ℝ𝑁×3 containing 𝑁 points, where 𝐩𝑖 = [𝑥𝑖 , 𝑦𝑖 , 𝑧𝑖 ]⊤ ∈ ℝ3 denotes the 𝑖-th point, we formulate unified forest point cloud segmentation as the joint prediction of semantic labels and individual-tree instance identities. For each point 𝐩𝑖 , the model predicts a semantic label 𝑦sem ∈ {1, … , 𝐶} and, for tree points, an 𝑖 ins instance label 𝑦𝑖 ∈ {1, … , 𝐾}, where 𝐶 is the number of semantic categories, and 𝐾 denotes the number of tree instances. Non-tree points are excluded from tree-instance assignment. As illustrated in Fig. 2, ForestQuery adopts a querybased formulation built upon shared sparse 3D features. The input point cloud is first voxelized and encoded by a sparse 3D backbone to obtain a feature tensor 𝐅 ∈ ℝ𝑀×𝑑 , where 𝑀

Zhan et al.: Preprint submitted to Elsevier

denotes the number of active voxels and 𝑑 is the feature dimension. Instance and semantic queries are then constructed through two complementary pathways. For instance queries, instance-discriminative features, coarse semantic cues, and boundary cues derived from 𝐅 are integrated through the proposed instance-semantic-aware (ISA)-boundary-guided query point selection strategy. In parallel, SA-SQE enhances a set of learnable semantic queries with learnable 3D anchors initialized according to forest vertical stratification, thereby introducing explicit spatial priors. ̃ sem denote the resulting instance and Let 𝐐ins and 𝐐 spatially anchored semantic queries, respectively. They are jointly processed by a Transformer decoder, with the shared sparse feature tensor 𝐅 serving as scene memory: ( ) ( ins sem ) ̃ sem , 𝐅 , (1) 𝐐𝐿 , 𝐐𝐿 = Dec(𝐿) 𝐐ins , 𝐐 where Dec(𝐿) denotes an 𝐿-layer query decoder. The decoded queries are subsequently mapped to individual-tree instance masks and semantic masks. During optimization, the predicted boundary uncertainty further modulates instancequery supervision through adaptive loss reweighting. The Page 4 of 16

ForestQuery

following sections detail instance query construction, semantic query enhancement, and unified query decoding and optimization.

3.2. Sparse Feature Encoding and Auxiliary Predictions Given the input point cloud 𝐏, we first voxelize the irregular points into a sparse voxel representation as 𝐕 = (𝐏) ∈ ℝ𝑀×3 , where (⋅) denotes the voxelization operation and 𝑀 is the number of active voxels, typically much smaller than 𝑁. A 3D sparse U-Net (Choy et al., 2019) then encodes 𝐕 to obtain the shared feature representation as 𝐅 = (𝐕) ∈ ℝ𝑀×𝑑 , where (⋅) denotes the sparse feature encoder and 𝑑 = 32 is the feature dimension. The resulting feature tensor 𝐅 serves as the shared scene representation for auxiliary prediction, instance query construction, and subsequent query decoding. To provide complementary cues for query point selection, three auxiliary prediction heads operate on the shared voxel features. The instance-discriminative and coarse semantic heads are defined as 𝐅1 = 𝜙1 (𝐅) ∈ ℝ𝑀×5 ,

𝐅2 = 𝜙2 (𝐅) ∈ ℝ𝑀×2 ,

(2)

where 𝐅1 denotes the instance-discriminative features and 𝐅2 denotes the coarse tree/non-tree log-probability scores. A dedicated boundary head predicts boundary logits from the shared features: 𝐅3 = 𝜙3 (𝐅) ∈ ℝ𝑀×1 .

(3)

These complementary outputs are subsequently exploited by the ISA-boundary-guided query point selection strategy.

3.3. Boundary-Aware Instance Query Construction 3.3.1. Boundary Target Construction and Supervision As illustrated in Fig. 3, we construct boundary targets from two cues. The bird’s-eye-view (BEV) morphological boundary captures the projected crown boundary of each tree, whereas the 3D contact boundary identifies local regions where neighboring tree instances are spatially adjacent. Let  = {𝑣𝑖 }𝑀 denote the set of active voxels in the 𝑖=1 voxelized representation 𝐕, where each voxel 𝑣𝑖 is associated with its center coordinate 𝐜𝑖 = [𝑥𝑖 , 𝑦𝑖 , 𝑧𝑖 ]⊤ . Let 𝑘 ⊆  denote the voxels belonging to the 𝑘-th tree instance. For each instance, we first retain its upper-canopy voxels and project them onto the BEV plane: { } | 𝑘bev = (𝑥𝑖 , 𝑦𝑖 ) | 𝑣𝑖 ∈ 𝑘 , 𝑧𝑖 > 𝑧low . 𝑘 | The lower canopy threshold is defined as ( max ) min 𝑧low − 𝑧min , 𝑘 𝑘 = 𝑧𝑘 + 𝜂 𝑧𝑘

(4)

(5)

where 𝑧min and 𝑧max denote the minimum and maximum 𝑘 𝑘 heights of the 𝑘-th tree instance, respectively, and 𝜂 controls the relative height used to separate the upper-canopy region.

Zhan et al.: Preprint submitted to Elsevier

Figure 3: Instance boundary target construction and prediction. Boundary targets are constructed from two complementary cues: adaptive BEV morphological boundaries characterize projected crown contours, while 3D contact boundaries identify locally adjacent regions belonging to different tree instances. A lightweight boundary head predicts boundary logits from the shared voxel features under the constructed supervision.

To adapt the boundary width to crown size, we estimate the projected crown area, equivalent radius, and erosion thickness as √ ( ) 𝐴𝑘 𝛼𝑟𝑘 | bev | 2 , 𝑡𝑘 = round , (6) 𝐴𝑘 = |𝑘 | 𝑠𝑣 , 𝑟𝑘 = | | 𝜋 𝑠𝑣 where 𝑠𝑣 denotes the BEV grid resolution and 𝛼 controls the relative boundary width. Applying morphological erosion yields 𝑘bev,ero , and the BEV morphological boundary is defined as bev,ero bev bev . 𝑘 = 𝑘 ⧵ 𝑘

(7)

The resulting boundary is mapped back to the voxel domain: { } | bev bev . 𝑘 = 𝑣𝑖 ∈ 𝑘 || (𝑥𝑖 , 𝑦𝑖 ) ∈ 𝑘

(8)

The BEV construction captures crown morphology but does not explicitly model local interactions between adjacent trees. We therefore additionally construct a 3D contact boundary. Let { } | tree = 𝑣𝑖 ∈  | 𝑦bi =1 , | 𝑖

(9)

where 𝑦bi 𝑖 ∈ {0, 1} denotes the ground-truth tree/non-tree label of voxel 𝑣𝑖 . For a tree voxel 𝑣𝑖 , its local 3D neighborhood Page 5 of 16

ForestQuery

is defined as {

}

|  (𝑣𝑖 ) = 𝑣𝑗 ∈  | ‖𝐜𝑗 − 𝐜𝑖 ‖2 ≤ 𝑟c , |

(10)

where 𝑟c denotes the neighborhood radius. A tree voxel is regarded as a contact-boundary voxel if at least one neighboring tree voxel belongs to a different instance: { | contact = 𝑣 ∈ tree | ∃ 𝑣′ ∈  (𝑣) ∩ tree , | (11) } 𝑦ins (𝑣) ≠ 𝑦ins (𝑣′ ) . The complete boundary set is obtained by combining both cues: (𝐾 ) ⋃ bev = 𝑘 ∪ contact , (12) 𝑘=1 gt

and the corresponding binary boundary target is set to 𝑏𝑖 = gt 1 for 𝑣𝑖 ∈  and 𝑏𝑖 = 0 otherwise. To obtain explicit boundary cues, a lightweight boundary head is applied to the shared voxel features. For voxel 𝑑 𝑣𝑖 with feature ( )𝐟𝑖 ∈ ℝ , the corresponding boundary logit is 𝓁𝑖bd = 𝜙3 𝐟𝑖 , where 𝜙3 (⋅) denotes the boundary prediction ( MLP. ) The corresponding boundary probability is 𝑏𝑖 = 𝜎 𝓁𝑖bd , where 𝜎(⋅) denotes the sigmoid function. Boundary supervision is applied only to tree locations, while positive boundary samples are assigned a larger weight to alleviate class imbalance. The boundary loss is defined as bd = −

𝑀 [ 1 ∑ bi gt 𝑦𝑖 𝑤pos 𝑏𝑖 log 𝑏𝑖 𝑀 𝑖=1

]

(13)

gt + (1 − 𝑏𝑖 ) log(1 − 𝑏𝑖 ) ,

where 𝑤pos > 1 denotes the positive-class weight. In implementation, the loss is computed directly from the boundary logits using the numerically stable binary-cross-entropywith-logits formulation. The resulting boundary probabilities provide explicit boundary cues for the subsequent ISAboundary-guided query point selection.

3.3.2. ISA-Boundary-Guided Query Point Selection The proposed ISA-boundary-guided query point selection combines coarse foreground filtering with farthest point sampling (FPS) (Qi et al., 2017b) in a boundary-augmented discriminative feature space. The coarse prediction 𝐅2 is first used to identify foreground tree voxels: 𝑦̂bi 𝑖 = arg max 𝐹2,𝑖,𝑐 . 𝑐

(14)

(15)

For each candidate voxel 𝑣𝑖 , 𝑖 ∈ fg , let 𝐞𝑖 ∈ ℝ5 denote its instance-discriminative feature from 𝐅1 and 𝑏𝑖 its Zhan et al.: Preprint submitted to Elsevier

where 𝛾bd controls the contribution of the boundary cue. FPS is then performed in the augmented feature space: ({ } ) 𝑞 = FPS 𝐳𝑖 ∣ 𝑖 ∈ f g , 𝑁ins , (17) where 𝑁ins denotes the number of instance query points and 𝑞 contains the selected indices. By augmenting the discriminative features with boundary probabilities, the sampling space jointly reflects instance separability and boundary variation. The selected voxels are subsequently used to construct the instance queries 𝐐ins ∈ ℝ𝑁ins ×𝑑 .

3.4. Spatially Anchored Semantic Query Enhancement As illustrated in Fig. 4, we introduce SA-SQE to incorporate explicit spatial priors into semantic query representations. SA-SQE associates each learnable semantic query with a learnable 3D anchor and adaptively integrates the resulting spatial representation with the semantic content. ]⊤ [ sem ∈ ℝ𝑁sem ×𝑑 denote the Let 𝐐sem = 𝐪sem , … , 𝐪 𝑁sem 1 learnable semantic queries, where 𝑁sem is the number of semantic queries and 𝑑 = 32 matches the feature dimension of the shared feature representation 𝐅. Each semantic query is associated with a learnable 3D anchor, forming 𝐀 = [ ]⊤ 𝐚1 , … , 𝐚𝑁sem ∈ ℝ𝑁sem ×3 . The anchors are initialized according to the vertical stratification prior of forest components and are jointly optimized with the network during training. The anchors are first encoded into positional queries through a 3D sinusoidal positional encoding followed by an MLP projection: ( ) 𝐐pos = 𝜙pos PE3D (𝐀) ∈ ℝ𝑁sem ×𝑑 , (18) where 𝜙pos (⋅) denotes the positional projection MLP. To adaptively balance semantic content and spatial information, a channel-wise fusion gate is predicted from the semantic and positional queries: ( ([ ])) 𝐆 = 𝜎 𝜙gate 𝐐sem ; 𝐐pos ∈ ℝ𝑁sem ×𝑑 , (19) where 𝜙gate (⋅) denotes the learnable gating projection and 𝜎(⋅) is the sigmoid function. The two representations are then adaptively fused as 𝐑 = 𝐆 ⊙ 𝐐sem + (𝟏 − 𝐆) ⊙ 𝐐pos ,

(20)

where ⊙ denotes element-wise multiplication. Finally, the spatially anchored semantic queries are obtained as

The corresponding foreground index set is { | } f g = 𝑖 | 𝑦̂bi =1 . | 𝑖

predicted boundary probability. The sampling representation is constructed as [ ] 𝐳𝑖 = 𝐞𝑖 ; 𝛾bd 𝑏𝑖 , 𝑖 ∈ f g , (16)

̃ sem = 𝜙f use (𝐑) ∈ ℝ𝑁sem ×𝑑 , 𝐐

(21)

where 𝜙f use (⋅) denotes the final fusion network. The resulting semantic queries are subsequently jointly decoded with the instance queries 𝐐ins ∈ ℝ𝑁ins ×𝑑 . Page 6 of 16

ForestQuery

Figure 4: Spatial anchoring and adaptive fusion for semantic query enhancement. Learnable 3D anchors initialized with forest vertical stratification priors are encoded into positional queries and adaptively fused with learnable semantic queries through channel-wise gating, yielding spatially anchored semantic queries for subsequent decoding.

3.5. Unified Query Decoding and Optimization The constructed instance queries 𝐐ins ∈ ℝ𝑁ins ×𝑑 and ̃ sem ∈ ℝ𝑁sem ×𝑑 the spatially anchored semantic queries 𝐐 are jointly decoded with the shared feature representation 𝐅. Following the query decoder design of (Kolodiazhnyi et al., 2024), we employ an 𝐿-layer Transformer decoder to iteratively update the two query sets through interactions with the scene features: ( ) ( ins sem ) , 𝐐sem ,𝐅 . (22) = Dec(𝑙) 𝐐ins 𝐐𝑙 , 𝐐𝑙 𝑙−1 𝑙−1 ̃ sem . Here, 𝑙 = 1, … , 𝐿, with 𝐐ins = 𝐐ins and 𝐐sem = 𝐐 0 0 The decoded instance and semantic queries are subsequently mapped to individual-tree instance masks and semantic masks, respectively. During training, we follow the one-to-many query association strategy of ForestFormer3D (Xiang et al., 2025), where each ground-truth tree instance can be matched with multiple positive instance queries. For the 𝑔-th ground-truth tree, let 𝑔+ denote the set of matched positive query indices and 𝑁𝑔 = |𝑔+ | the corresponding number of positive queries. This association provides multiple supervised query hypotheses for each tree instance and enables query-specific reweighting according to boundary probability during subsequent optimization. To account for the different reliability of these positive queries, we further use the predicted boundary probabilities to adaptively reweight their supervision. For a positive query 𝑖 ∈ 𝑔+ , let 𝑏𝑞𝑖 denote the boundary probability at its sampled query location. The corresponding supervision weight is defined as ( ) 𝑁𝑔 exp −𝑏𝑞𝑖 ∕𝜏 𝑤𝑖 = ∑ (23) ( ) , 𝑖 ∈ 𝑔+ , 𝑞 exp −𝑏𝑗 ∕𝜏 𝑗∈𝑔+

where 𝜏 controls the sensitivity to boundary probability. The ∑ factor 𝑁𝑔 normalizes the weights such that 𝑖∈𝑔+ 𝑤𝑖 = 𝑁𝑔 , Zhan et al.: Preprint submitted to Elsevier

while queries with higher boundary probabilities receive smaller relative weights. The boundary-aware weights are subsequently applied to the supervision of the matched positive instance queries. For decoder layer 𝑙, the loss associated with query 𝑖 ∈ 𝑔+ is defined as ( ) ( ) gt gt (𝑙) (𝑙) ̂ ̂ (𝑙) 𝐦 𝓁𝑖,𝑔 +  , 𝐦 =bce 𝐦 , 𝐦 𝑔 𝑔 dice 𝑖 𝑖 (24) ( ( )) (𝑙) (𝑙) ̂ 𝑖 , 𝐦gt + 𝜆score score 𝑠̂𝑖 , IoU 𝐦 , 𝑔 (𝑙) ̂ (𝑙) where 𝐦 𝑖 and 𝑠̂𝑖 denote the predicted mask and confidence gt score of the 𝑖-th instance query, respectively, and 𝐦𝑔 is the ground-truth mask of the 𝑔-th tree instance. The boundaryaware instance loss at decoder layer 𝑙 is then ∑∑ (𝑙) (𝑙) 𝑤𝑖 𝓁𝑖,𝑔 = . (25) ins 𝑔 𝑖∈𝑔+

The semantic loss (𝑙) sem is computed using cross-entropy at decoder layer 𝑙, and supervision is applied to all decoder layers: dec =

𝐿 ( ) ∑ (𝑙) (𝑙) + 𝜆  sem sem , ins

(26)

𝑙=1

where 𝜆sem balances the instance and semantic objectives. In addition to decoder supervision, the auxiliary predictions introduced in Section 3.2 are jointly optimized. Following ForestFormer3D, we adopt disc and bi to supervise the instance-discriminative features 𝐅1 and the coarse tree/nontree prediction 𝐅2 , respectively. The proposed boundary branch is additionally supervised by bd , as defined above. The overall training objective is total = dec + disc + bi + bd .

(27)

Page 7 of 16

ForestQuery Table 1 Summary of the datasets used in this study. FOR-instanceV2 is detailed by subset, with LAUTx and our self-collected dataset included for additional evaluation. Number of Plots Train

Val

Test

Number of Trees

Country

FOR-instanceV2

Scanning Mode (Sensor)

CULS NIBIO RMIT SCION TUWIEN NIBIO2 NIBIO_MLS BlueCat Yuchen

ULS (Riegl VUX-1) ULS (Riegl MiniVUX-1) ULS (Riegl MiniVUX-1) ULS (Riegl MiniVUX-1) ULS (Riegl VUX-1) ULS (Riegl VUX-1) MLS (GeoSLAM ZEB-HORIZON) TLS (Leica P20) ULS (Riegl MiniVUX-1)

1 8 1 2 1 29 4 1 1

1 6 0 1 0 6 1 1 1

1 6 1 2 1 15 1 1 1

37 575 223 135 150 3062 258 6304 281

Czech Republic Norway Australia New Zealand Austria Norway Norway Czech Republic French Guiana

Extra

Dataset

LAUTx Self-collected

MLS (GeoSLAM ZEB-HORIZON) MLS (SHARE SLAM S20) & ULS (RoboSense Airy)

0 0

0 0

6 12

516 137

Austria China

4. Experiments 4.1. Experimental Setup 4.1.1. Datasets We evaluate ForestQuery on three forest point cloud datasets covering diverse sensing platforms, acquisition configurations, and scene conditions: FOR-instanceV2 (Xiang et al., 2025), LAUTx (Tockner et al., 2022), and our selfcollected dataset. Their main characteristics, including scanning modality, data split, number of annotated trees, and geographic region, are summarized in Table 1. FOR-instanceV2 is used for model training, validation, and benchmark evaluation, whereas LAUTx and the self-collected dataset are reserved for assessing generalization to unseen forest scenes and acquisition conditions. FOR-instanceV2 comprises forest point clouds collected across multiple forest areas and sensing configurations, with point-level semantic annotations and individual-tree instance labels. For semantic segmentation, each point is categorized as ground, wood, or leaf, while for instance segmentation, the wood and leaf points belonging to the same tree share a common instance identity and ground points are excluded from instance assignment. Following the official split, the dataset contains 48 training scenes, 17 validation scenes, and 29 test scenes. LAUTx contains six forest scenes acquired using mobile laser scanning (MLS) and serves as an external dataset for cross-dataset generalization evaluation. We further construct a self-collected dataset comprising 12 test scenes acquired using UAV laser scanning (ULS) and MLS. The self-collected scenes are acquired from geographic regions distinct from those represented in FOR-instanceV2 and LAUTx, introducing additional variations in tree species composition. Together, these two held-out datasets differ from the training benchmark in observation viewpoint, spatial coverage, sampling characteristics, point density,

Zhan et al.: Preprint submitted to Elsevier

occlusion patterns, and forest composition, providing complementary settings for evaluating the generalization of ForestQuery to unseen forest scenes, sensing conditions, and tree species.

4.1.2. Evaluation Metrics For individual-tree instance segmentation, predicted and ground-truth tree instances are matched according to their point-set intersection over union (IoU), with a threshold of 0.5. Matched predictions are counted as true positives (TP), while unmatched predictions and ground-truth instances are counted as false positives (FP) and false negatives (FN), respectively. We report precision (P), recall (R), and F1score: P=

TP , TP + FP

R=

TP , TP + FN

F1 =

2PR . (28) P+R

Following the evaluation protocol of ForestFormer3D (Xiang et al., 2025), we also report coverage (Cov) to quantify instance delineation quality: Cov =

𝐺 ( ) 1 ∑ ̂ 𝑘 , 𝐦gt max IoU 𝐦 𝑔 , 𝐺 𝑔=1 𝑘

(29)

where 𝐺 denotes the number of ground-truth tree instances, ̂ 𝑘 and 𝐦gt and 𝐦 𝑔 denote the 𝑘-th predicted instance mask and the 𝑔-th ground-truth instance mask, respectively. For semantic segmentation, we report the class-wise IoU and mean IoU (mIoU): TP𝑐 IoU𝑐 = , TP𝑐 + FP𝑐 + FN𝑐

1 ∑ mIoU = IoU𝑐 , (30) 𝐶 𝑐=1 𝐶

where 𝐶 is the number of semantic categories.

4.1.3. Implementation Details The network takes only 3D point coordinates as input, without using additional color or intensity attributes. During Page 8 of 16

ForestQuery Table 2 Quantitative comparison with state-of-the-art methods on the FOR-instanceV2 test set. Bold and underlined values denote the best and second-best results, respectively. ∗ TreeLearn results exclude the Yuchen region. Individual Tree Seg. (%)

Method ForAINetV2_R16 TreeLearn∗ OneFormer3D ForestFormer3D ForestQuery (ours)

Semantic Seg. (%)

P

R

F1

Cov

Ground

Wood

Leaf

mIoU

88.1 82.0 72.0 92.4 91.8

59.2 36.6 74.3 75.0 77.3

70.8 50.6 73.1 82.8 84.0

72.7 52.2 80.0 81.2 90.0

98.8 – 97.8 96.9 99.0

69.2 – 63.1 67.7 69.6

94.5 – 93.4 94.1 94.3

87.5 – 84.8 86.2 87.6

training, cylindrical regions with a radius of 16 m are randomly cropped from individual forest scenes and voxelized at a resolution of 0.2 m. Data augmentation includes random horizontal flipping, rotation around the vertical axis within [−𝜋, 𝜋], scaling within [0.8, 1.2], and Gaussian translation with a standard deviation of 0.1 m. For boundary target construction, the upper-canopy threshold ratio and relative boundary width are set to 𝜂 = 0.25 and 𝛼 = 0.2, respectively. The BEV grid resolution is set to 𝑠𝑣 = 0.2 m, consistent with the voxel resolution. The 3D contact boundary is constructed within a local neighborhood of radius 𝑟c = 0.5 m, and the positive-class weight in the boundary loss is set to 𝑤pos = 5.0. For ISA-boundary-guided query point selection, the boundary scaling factor is set to 𝛾bd = 2, and the number of instance queries is 𝑁ins = 300. For SA-SQE, we use 𝑁sem = 3 semantic queries. The corresponding learnable anchors are initialized with zero horizontal coordinates and vertical positions of 0, 3, and 8 m for ground, wood, and leaf, respectively, and are jointly optimized during training. The Transformer decoder contains 𝐿 = 6 layers. The BCE and Dice losses for instance segmentation are assigned unit weights, while 𝜆score = 0.5 and 𝜆sem = 0.2. For adaptive boundary-aware reweighting, the temperature parameter is set to 𝜏 = 1.0. The network is optimized using AdamW with an initial learning rate of 2 × 10−4 , a weight decay of 0.05, and gradient clipping with a maximum norm of 10. A PolyLR schedule with a power of 0.9 is adopted for learningrate decay. Training is performed for 450,000 iterations on a single NVIDIA RTX PRO 6000 GPU with a batch size of 2. At inference, each scene is processed using overlapping cylindrical windows with a radius of 16 m and a stride of 4 m. Predictions whose mask centroids lie within 0.5 m of the cylindrical boundary are discarded, and the remaining predictions are transformed to the original point-cloud coordinate system. Following ForestFormer3D, instance masks are ranked by decoder confidence scores and merged using score-based non-maximum suppression (NMS), while semantic predictions are aggregated by majority voting. Finally, instance assignments on ground points are removed, and instances containing fewer than 10 points are filtered out.

Zhan et al.: Preprint submitted to Elsevier

4.2. Comparison with State-of-the-Art Methods We compare ForestQuery with representative state-ofthe-art (SOTA) methods selected according to relevance, recency, and reproducibility, prioritizing approaches with publicly available implementations or sufficiently complete experimental configurations. The baselines include forestspecific methods ForAINetV2 (Xiang et al., 2024) and TreeLearn (Henrich et al., 2024), the general unified segmentation framework OneFormer3D (Kolodiazhnyi et al., 2024), and the recent query-based forest segmentation method ForestFormer3D (Xiang et al., 2025).

4.2.1. Quantitative Results Table 2 reports the quantitative results on the FORinstanceV2 test set. TreeLearn fails to predict individual trees in the Yuchen region due to the extreme sparsity of stem points; its reported results therefore exclude this region. For individual-tree segmentation, ForestQuery achieves the best F1-score of 84.0% and coverage of 90.0%. Compared with ForestFormer3D, the F1-score and coverage are improved by 1.2 and 8.8 percentage points, respectively. Although ForestFormer3D attains a slightly higher precision (92.4% vs. 91.8%), ForestQuery increases recall from 75.0% to 77.3%, yielding a more favorable precision–recall balance. The advantage is more pronounced in coverage, where ForestQuery substantially outperforms all competing methods. This indicates that its predicted instances not only recover a larger fraction of ground-truth trees, but also achieve stronger spatial overlap with the corresponding instances. In contrast, ForAINetV2_R16 and TreeLearn exhibit considerably lower recall and coverage, while OneFormer3D sacrifices precision for higher recall, highlighting the difficulty of simultaneously maintaining detection accuracy and instance completeness. For semantic segmentation, ForestQuery achieves the best ground and wood IoUs of 99.0% and 69.6%, respectively, together with the highest mIoU of 87.6%. Its leaf IoU reaches 94.3%, only 0.2 percentage points below the best result of ForAINetV2_R16. Compared with ForestFormer3D, ForestQuery improves ground, wood, and mIoU by 2.1, 1.9, and 1.4 percentage points, respectively, while maintaining competitive performance on leaf. The class-wise results also show that wood remains substantially more challenging than Page 9 of 16

ForestQuery (ours)

ForestFormer3D

OneFormer3D

Ground Truth

ForestQuery

BlueCat

NIBIO2

NIBIO_MLS

Figure 5: Qualitative comparison of semantic segmentation on representative scenes from the BlueCat, NIBIO2, and NIBIO_MLS subsets of the FOR-instanceV2 test set. Leaf, wood, and ground points are shown in green, brown, and gray, respectively. Red circles highlight confusion between wood and leaf points, while blue circles indicate tree points incorrectly classified as ground.

ground and leaf across all methods, suggesting that semantic ambiguity is concentrated mainly in woody structures rather than being evenly distributed across classes. Overall, ForestQuery achieves a more balanced improvement across both individual-tree and semantic segmentation, without trading a substantial loss in either task.

Zhan et al.: Preprint submitted to Elsevier

4.2.2. Qualitative Results To complement the quantitative evaluation, qualitative comparisons focus on OneFormer3D and ForestFormer3D, the most directly comparable query-based unified segmentation baselines. TreeLearn does not provide multi-class semantic predictions, while ForAINetV2 adopts a non-querybased formulation; both are retained in the quantitative Page 10 of 16

ForestQuery (ours)

ForestFormer3D

OneFormer3D

Ground Truth

ForestQuery

BlueCat

NIBIO2

NIBIO_MLS

Figure 6: Qualitative comparison of individual-tree segmentation on representative scenes from the BlueCat, NIBIO2, and NIBIO_MLS subsets of the FOR-instanceV2 test set. Ground points are shown in gray, while different colors denote individualtree instances. Yellow circles highlight over-segmentation, blue circles indicate undetected trees, and red circles mark points incorrectly assigned from one tree to a neighboring tree.

comparison. Representative scenes are selected from the BlueCat, NIBIO2, and NIBIO_MLS subsets of the FORinstanceV2 test set to cover diverse sensing characteristics and forest structures. Fig. 5 compares the semantic segmentation results. OneFormer3D and ForestFormer3D exhibit noticeable confusion between wood and leaf components, particularly in Zhan et al.: Preprint submitted to Elsevier

structurally complex canopy regions where branches and foliage are spatially intertwined. Tree points are also occasionally misclassified as ground near the lower parts of the scenes, indicating difficulty in maintaining semantic consistency across tree–ground transitions. In comparison, ForestQuery produces cleaner and more spatially coherent semantic predictions across all three subsets, with fewer wood–leaf Page 11 of 16

ForestQuery Table 3 Ablation study on the FOR-instanceV2 test set. Bold and underlined values denote the best and second-best results, respectively. Baseline

BU

✓ ✓ ✓ ✓

SA-SQE

✓ ✓

✓ ✓

Individual Tree Seg. (%) P

R

F1

Cov

Ground

Wood

Leaf

mIoU

92.4 91.7 90.9 91.8

75.0 77.9 76.7 77.3

82.8 84.2 83.2 84.0

81.2 90.1 90.1 90.0

96.9 98.5 99.0 99.0

67.7 66.9 68.0 69.6

94.1 94.0 94.1 94.3

86.2 86.5 87.0 87.6

confusions and tree-to-ground errors. The improvement is particularly visible in regions with dense canopy structure and around tree bases, where class boundaries are less geometrically distinct. These observations are consistent with the higher wood IoU and overall mIoU reported in Table 2, while the already high leaf IoU across methods suggests that the remaining semantic difficulty is concentrated mainly in woody and transition regions. Fig. 6 presents the corresponding individual-tree segmentation results. Yellow, blue, and red circles denote over-segmentation, undetected trees, and incorrect point assignment across neighboring instances, respectively. OneFormer3D exhibits all three types of errors, reflecting difficulties in simultaneously preserving complete tree instances and separating neighboring trees. ForestFormer3D reduces several of these errors, but inaccurate separation and crossinstance assignment remain visible in regions with dense tree distributions and overlapping crowns. By comparison, ForestQuery produces more coherent tree instances, with fewer missed trees, fragmented instances, and erroneous assignments between adjacent trees. The improvement is consistently observed across the three subsets rather than being confined to a particular scene configuration. This behavior agrees with the quantitative results, where ForestQuery achieves both the highest F1-score and a substantially higher coverage, indicating improved instance recognition together with more complete tree delineation.

4.3. Ablation Studies We conduct component-wise ablations on the FORinstanceV2 test set to examine the contributions of boundary uncertainty (BU) and SA-SQE. ForestFormer3D serves as the common baseline, providing a query-based unified segmentation framework for controlled evaluation of the proposed designs. BU and SA-SQE are introduced individually and jointly, allowing their task-specific contributions and joint effect to be assessed under the same experimental setting. Table 3 reports both individual-tree and semantic segmentation metrics. Effect of BU. Introducing BU leads to clear improvements in individual-tree segmentation. Compared with the baseline, recall increases from 75.0% to 77.9%, F1-score from 82.8% to 84.2%, and coverage from 81.2% to 90.1%, while precision decreases only slightly from 92.4% to 91.7%. The substantially larger gain in coverage indicates that BU Zhan et al.: Preprint submitted to Elsevier

Semantic Seg. (%)

is particularly effective in improving the spatial completeness of predicted tree instances, in addition to increasing instance-level recognition performance. Its effect on semantic segmentation is comparatively limited: mIoU increases from 86.2% to 86.5%, with an improvement in ground IoU but minor decreases in wood and leaf IoUs. This performance pattern is consistent with the design objective of BU, which focuses on ambiguous instance boundaries and individual-tree delineation. Effect of SA-SQE. SA-SQE exhibits a different performance profile, with its primary improvement appearing in semantic segmentation. It improves ground IoU from 96.9% to 99.0% and wood IoU from 67.7% to 68.0%, while preserving the leaf IoU at 94.1%, resulting in an mIoU increase from 86.2% to 87.0%. The semantic gain is driven mainly by the improved ground prediction, together with a moderate improvement in the more challenging wood class, without degrading leaf segmentation. SA-SQE also benefits individualtree segmentation: recall and F1-score increase to 76.7% and 83.2%, respectively, while coverage rises markedly to 90.1%. This cross-task improvement suggests that spatially structured semantic queries can provide useful information to the shared unified segmentation process beyond semantic prediction alone. Joint Effect of BU and SA-SQE. Combining BU and SA-SQE provides the strongest overall balance between individual-tree and semantic segmentation. Relative to the baseline, the full model improves recall, F1-score, and coverage by 2.3, 1.2, and 8.8 percentage points, respectively, while improving ground, wood, and leaf IoUs by 2.1, 1.9, and 0.2 percentage points and mIoU by 1.4 percentage points. Although BU alone achieves slightly higher F1-score and coverage (84.2% and 90.1% versus 84.0% and 90.0%), incorporating SA-SQE largely preserves these instance-level gains while increasing wood IoU from 66.9% to 69.6% and mIoU from 86.5% to 87.6%. Conversely, adding BU to SA-SQE further improves precision, recall, and F1-score by 0.9, 0.6, and 0.8 percentage points, respectively, while maintaining nearly identical coverage and further increasing wood IoU and mIoU by 1.6 and 0.6 percentage points. These two directions of improvement show that BU and SA-SQE address complementary aspects of the unified task: the former primarily strengthens individual-tree delineation, whereas the latter enhances semantic discrimination. Their

Page 12 of 16

ForestQuery Table 4 Quantitative comparison of generalization performance on the LAUTx and self-collected datasets. Bold and underlined values denote the best and second-best results, respectively. Dataset

Method

Individual Tree Seg. (%)

Semantic Seg. (%)

P

R

F1

Cov

Ground

Tree

mIoU

LAUTx

ForAINetV2_R16 TreeLearn OneFormer3D ForestFormer3D ForestQuery (ours)

93.7 96.8 58.1 95.7 92.5

77.7 87.9 80.2 85.9 88.6

85.0 92.1 67.4 90.5 90.5

74.7 85.2 74.6 79.1 86.8

95.9 92.6 94.6 94.1 95.1

96.7 95.9 96.9 96.7 96.7

96.3 94.3 95.8 95.4 95.9

Self-collected

ForAINetV2_R16 TreeLearn OneFormer3D ForestFormer3D ForestQuery (ours)

78.3 38.4 69.3 90.2 97.7

86.9 33.3 77.4 87.6 93.4

82.4 35.7 73.1 88.9 95.5

84.3 47.2 74.2 86.8 90.7

93.5 70.3 57.5 97.3 98.2

99.0 92.8 93.8 99.5 99.7

96.2 81.6 75.6 98.4 99.0

combination therefore yields a more balanced improvement across both instance and semantic segmentation.

4.4. Generalization Evaluation We further evaluate ForestQuery on LAUTx and the selfcollected dataset, neither of which is used during training. Compared with FOR-instanceV2, both datasets introduce substantial domain shifts in sensing conditions and forest structure, while the self-collected scenes further differ in geographic region and tree species composition. Both datasets provide individual-tree annotations and binary semantic labels for ground and tree, enabling evaluation of individualtree segmentation and binary semantic generalization under unseen forest environments. For semantic evaluation, the predicted wood and leaf classes are merged into the tree class, yielding two semantic categories (𝐶 = 2).

4.4.1. Quantitative Results Table 4 reports the quantitative results on the two heldout generalization datasets. On LAUTx, ForestQuery achieves the highest recall of 88.6% and coverage of 86.8%, outperforming the second-best results by 0.7 and 1.6 percentage points, respectively. Its F1-score reaches 90.5%, tying with ForestFormer3D for the second-best result, although TreeLearn attains higher precision and the highest F1-score. This performance profile indicates that ForestQuery favors more complete tree recovery and delineation under the domain shift, as reflected by its leading recall and coverage, while maintaining competitive overall instance accuracy. For binary semantic segmentation, ForestQuery achieves ground and tree IoUs of 95.1% and 96.7%, respectively, yielding an mIoU of 95.9%. This is the second-best semantic result on LAUTx and is only 0.4 percentage points below the highest mIoU of 96.3%, showing that the instance-level gains are obtained without a substantial degradation in tree–ground discrimination. More pronounced gains are observed on the self-collected dataset, which introduces additional variations in acquisition Zhan et al.: Preprint submitted to Elsevier

conditions, geographic regions, and tree species composition. ForestQuery achieves the best results across all reported metrics, reaching 97.7% precision, 93.4% recall, 95.5% F1-score, and 90.7% coverage for individual-tree segmentation. Compared with ForestFormer3D, the strongest competing query-based baseline, these results correspond to improvements of 7.5, 5.8, 6.6, and 3.9 percentage points, respectively. The simultaneous gains in precision and recall are particularly notable, suggesting that the improvement is not obtained by simply favoring either conservative or aggressive instance prediction. Instead, ForestQuery maintains both accurate tree recognition and more complete instance delineation under the unseen acquisition and forest conditions. For binary semantic segmentation, it further achieves 98.2% ground IoU, 99.7% tree IoU, and 99.0% mIoU, improving over ForestFormer3D by 0.9, 0.2, and 0.6 percentage points, respectively. Taken together, the results across LAUTx and the self-collected dataset show that ForestQuery preserves strong semantic discrimination while exhibiting particularly consistent cross-dataset generalization for individual-tree segmentation.

4.4.2. Qualitative Results Qualitative evaluation focuses on individual-tree segmentation to maintain a consistent visualization protocol across the two held-out datasets. Since both LAUTx and the self-collected dataset provide only binary ground/tree semantic annotations, we omit separate semantic visualizations and present instance segmentation results instead. Because tree points are assigned to individual instances while ground remains unassigned, these results jointly reflect individual-tree delineation and tree–ground separation. Fig. 7 compares the predictions across three representative held-out scenes with different tree distributions

Page 13 of 16

ForestQuery (ours)

ForestFormer3D

OneFormer3D

Ground Truth

ForestQuery

Scene01

Scene02

Scene03

Figure 7: Qualitative comparison of individual-tree segmentation on the generalization datasets. Ground points are shown in gray, while different colors denote individual-tree instances. Yellow circles highlight over-segmentation, blue circles indicate undetected trees, red circles mark points incorrectly assigned from one tree to a neighboring tree, and black circles indicate ground points incorrectly assigned to tree instances.

and structural configurations. OneFormer3D exhibits several recurring failure modes, including missed trees, oversegmentation, cross-instance assignment, and ground leakage into predicted tree instances. These errors are particularly evident around slender trees, closely spaced individuals, and ambiguous tree–ground transition regions. ForestFormer3D produces more coherent instances and reduces many of these errors, but residual failures remain near tree bases and between adjacent trees, where local geometric ambiguity makes instance separation difficult. Zhan et al.: Preprint submitted to Elsevier

In comparison, ForestQuery produces more complete and spatially coherent tree instances across the three scenes, with fewer missed trees, fragmented predictions, crossinstance assignments, and ground leakage. The remaining errors are mainly confined to locally complex regions involving closely interacting neighboring trees. Overall, the qualitative results indicate improved robustness to unseen scene structures while preserving both instance completeness and tree–ground consistency. This observation is consistent with the quantitative results in Table 4, where ForestQuery shows Page 14 of 16

ForestQuery

particularly strong recall and coverage across the held-out evaluation settings.

5. Conclusion and Future Work In this work, we presented ForestQuery, a unified querybased framework for semantic and individual-tree segmentation of forest point clouds. ForestQuery introduces a boundary-aware mechanism for instance query learning and SA-SQE for incorporating forest-specific spatial priors into semantic queries. Experiments on FOR-instanceV2 demonstrate improved individual-tree delineation and semantic segmentation over representative state-of-the-art methods, while the ablation studies verify the complementary effects of BU and SA-SQE. Evaluations on LAUTx and the selfcollected dataset further show promising generalization to unseen sensing conditions and forest environments. Future work will explore both methodological and datacentric extensions. More adaptive spatial priors and stronger cross-domain modeling may further improve robustness to diverse forest structures and sensing conditions. Meanwhile, larger and more diverse forest point cloud datasets could support large-scale pretraining toward generalizable forest point cloud foundation models, enabling transferable representations across species, regions, sensing platforms, and downstream forest understanding tasks.

References Chen, S., Fang, J., Zhang, Q., Liu, W., Wang, X., 2021. Hierarchical aggregation for 3d instance segmentation, in: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 15467–15476. Choy, C., Gwak, J., Savarese, S., 2019. 4d spatio-temporal convnets: Minkowski convolutional neural networks, in: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 3075–3084. Du, S., Stoter, J., Kooij, J.F., Nan, L., 2026. Satree: Structure-aware tree instance segmentation from 3d lidar point clouds. Urban Forestry & Urban Greening , 129414. Henrich, J., van Delden, J., Seidel, D., Kneib, T., Ecker, A.S., 2024. Treelearn: A deep learning method for segmenting individual trees from ground-based lidar forest point clouds. Ecological Informatics 84, 102888. Hou, J., Dai, A., Nießner, M., 2019. 3d-sis: 3d semantic instance segmentation of rgb-d scans, in: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), IEEE. pp. 4416–4425. Hu, Q., Yang, B., Xie, L., Rosa, S., Guo, Y., Wang, Z., Trigoni, N., Markham, A., 2020. Randla-net: Efficient semantic segmentation of large-scale point clouds, in: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), IEEE. pp. 11105–11114. Jiang, L., Zhao, H., Shi, S., Liu, S., Fu, C.W., Jia, J., 2020. Pointgroup: Dual-set point grouping for 3d instance segmentation, in: Proceedings of the IEEE/CVF conference on computer vision and Pattern recognition, pp. 4867–4876. Kolodiazhnyi, M., Vorontsova, A., Konushin, A., Rukhovich, D., 2024. Oneformer3d: One transformer for unified point cloud segmentation, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 20943–20953. Lai, X., Liu, J., Jiang, L., Wang, L., Zhao, H., Liu, S., Qi, X., Jia, J., 2022. Stratified transformer for 3d point cloud segmentation, in: 2022 IEEE/CVF conference on computer vision and pattern recognition (CVPR), IEEE. pp. 8490–8499.

Zhan et al.: Preprint submitted to Elsevier

Liu, H., Dong, P., Wu, C., Wang, P., Fang, M., 2021. Individual tree identification using a new cluster-based approach with discrete-return airborne lidar data. Remote Sensing of Environment 258, 112382. Lu, J., Deng, J., Wang, C., He, J., Zhang, T., 2023. Query refinement transformer for 3d instance segmentation, in: 2023 IEEE/CVF International Conference on Computer Vision (ICCV), IEEE. pp. 18470–18480. Ma, J., Han, T., Wang, C., Zhang, X., Zhang, X., Zhang, W., Chen, Y., 2025. Individual tree segmentation via contrastive learning and semantic priors in point clouds. Urban Forestry & Urban Greening , 129018. Naveed, F., Hu, B., Wang, J., Hall, G.B., 2019. Individual tree crown delineation using multispectral lidar data. Sensors 19, 5421. Nguyen, T.T., Vu, T.A., Le, D.V., Kawanishi, Y., Komamizu, T., Ide, I., Kattenborn, T., 2026. Forestmamba: Sparse mamba with geometryguided queries for 3d forest point cloud segmentation. arXiv preprint arXiv:2606.01549 . Qi, C.R., Su, H., Mo, K., Guibas, L.J., 2017a. Pointnet: Deep learning on point sets for 3d classification and segmentation, in: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 652– 660. Qi, C.R., Yi, L., Su, H., Guibas, L.J., 2017b. Pointnet++: Deep hierarchical feature learning on point sets in a metric space. Advances in neural information processing systems 30. Schult, J., Engelmann, F., Hermans, A., Litany, O., Tang, S., Leibe, B., 2023. Mask3d: Mask transformer for 3d semantic instance segmentation, in: 2023 IEEE International Conference on Robotics and Automation (ICRA), IEEE. pp. 8216–8223. Tan, Y., Yang, X., Zhang, J., Xu, X., Cao, Y., Wang, D., Shigeno, M., 2026. Tlnet: A deep learning framework for tree detection in forest point clouds using multi-layered forest structure. ISPRS Journal of Photogrammetry and Remote Sensing 234, 227–241. Tchapmi, L., Choy, C., Armeni, I., Gwak, J., Savarese, S., 2017. Segcloud: Semantic segmentation of 3d point clouds, in: 2017 international conference on 3D vision (3DV), IEEE. pp. 537–547. Thomas, H., Qi, C.R., Deschaud, J.E., Marcotegui, B., Goulette, F., Guibas, L.J., 2019. Kpconv: Flexible and deformable convolution for point clouds, in: Proceedings of the IEEE/CVF international conference on computer vision, pp. 6411–6420. Tockner, A., Gollob, C., Ritter, T., Nothdurft, A., 2022. Lautxindividual tree point clouds from austrian forest inventory plots. URL: https://zenodo. org/record/6560112#. Y6IPZezMJQK, doi 10, 5281. Vu, T., Kim, K., Luu, T.M., Nguyen, T., Yoo, C.D., 2022. Softgroup for 3d instance segmentation on point clouds, in: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 2708–2717. Wang, W., Yu, R., Huang, Q., Neumann, U., 2018. Sgpn: Similarity group proposal network for 3d point cloud instance segmentation, in: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, IEEE. pp. 2569–2578. Waser, L.T., Rüetschi, M., Psomas, A., Small, D., Rehush, N., 2021. Mapping dominant leaf type based on combined sentinel-1/-2 data– challenges for mountainous countries. ISPRS Journal of Photogrammetry and Remote Sensing 180, 209–226. Wielgosz, M., Puliti, S., Astrup, R., 2026. Segmentanytreev2: Scaling transformer-based tree instance segmentation across sensors, platforms, and forests. arXiv preprint arXiv:2606.08206 . Wielgosz, M., Puliti, S., Xiang, B., Schindler, K., Astrup, R., 2024. Segmentanytree: A sensor and platform agnostic deep learning model for tree segmentation using laser scanning data. Remote Sensing of Environment 313, 114367. Wilkes, P., Disney, M., Armston, J., Bartholomeus, H., Bentley, L., Brede, B., Burt, A., Calders, K., Chavana-Bryant, C., Clewley, D., et al., 2023. Tls2trees: A scalable tree segmentation pipeline for tls data. Methods in Ecology and Evolution 14, 3083–3099. Wu, X., Jiang, L., Wang, P.S., Liu, Z., Liu, X., Qiao, Y., Ouyang, W., He, T., Zhao, H., 2024. Point transformer v3: Simpler faster stronger, in: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 4840–4851.

Page 15 of 16

ForestQuery Xiang, B., Wielgosz, M., Kontogianni, T., Peters, T., Puliti, S., Astrup, R., Schindler, K., 2024. Automated forest inventory: Analysis of highdensity airborne lidar point clouds with 3d deep learning. Remote Sensing of Environment 305, 114078. Xiang, B., Wielgosz, M., Puliti, S., Král, K., Krůček, M., Missarov, A., Astrup, R., 2025. Forestformer3d: A unified framework for end-toend segmentation of forest lidar 3d point clouds, in: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 24717– 24727. Xie, Y., Chen, Z., Xu, S., 2026. Soft semantic grouping and feature clustering for panoptic segmentation of tree crowns and trunks in lidar point clouds. Computers and Electronics in Agriculture 251, 111965. Xiu, T., Qi, H., Xu, J., Liang, X., 2025. Individual tree extraction through 3d promptable segmentation networks. Methods in Ecology and Evolution 16, 1749–1762. Yi, L., Zhao, W., Wang, H., Sung, M., Guibas, L.J., 2019. Gspn: Generative shape proposal network for 3d instance segmentation in point cloud, in: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), IEEE. pp. 3942–3951. Zhan, Z., Ming, Y., Li, S., Yuan, J., 2025. Agrilira4d: A multi-sensor uav dataset for robust slam in challenging agricultural fields. arXiv preprint arXiv:2512.01753 . Zhan, Z., Tao, L., Li, S., Fang, C., Yang, X., Li, L., Fan, R., Ming, Y., 2026. Terrain perception for agricultural uavs in complex farmland via rotating mmwave radar. arXiv preprint arXiv:2605.01340 . Zhang, K., Zhang, W., Jing, W., Li, L., Li, C., 2026. Forestseg3d: A semantically guided framework for individual tree segmentation from lidar point clouds. Plant Phenomics , 100261. Zhao, H., Jiang, L., Jia, J., Torr, P.H., Koltun, V., 2021. Point transformer, in: Proceedings of the IEEE/CVF international conference on computer vision, pp. 16259–16268.

Zhan et al.: Preprint submitted to Elsevier

Page 16 of 16

Related documents

• ForestQuery: Boundary-Aware and Spatially Anchored Query Learning for Unified Forest Point Cloud Segmentation#1188081
• SelectAnyTree: A Promptable Instance Segmentation Model for 3D Forest LiDAR Point Clouds#677655
• SelectAnyTree: A Promptable Instance Segmentation Model for 3D Forest LiDAR Point Clouds#674454
• Training weights to classify tree species from single tree LiDAR point clouds - the DetailView AI-model.#206021
• Training weights to classify tree species from single tree LiDAR point clouds - the DetailView AI-model.#206020