SemRegionNet: Region ensemble 3D semantic instance segmentation network with semantic spatial aware discriminative loss.

[Display omitted] The figure below is an overview of the proposed framework in this paper. Point clouds with xyz coordinates and RGB attributes are fed into the network, and the output includes semantic labels and instance labels. The proposed region ensemble structure includes Random Sampling-based Set Abstraction (RS-SA), Adaptive Regional Feature Complementary (ARFC), and Affinity-based Regional Relation Reasoning (AR3) modules. RS-SA denotes the random sampling-based set abstraction. The down-sampling in RS-SA sequentially samples the input point clouds. The ARFC module aims to adaptively complement low-level features to account for the information loss caused by the random sampling and inherent non-uniformity. The AR3 module focuses on reasoning about relationships among high-level regional features. The semantic and spatial aware discriminative loss is proposed to supervise the instance embedding learning leveraging the semantic and spatial knowledge. • Random sampling is introduced to improve set abstraction efficiency. • A region ensemble structure is developed to enhance point cloud comprehension. • A semantic spatial awareness is proposed to mitigate instance confusion problem. • Experimental results demonstrate that the proposed method achieves promising boosts. The semantic instance segmentation task on 3D data has made great progress. However, for unstructured 3D point cloud data, the mining of regional knowledge and explicit assistance of semantic for the instance segmentation task are still rarely explored. In this paper, we propose a region ensemble structure including random sampling-based set abstraction (RS-SA), an adaptive regional feature complementary (ARFC) module, and an affinity-based regional relational reasoning (AR3) module in feature encoding to enhance the point cloud comprehension. Random sampling is introduced into the set abstraction of feature encoding to improve computational efficiency. The ARFC module aims to complement low-level features to adaptively compensate for the information loss caused by random sampling and inherent non-uniformity of point clouds, and the AR3 module emphasizes mining the potential reasoning relationships among high-level features based on affinity. Furthermore, a novel semantic spatial aware discriminative loss is proposed to improve the discrimination of instance embedding. The proposed region ensemble structure and semantic spatial awareness for discriminative loss are demonstrated promising boosts on the 3D point cloud semantic instance segmentation task, and the framework achieves state-of-the-art performance on the S3DIS, ScanNet-v2, and ShapeNet datasets. [ABSTRACT FROM AUTHOR]