Wang Weiyue Yu Ronald Huang Qiangui Neumann Ulrich

Abstract

We introduce Similarity Group Proposal Network (SGPN), a simple and intuitivedeep learning framework for 3D object instance segmentation on point clouds.SGPN uses a single network to predict point grouping proposals and acorresponding semantic class for each proposal, from which we can directlyextract instance segmentation results. Important to the effectiveness of SGPNis its novel representation of 3D instance segmentation results in the form ofa similarity matrix that indicates the similarity between each pair of pointsin embedded feature space, thus producing an accurate grouping proposal foreach point. To the best of our knowledge, SGPN is the first framework to learn3D instance-aware semantic segmentation on point clouds. Experimental resultson various 3D scenes show the effectiveness of our method on 3D instancesegmentation, and we also evaluate the capability of SGPN to improve 3D objectdetection and semantic segmentation results. We also demonstrate itsflexibility by seamlessly incorporating 2D CNN features into the framework toboost performance.