Yan Xu ; Gao Jiantao ; Li Jie ; Zhang Ruimao ; Li Zhen ; Huang Rui ; Cui Shuguang

Abstract

LiDAR point cloud analysis is a core task for 3D computer vision, especiallyfor autonomous driving. However, due to the severe sparsity and noiseinterference in the single sweep LiDAR point cloud, the accurate semanticsegmentation is non-trivial to achieve. In this paper, we propose a novelsparse LiDAR point cloud semantic segmentation framework assisted by learnedcontextual shape priors. In practice, an initial semantic segmentation (SS) ofa single sweep point cloud can be achieved by any appealing network and thenflows into the semantic scene completion (SSC) module as the input. By mergingmultiple frames in the LiDAR sequence as supervision, the optimized SSC modulehas learned the contextual shape priors from sequential LiDAR data, completingthe sparse single sweep point cloud to the dense one. Thus, it inherentlyimproves SS optimization through fully end-to-end training. Besides, aPoint-Voxel Interaction (PVI) module is proposed to further enhance theknowledge fusion between SS and SSC tasks, i.e., promoting the interaction ofincomplete local geometry of point cloud and complete voxel-wise globalstructure. Furthermore, the auxiliary SSC and PVI modules can be discardedduring inference without extra burden for SS. Extensive experiments confirmthat our JS3C-Net achieves superior performance on both SemanticKITTI andSemanticPOSS benchmarks, i.e., 4% and 3% improvement correspondingly.