Zeng Yiming ; Qian Yue ; Zhu Zhiyu ; Hou Junhui ; Yuan Hui ; He Ying

Abstract

Motivated by the intuition that one can transform two aligned point clouds toeach other more easily and meaningfully than a misaligned pair, we proposeCorrNet3D -- the first unsupervised and end-to-end deep learning-basedframework -- to drive the learning of dense correspondence between 3D shapes bymeans of deformation-like reconstruction to overcome the need for annotateddata. Specifically, CorrNet3D consists of a deep feature embedding module andtwo novel modules called correspondence indicator and symmetric deformer.Feeding a pair of raw point clouds, our model first learns the pointwisefeatures and passes them into the indicator to generate a learnablecorrespondence matrix used to permute the input pair. The symmetric deformer,with an additional regularized loss, transforms the two permuted point cloudsto each other to drive the unsupervised learning of the correspondence. Theextensive experiments on both synthetic and real-world datasets of rigid andnon-rigid 3D shapes show our CorrNet3D outperforms state-of-the-art methods toa large extent, including those taking meshes as input. CorrNet3D is a flexibleframework in that it can be easily adapted to supervised learning if annotateddata are available. The source code and pre-trained model will be available athttps://github.com/ZENGYIMING-EAMON/CorrNet3D.git.