Lahav Alon ; Tal Ayellet

Abstract

Most attempts to represent 3D shapes for deep learning have focused onvolumetric grids, multi-view images and point clouds. In this paper we look atthe most popular representation of 3D shapes in computer graphics - atriangular mesh - and ask how it can be utilized within deep learning. The fewattempts to answer this question propose to adapt convolutions & pooling tosuit Convolutional Neural Networks (CNNs). This paper proposes a very differentapproach, termed MeshWalker, to learn the shape directly from a given mesh. Thekey idea is to represent the mesh by random walks along the surface, which"explore" the mesh's geometry and topology. Each walk is organized as a list ofvertices, which in some manner imposes regularity on the mesh. The walk is fedinto a Recurrent Neural Network (RNN) that "remembers" the history of the walk.We show that our approach achieves state-of-the-art results for two fundamentalshape analysis tasks: shape classification and semantic segmentation.Furthermore, even a very small number of examples suffices for learning. Thisis highly important, since large datasets of meshes are difficult to acquire.