Zhu Pengfei ; Yao Xinjie ; Wang Yu ; Hui Binyuan ; Du Dawei ; Hu Qinghua

Abstract

Multi-view subspace clustering aims to discover the inherent structure ofdata by fusing multiple views of complementary information. Most existingmethods first extract multiple types of handcrafted features and then learn ajoint affinity matrix for clustering. The disadvantage of this approach lies intwo aspects: 1) multi-view relations are not embedded into feature learning,and 2) the end-to-end learning manner of deep learning is not suitable formulti-view clustering. Even when deep features have been extracted, it is anontrivial problem to choose a proper backbone for clustering on differentdatasets. To address these issues, we propose the Multi-view Deep SubspaceClustering Networks (MvDSCN), which learns a multi-view self-representationmatrix in an end-to-end manner. The MvDSCN consists of two sub-networks, \ie, adiversity network (Dnet) and a universality network (Unet). A latent space isbuilt using deep convolutional autoencoders, and a self-representation matrixis learned in the latent space using a fully connected layer. Dnet learnsview-specific self-representation matrices, whereas Unet learns a commonself-representation matrix for all views. To exploit the complementarity ofmulti-view representations, the Hilbert--Schmidt independence criterion (HSIC)is introduced as a diversity regularizer that captures the nonlinear,high-order inter-view relations. Because different views share the same labelspace, the self-representation matrices of each view are aligned to the commonone by universality regularization. The MvDSCN also unifies multiple backbonesto boost clustering performance and avoid the need for model selection.Experiments demonstrate the superiority of the MvDSCN.