Sarlin Paul-Edouard ; DeTone Daniel ; Malisiewicz Tomasz ; Rabinovich Andrew

Abstract

This paper introduces SuperGlue, a neural network that matches two sets oflocal features by jointly finding correspondences and rejecting non-matchablepoints. Assignments are estimated by solving a differentiable optimal transportproblem, whose costs are predicted by a graph neural network. We introduce aflexible context aggregation mechanism based on attention, enabling SuperGlueto reason about the underlying 3D scene and feature assignments jointly.Compared to traditional, hand-designed heuristics, our technique learns priorsover geometric transformations and regularities of the 3D world throughend-to-end training from image pairs. SuperGlue outperforms other learnedapproaches and achieves state-of-the-art results on the task of pose estimationin challenging real-world indoor and outdoor environments. The proposed methodperforms matching in real-time on a modern GPU and can be readily integratedinto modern SfM or SLAM systems. The code and trained weights are publiclyavailable at https://github.com/magicleap/SuperGluePretrainedNetwork.