Sun Peize ; Cao Jinkun ; Jiang Yi ; Zhang Rufeng ; Xie Enze ; Yuan Zehuan ; Wang Changhu ; Luo Ping

Abstract

In this work, we propose TransTrack, a simple but efficient scheme to solvethe multiple object tracking problems. TransTrack leverages the transformerarchitecture, which is an attention-based query-key mechanism. It appliesobject features from the previous frame as a query of the current frame andintroduces a set of learned object queries to enable detecting new-comingobjects. It builds up a novel joint-detection-and-tracking paradigm byaccomplishing object detection and object association in a single shot,simplifying complicated multi-step settings in tracking-by-detection methods.On MOT17 and MOT20 benchmark, TransTrack achieves 74.5\% and 64.5\% MOTA,respectively, competitive to the state-of-the-art methods. We expect TransTrackto provide a novel perspective for multiple object tracking. The code isavailable at: \url{https://github.com/PeizeSun/TransTrack}.