Wang Shihao ; Liu Yingfei ; Wang Tiancai ; Li Ying ; Zhang Xiangyu

Abstract

In this paper, we propose a long-sequence modeling framework, namedStreamPETR, for multi-view 3D object detection. Built upon the sparse querydesign in the PETR series, we systematically develop an object-centric temporalmechanism. The model is performed in an online manner and the long-termhistorical information is propagated through object queries frame by frame.Besides, we introduce a motion-aware layer normalization to model the movementof the objects. StreamPETR achieves significant performance improvements onlywith negligible computation cost, compared to the single-frame baseline. On thestandard nuScenes benchmark, it is the first online multi-view method thatachieves comparable performance (67.6% NDS & 65.3% AMOTA) with lidar-basedmethods. The lightweight version realizes 45.0% mAP and 31.7 FPS, outperformingthe state-of-the-art method (SOLOFusion) by 2.3% mAP and 1.8x faster FPS. Codehas been available at https://github.com/exiawsh/StreamPETR.git.