Zhang Boyu ; Chen Jiayuan ; Xu Yinfei ; Zhang Hui ; Yang Xu ; Geng Xin

Abstract

The action quality assessment (AQA) of videos is a challenging vision tasksince the relation between videos and action scores is difficult to model.Thus, AQA has been widely studied in the literature. Traditionally, AQA istreated as a regression problem to learn the underlying mappings between videosand action scores. But previous methods ignored data uncertainty in AQAdataset. To address aleatoric uncertainty, we further develop a plug-and-playmodule Distribution Auto-Encoder (DAE). Specifically, it encodes videos intodistributions and uses the reparameterization trick in variationalauto-encoders (VAE) to sample scores, which establishes a more accurate mappingbetween videos and scores. Meanwhile, a likelihood loss is used to learn theuncertainty parameters. We plug our DAE approach into MUSDL and CoRe.Experimental results on public datasets demonstrate that our method achievesstate-of-the-art on AQA-7, MTL-AQA, and JIGSAWS datasets. Our code is availableat https://github.com/InfoX-SEU/DAE-AQA.