Liu Hanwei ; An Rudong ; Zhang Zhimeng ; Ma Bowen ; Zhang Wei ; Song Yan ; Hu Yujing ; Chen Wei ; Ding Yu

Abstract

Facial Expression Analysis remains a challenging task due to unexpectedtask-irrelevant noise, such as identity, head pose, and background. To addressthis issue, this paper proposes a novel framework, called Norface, that isunified for both Action Unit (AU) analysis and Facial Emotion Recognition (FER)tasks. Norface consists of a normalization network and a classificationnetwork. First, the carefully designed normalization network struggles todirectly remove the above task-irrelevant noise, by maintaining facialexpression consistency but normalizing all original images to a common identitywith consistent pose, and background. Then, these additional normalized imagesare fed into the classification network. Due to consistent identity and otherfactors (e.g. head pose, background, etc.), the normalized images enable theclassification network to extract useful expression information moreeffectively. Additionally, the classification network incorporates a Mixture ofExperts to refine the latent representation, including handling the input offacial representations and the output of multiple (AU or emotion) labels.Extensive experiments validate the carefully designed framework with theinsight of identity normalization. The proposed method outperforms existingSOTA methods in multiple facial expression analysis tasks, including AUdetection, AU intensity estimation, and FER tasks, as well as theircross-dataset tasks. For the normalized datasets and code please visit{https://norface-fea.github.io/}.