Kar Purbayan ; Chudasama Vishal ; Onoe Naoyuki ; Wasnik Pankaj ; Balasubramanian Vineeth

Abstract

Deep learning methods have led to significant improvements in the performanceon the facial landmark detection (FLD) task. However, detecting landmarks inchallenging settings, such as head pose changes, exaggerated expressions, oruneven illumination, continue to remain a challenge due to high variability andinsufficient samples. This inadequacy can be attributed to the model'sinability to effectively acquire appropriate facial structure information fromthe input images. To address this, we propose a novel image augmentationtechnique specifically designed for the FLD task to enhance the model'sunderstanding of facial structures. To effectively utilize the newly proposedaugmentation technique, we employ a Siamese architecture-based trainingmechanism with a Deep Canonical Correlation Analysis (DCCA)-based loss toachieve collective learning of high-level feature representations from twodifferent views of the input images. Furthermore, we employ a Transformer +CNN-based network with a custom hourglass module as the robust backbone for theSiamese framework. Extensive experiments show that our approach outperformsmultiple state-of-the-art approaches across various benchmark datasets.