Wu Cho-Ying ; Xu Qiangeng ; Neumann Ulrich

Abstract

This work studies learning from a synergy process of 3D Morphable Models(3DMM) and 3D facial landmarks to predict complete 3D facial geometry,including 3D alignment, face orientation, and 3D face modeling. Our synergyprocess leverages a representation cycle for 3DMM parameters and 3D landmarks.3D landmarks can be extracted and refined from face meshes built by 3DMMparameters. We next reverse the representation direction and show thatpredicting 3DMM parameters from sparse 3D landmarks improves the informationflow. Together we create a synergy process that utilizes the relation between3D landmarks and 3DMM parameters, and they collaboratively contribute to betterperformance. We extensively validate our contribution on full tasks of facialgeometry prediction and show our superior and robust performance on these tasksfor various scenarios. Particularly, we adopt only simple and widely-usednetwork operations to attain fast and accurate facial geometry prediction.Codes and data: https://choyingw.github.io/works/SynergyNet/