Yue Zongsheng ; Loy Chen Change

Abstract

While deep learning-based methods for blind face restoration have achievedunprecedented success, they still suffer from two major limitations. First,most of them deteriorate when facing complex degradations out of their trainingdata. Second, these methods require multiple constraints, e.g., fidelity,perceptual, and adversarial losses, which require laborious hyper-parametertuning to stabilize and balance their influences. In this work, we propose anovel method named DifFace that is capable of coping with unseen and complexdegradations more gracefully without complicated loss designs. The key of ourmethod is to establish a posterior distribution from the observed low-quality(LQ) image to its high-quality (HQ) counterpart. In particular, we design atransition distribution from the LQ image to the intermediate state of apre-trained diffusion model and then gradually transmit from this intermediatestate to the HQ target by recursively applying a pre-trained diffusion model.The transition distribution only relies on a restoration backbone that istrained with $L_2$ loss on some synthetic data, which favorably avoids thecumbersome training process in existing methods. Moreover, the transitiondistribution can contract the error of the restoration backbone and thus makesour method more robust to unknown degradations. Comprehensive experiments showthat DifFace is superior to current state-of-the-art methods, especially incases with severe degradations. Code and model are available athttps://github.com/zsyOAOA/DifFace.