Liang Jingyun ; Lugmayr Andreas ; Zhang Kai ; Danelljan Martin ; Van Gool Luc ; Timofte Radu

Abstract

Normalizing flows have recently demonstrated promising results for low-levelvision tasks. For image super-resolution (SR), it learns to predict diversephoto-realistic high-resolution (HR) images from the low-resolution (LR) imagerather than learning a deterministic mapping. For image rescaling, it achieveshigh accuracy by jointly modelling the downscaling and upscaling processes.While existing approaches employ specialized techniques for these two tasks, weset out to unify them in a single formulation. In this paper, we propose thehierarchical conditional flow (HCFlow) as a unified framework for image SR andimage rescaling. More specifically, HCFlow learns a bijective mapping betweenHR and LR image pairs by modelling the distribution of the LR image and therest high-frequency component simultaneously. In particular, the high-frequencycomponent is conditional on the LR image in a hierarchical manner. To furtherenhance the performance, other losses such as perceptual loss and GAN loss arecombined with the commonly used negative log-likelihood loss in training.Extensive experiments on general image SR, face image SR and image rescalinghave demonstrated that the proposed HCFlow achieves state-of-the-artperformance in terms of both quantitative metrics and visual quality.