Li Yiyu ; Xu Ke ; Hancke Gerhard Petrus ; Lau Rynson W. H.

Abstract

Images captured under sub-optimal illumination conditions may contain bothover- and under-exposures. Current approaches mainly focus on adjusting imagebrightness, which may exacerbate the color tone distortion in under-exposedareas and fail to restore accurate colors in over-exposed regions. We observethat over- and under-exposed regions display opposite color tone distributionshifts with respect to each other, which may not be easily normalized in jointmodeling as they usually do not have ``normal-exposed'' regions/pixels asreference. In this paper, we propose a novel method to enhance images with bothover- and under-exposures by learning to estimate and correct such colorshifts. Specifically, we first derive the color feature maps of the brightenedand darkened versions of the input image via a UNet-based network, followed bya pseudo-normal feature generator to produce pseudo-normal color feature maps.We then propose a novel COlor Shift Estimation (COSE) module to estimate thecolor shifts between the derived brightened (or darkened) color feature mapsand the pseudo-normal color feature maps. The COSE module corrects theestimated color shifts of the over- and under-exposed regions separately. Wefurther propose a novel COlor MOdulation (COMO) module to modulate theseparately corrected colors in the over- and under-exposed regions to producethe enhanced image. Comprehensive experiments show that our method outperformsexisting approaches. Project webpage: https://github.com/yiyulics/CSEC.