Ebel Patrick ; Garnot Vivien Sainte Fare ; Schmitt Michael ; Wegner Jan Dirk ; Zhu Xiao Xiang

Abstract

Clouds and haze often occlude optical satellite images, hindering continuous,dense monitoring of the Earth's surface. Although modern deep learning methodscan implicitly learn to ignore such occlusions, explicit cloud removal aspre-processing enables manual interpretation and allows training models whenonly few annotations are available. Cloud removal is challenging due to thewide range of occlusion scenarios -- from scenes partially visible throughhaze, to completely opaque cloud coverage. Furthermore, integratingreconstructed images in downstream applications would greatly benefit fromtrustworthy quality assessment. In this paper, we introduce UnCRtainTS, amethod for multi-temporal cloud removal combining a novel attention-basedarchitecture, and a formulation for multivariate uncertainty prediction. Thesetwo components combined set a new state-of-the-art performance in terms ofimage reconstruction on two public cloud removal datasets. Additionally, weshow how the well-calibrated predicted uncertainties enable a precise controlof the reconstruction quality.