Vaksman Gregory ; Elad Michael ; Milanfar Peyman

Abstract

The non-local self-similarity property of natural images has been exploitedextensively for solving various image processing problems. When it comes tovideo sequences, harnessing this force is even more beneficial due to thetemporal redundancy. In the context of image and video denoising, manyclassically-oriented algorithms employ self-similarity, splitting the data intooverlapping patches, gathering groups of similar ones and processing thesetogether somehow. With the emergence of convolutional neural networks (CNN),the patch-based framework has been abandoned. Most CNN denoisers operate on thewhole image, leveraging non-local relations only implicitly by using a largereceptive field. This work proposes a novel approach for leveragingself-similarity in the context of video denoising, while still relying on aregular convolutional architecture. We introduce a concept of patch-craftframes - artificial frames that are similar to the real ones, built by tilingmatched patches. Our algorithm augments video sequences with patch-craft framesand feeds them to a CNN. We demonstrate the substantial boost in denoisingperformance obtained with the proposed approach.