Stoffregen Timo ; Scheerlinck Cedric ; Scaramuzza Davide ; Drummond Tom ; Barnes Nick ; Kleeman Lindsay ; Mahony Robert

Abstract

Event cameras are paradigm-shifting novel sensors that report asynchronous,per-pixel brightness changes called 'events' with unparalleled low latency.This makes them ideal for high speed, high dynamic range scenes whereconventional cameras would fail. Recent work has demonstrated impressiveresults using Convolutional Neural Networks (CNNs) for video reconstruction andoptic flow with events. We present strategies for improving training data forevent based CNNs that result in 20-40% boost in performance of existingstate-of-the-art (SOTA) video reconstruction networks retrained with ourmethod, and up to 15% for optic flow networks. A challenge in evaluating eventbased video reconstruction is lack of quality ground truth images in existingdatasets. To address this, we present a new High Quality Frames (HQF) dataset,containing events and ground truth frames from a DAVIS240C that arewell-exposed and minimally motion-blurred. We evaluate our method on HQF +several existing major event camera datasets.