Kolotouros Nikos ; Pavlakos Georgios ; Jayaraman Dinesh ; Daniilidis Kostas

Abstract

This paper focuses on the problem of 3D human reconstruction from 2Devidence. Although this is an inherently ambiguous problem, the majority ofrecent works avoid the uncertainty modeling and typically regress a singleestimate for a given input. In contrast to that, in this work, we propose toembrace the reconstruction ambiguity and we recast the problem as learning amapping from the input to a distribution of plausible 3D poses. Our approach isbased on the normalizing flows model and offers a series of advantages. Forconventional applications, where a single 3D estimate is required, ourformulation allows for efficient mode computation. Using the mode leads toperformance that is comparable with the state of the art among deterministicunimodal regression models. Simultaneously, since we have access to thelikelihood of each sample, we demonstrate that our model is useful in a seriesof downstream tasks, where we leverage the probabilistic nature of theprediction as a tool for more accurate estimation. These tasks includereconstruction from multiple uncalibrated views, as well as human modelfitting, where our model acts as a powerful image-based prior for meshrecovery. Our results validate the importance of probabilistic modeling, andindicate state-of-the-art performance across a variety of settings. Code andmodels are available at: https://www.seas.upenn.edu/~nkolot/projects/prohmr.