Alex X. Lee; Coline Devin; Yuxiang Zhou; Thomas Lampe; Konstantinos Bousmalis; Jost Tobias Springenberg; Arunkumar Byravan; Abbas Abdolmaleki; Nimrod Gileadi; David Khosid; Claudio Fantacci; Jose Enrique Chen; Akhil Raju; Rae Jeong; Michael Neunert; Antoine Laurens; Stefano Saliceti; Federico Casarini; Martin Riedmiller; Raia Hadsell; Francesco Nori

Abstract

We study the problem of robotic stacking with objects of complex geometry. We propose a challenging and diverse set of such objects that was carefully designed to require strategies beyond a simple "pick-and-place" solution. Our method is a reinforcement learning (RL) approach combined with vision-based interactive policy distillation and simulation-to-reality transfer. Our learned policies can efficiently handle multiple object combinations in the real world and exhibit a large variety of stacking skills. In a large experimental study, we investigate what choices matter for learning such general vision-based agents in simulation, and what affects optimal transfer to the real robot. We then leverage data collected by such policies and improve upon them with offline RL. A video and a blog post of our work are provided as supplementary material.