Recently, researchers have made significant progress combining the advancesin deep learning for learning feature representations with reinforcementlearning. Some notable examples include training agents to play Atari gamesbased on raw pixel data and to acquire advanced manipulation skills using rawsensory inputs. However, it has been difficult to quantify progress in thedomain of continuous control due to the lack of a commonly adopted benchmark.In this work, we present a benchmark suite of continuous control tasks,including classic tasks like cart-pole swing-up, tasks with very high state andaction dimensionality such as 3D humanoid locomotion, tasks with partialobservations, and tasks with hierarchical structure. We report novel findingsbased on the systematic evaluation of a range of implemented reinforcementlearning algorithms. Both the benchmark and reference implementations arereleased at https://github.com/rllab/rllab in order to facilitate experimentalreproducibility and to encourage adoption by other researchers.