Li Kailin ; Yang Lixin ; Zhan Xinyu ; Lv Jun ; Xu Wenqiang ; Li Jiefeng ; Lu Cewu

Abstract

Estimating the articulated 3D hand-object pose from a single RGB image is ahighly ambiguous and challenging problem, requiring large-scale datasets thatcontain diverse hand poses, object types, and camera viewpoints. Mostreal-world datasets lack these diversities. In contrast, data synthesis caneasily ensure those diversities separately. However, constructing both validand diverse hand-object interactions and efficiently learning from the vastsynthetic data is still challenging. To address the above issues, we proposeArtiBoost, a lightweight online data enhancement method. ArtiBoost can coverdiverse hand-object poses and camera viewpoints through sampling in aComposited hand-object Configuration and Viewpoint space (CCV-space) and canadaptively enrich the current hard-discernable items by loss-feedback andsample re-weighting. ArtiBoost alternatively performs data exploration andsynthesis within a learning pipeline, and those synthetic data are blended intoreal-world source data for training. We apply ArtiBoost on a simple learningbaseline network and witness the performance boost on several hand-objectbenchmarks. Our models and code are available athttps://github.com/lixiny/ArtiBoost.