Zhou Peng ; Xie Lingxi ; Ni Bingbing ; Tian Qi

Abstract

The style-based GAN (StyleGAN) architecture achieved state-of-the-art resultsfor generating high-quality images, but it lacks explicit and precise controlover camera poses. The recently proposed NeRF-based GANs made great progresstowards 3D-aware generators, but they are unable to generate high-qualityimages yet. This paper presents CIPS-3D, a style-based, 3D-aware generator thatis composed of a shallow NeRF network and a deep implicit neural representation(INR) network. The generator synthesizes each pixel value independently withoutany spatial convolution or upsampling operation. In addition, we diagnose theproblem of mirror symmetry that implies a suboptimal solution and solve it byintroducing an auxiliary discriminator. Trained on raw, single-view images,CIPS-3D sets new records for 3D-aware image synthesis with an impressive FID of6.97 for images at the $256\times256$ resolution on FFHQ. We also demonstrateseveral interesting directions for CIPS-3D such as transfer learning and3D-aware face stylization. The synthesis results are best viewed as videos, sowe recommend the readers to check our github project athttps://github.com/PeterouZh/CIPS-3D