1. IDOL: Instant Photorealistic 3D Human Creation from a Single Image
- Author
-
Zhuang, Yiyu, Lv, Jiaxi, Wen, Hao, Shuai, Qing, Zeng, Ailing, Zhu, Hao, Chen, Shifeng, Yang, Yujiu, Cao, Xun, and Liu, Wei
- Subjects
Computer Science - Computer Vision and Pattern Recognition ,Computer Science - Graphics ,Computer Science - Machine Learning ,68U05, 68T07, 68T45 ,I.3.7 ,I.2.10 ,I.2.6 - Abstract
Creating a high-fidelity, animatable 3D full-body avatar from a single image is a challenging task due to the diverse appearance and poses of humans and the limited availability of high-quality training data. To achieve fast and high-quality human reconstruction, this work rethinks the task from the perspectives of dataset, model, and representation. First, we introduce a large-scale HUman-centric GEnerated dataset, HuGe100K, consisting of 100K diverse, photorealistic sets of human images. Each set contains 24-view frames in specific human poses, generated using a pose-controllable image-to-multi-view model. Next, leveraging the diversity in views, poses, and appearances within HuGe100K, we develop a scalable feed-forward transformer model to predict a 3D human Gaussian representation in a uniform space from a given human image. This model is trained to disentangle human pose, body shape, clothing geometry, and texture. The estimated Gaussians can be animated without post-processing. We conduct comprehensive experiments to validate the effectiveness of the proposed dataset and method. Our model demonstrates the ability to efficiently reconstruct photorealistic humans at 1K resolution from a single input image using a single GPU instantly. Additionally, it seamlessly supports various applications, as well as shape and texture editing tasks., Comment: 21 pages, 15 figures, includes main content, supplementary materials, and references
- Published
- 2024