Current Issue Cover
三维穿衣人体重建——从传统方法到高保真模型

陈鸿鹄, 陶云帆, 张举勇(中国科学技术大学)

摘 要
三维穿衣人体重建,在计算机图形学和三维视觉领域中占有重要地位,广泛应用于多个领域。人体穿衣的多样性和动作的复杂性使得尤其是穿衣人体的高保真重建变得极其困难。深度学习技术优化了数据特征提取、隐式几何表示、神经渲染等关键环节,也推动了高保真穿衣人体重建技术的革命性进步。本文综述了人体重建的基本流程和组成模块,如各类输入数据、人体几何与动作表示、参数化模型以及三维到二维的渲染技术。同时,介绍了公开的穿衣人体数据集,简要回顾了近十年来人体重建算法的快速发展。文中详细探讨了几种主要的重建方法:稠密视角重建、非刚性运动重建(non-rigid structure from motion, NRSFM)、基于像素对齐的隐式几何重建以及生成模型方法。特别是,稠密视角重建能够生成高质量的人体几何,而NRSFM方法减少了对多视角的需求。基于像素对齐的方法重建细节丰富的人体几何,而生成模型方法利用多模态输入信息实现重建。文章最后总结了现有方法,并展望了未来研究方向,包括实现低成本高保真重建、加速重建过程、增强重建结果的可编辑性,以及在自然环境下进行重建的可能性。本文总结了近年来穿衣人体重建技术的进步,同时指出了未来研究可能集中的方向。
关键词
3D Clothed Human Reconstruction: From Traditional Techniques to High-Fidelity Models

Chen Honghu, Tao Yunfan, Zhang Juyong(University of Science and Technology of China)

Abstract
Three-dimensional human body reconstruction is a fundamental task in computer graphics and computer vision, with wide-ranging applications in virtual reality, human-computer interaction, motion analysis, and many other fields. The goal of human body reconstruction is to accurately recover a three-dimensional model of the human body from given input data, enabling further analysis and applications. However, achieving high-fidelity reconstruction of clothed human bodies remains challenging due to the diversity of human body shapes, variations in clothing, and complex human motion. In recent years, significant progress has been made in the field of three-dimensional human body reconstruction, thanks to the rapid development of deep learning methods. Deep learning techniques leverage multi-layer neural network models to effectively extract features from input data and learn more discriminative representations. In the context of human body reconstruction, deep learning methods have revolutionized data feature extraction, implicit geometric representation, and neural rendering, leading to remarkable advancements. This article aims to provide a comprehensive and accessible overview of three-dimensional human body reconstruction, elucidating the underlying methodologies, techniques, and algorithms used in this complex process. The article begins by introducing the classical framework of human body reconstruction, which consists of several key modules that collectively contribute to the reconstruction pipeline. These modules encompass various types of input data, including images, videos, and three-dimensional scans, which serve as the fundamental building blocks for the reconstruction process. One vital aspect of human body reconstruction is the representation of human body geometry. Capturing the nuanced contours and shapes that define the human form is a challenging task. The article explores different techniques for geometric representation, ranging from mesh-based approaches to implicit representations and voxel grids. These techniques aim to capture the intricate details of the human body while maintaining realistic body shapes and poses. The article also delves into the challenges associated with reconstructing clothed human bodies and discusses the efficacy of parametric models in encapsulating the complexities of clothing deformations. Another crucial component of human body reconstruction is the representation of human body motion. Accurately modeling and capturing the dynamic nature of human movements is essential for realistic reconstructions. The article comprehensively explores various approaches to modeling human body motion, including both articulated and non-rigid motion. Techniques such as skeletal animation, motion capture, and spatio-temporal analysis are discussed to enable accurate and lifelike representations of human body motion. Parametric models play a pivotal role in human body reconstruction as they provide a concise and expressive representation of the complete human body. The article examines optimization-based methods, regression-based approaches, and popular parametric models like SMPL (skinned multi-person linear) and SMPL+D (skinned multi-person linear plus offsets) for human body reconstruction. These models enable the capture of realistic body shapes, poses, and clothing deformations. The article discusses the advantages and limitations of these models and their applications in various domains. In recent years, deep learning techniques have had a transformative impact on three-dimensional human body reconstruction. The article explores the application of deep learning methodologies for data feature extraction, implicit geometric representation, and neural rendering. It highlights the advancements made in leveraging CNNs (convolutional neural networks), RNNs (recurrent neural networks), and GANs (generative adversarial networks) for various aspects of the reconstruction pipeline. These deep learning techniques have significantly improved the accuracy and realism of reconstructed human bodies. Furthermore, the article introduces publicly available datasets that have been specifically curated for clothed human body reconstruction. These datasets serve as invaluable resources for benchmarking and evaluating the performance of different reconstruction algorithms. They enable researchers to compare and analyze the effectiveness of various techniques, fostering advancements in the field. The article also provides a comprehensive survey of the rapid advancements in human body reconstruction algorithms over the past decade. It highlights breakthroughs in dense view reconstruction, non-rigid SFM (structure from motion) methods, pixel-aligned implicit geometry reconstruction, generative models, and parameterized models. The strengths, limitations, and potential applications of each approach are discussed, providing readers with a holistic understanding of the current state-of-the-art techniques. In conclusion, this article offers an in-depth and accessible exploration of three-dimensional human body reconstruction. It covers a wide range of topics, including data acquisition, geometry representation, motion modeling, and rendering modules. The article not only summarizes existing methods but also provides insights into future research directions, such as the pursuit of high-fidelity reconstructions at reduced costs, accelerated reconstruction speeds, editable reconstruction outcomes, and the ability to reconstruct human bodies within natural environments. These research endeavors aim to enhance the accuracy, realism, and practicality of three-dimensional human body reconstruction systems, unlocking new possibilities for various applications in academia and industry.
Keywords

订阅号|日报