Current Issue Cover
数字人脸渲染与外观恢复方法综述

郝琮晖1, 杜悠扬2, 王璐2, 王贝贝1(1.南开大学计算机学院;2.山东大学软件学院)

摘 要
近年来,数字人技术引起了广泛的兴趣和探讨,其中人脸作为数字人实现的关键成为人们关注的焦点,其相关技术已经在电影、游戏等领域得到了广阔应用。人们对实现逼真的人脸效果以及精确恢复人脸的需求日益增长,但由于人脸的多层材质结构、复杂的半透明皮肤效果以及毛孔、褶皱等微观特征的综合影响,实现高保真的、高效的人脸渲染一直是领域内的难题。此外,通过采集设备对人脸的几何和外观进行恢复是构建人脸数据的重要方式,然而对人脸的恢复也同样受限于高成本的采集设备、图像场景信息的分离困难和相关数据集的不足。本文对近年来关于数字人脸的渲染与恢复的相关方法进行综述。首先介绍了真实感人脸渲染的实现方法,根据实现的方式将它们分为基于扩散近似的方法和基于蒙特卡洛方法的方法,并着重介绍了其中基于近似扩散的方法的发展情况及面临的问题。之后对高精度人脸恢复方法中的主动照明和被动捕获分支分别进行梳理,对捕获人脸所需设备方法进行了总结。对结合深度学习的轻量级人脸恢复方法,本文将其分为几何细节的恢复、纹理贴图的恢复以及人脸几何与反射信息的恢复三个方向,依次进行详细介绍。最后,针对现有工作的发展情况,本文对未来人脸渲染及恢复方法的发展趋势进行了展望,希望本文可以对人脸渲染和外观恢复的初学者提供一些背景知识。
关键词
Survey of Digital Face Rendering and Appearance Recovery Methods

haoconghui, duyouyang1, wanglu, wangbeibei2(1.School of software, Shandong University;2.College of computer science,Nankai University)

Abstract
In recent years, digital human has sparked widespread interest and discussion. Within this domain, the human face has become a central focus of attention as a key component in achieving realistic digital humans. Consequently, the associated techniques have found extensive applications in fields such as film, gaming and virtual reality. There has been a growing demand for achieving facial realism rendering and high-quality facial inverse recovery. However, due to the complex multi-layered material structure of face, it’s challenging to achieve facial realism rendering. Furthermore, the rendering of skin is highly influenced by the composition of internal skin chemicals, such as melanin and hemoglobin. Even factors like temperature and blood flow rate may have an influence on skin’s appearance. The semi-transparency of skin introduces difficulties in simulating subsurface scattering effects, not to mention the micro features present on the face. All of the above makes it a challenging problem in the rendering domain. Additionally, due to people""s exposure to numerous real human faces in daily life, there is a heightened sensitivity to the texture and details of digital human faces, which places even greater demands on their realism and accuracy. Meanwhile, the recovery of facial geometry and appearance is a crucial method for building facial datasets. However, high-quality facial recovery is often constrained by the high costs of acquisition equipment, and many studies are also limited by the acquisition speed for facial data, making it challenging to capture dynamic facial appearance. Lightweight recovery methods also face challenges related to the lack of facial material datasets and the difficulty of separating scene information from images. This paper provides an overview of recent advances in the rendering and recovery of digital human faces. Firstly, we introduce methods for achieving realistic facial rendering, categorizing them based on diffusion approximation and Monte Carlo approaches. Among these, methods based on diffusion approximation are limited by strict assumptions and come with some errors, but they offer fast computation. On the other hand, methods based on the Monte Carlo approach provide high precision and robust results but require longer computation times to converge. We have placed particular emphasis on the development and challenges of methods based on diffusion approximation, as well as recent Monte Carlo research aimed at improving the convergence rate for applications in facial rendering including zero-variance random walks, next event estimation and path guiding. Secondly, this paper further categorizes high-precision facial recovery methods based on whether specialized lighting equipment is used, distinguishing between active illumination and passive capture techniques, and provides detailed explanations for each category. Additionally, it summarizes the equipment required for these recovery processes. We also explore lightweight facial recovery methods incorporating deep learning, classifying them into three categories: geometric detail recovery, texture mapping recovery, and facial geometry combined with reflection information recovery. It offers in-depth insights into each of these approaches. Finally, the paper outlines the future trends in facial realism rendering and recovery methods based on the current state of research. It is hoped that this paper can provide novice researchers in the field of facial rendering and appearance recovery with valuable background knowledge.
Keywords

订阅号|日报