Current Issue Cover

牛雪松1,2, 韩琥1,3, 山世光1,2,4(1.中国科学院计算技术研究所智能信息处理重点实验室, 北京 100190;2.中国科学院大学, 北京 100049;3.鹏城实验室, 深圳 518066;4.中国科学院脑科学与智能技术卓越创新中心, 上海 200031)

摘 要
远程光电容积脉搏波描记法(remote photoplethysmography,rPPG)是指通过摄像头等传感器来捕捉由心动周期造成的皮肤颜色周期性变化的技术。利用rPPG技术可以提取血液体积脉冲信号,进而测量心率、呼吸率和心跳变异性等心动周期相关的生理指标。近年基于rPPG的生理指标测量方法取得了飞速发展,准确性和鲁棒性已得到了大幅提高。该类技术的算法流程主要包括图像/视频的获取和感兴趣区域提取、血液体积脉冲信号提取和生理指标测量等步骤。基于这一算法流程,本文从算法所依据的假设或先验知识出发,对相关文献进行了总结和讨论。此外,还从评测任务、评测数据、评测指标及评测协议等4个方面系统整理针对基于rPPG的生理指标测量方法的评价体系。最后,本文讨论了该领域当前所面临的挑战并展望了可能的技术路线。
Remote photoplethysmography-based physiological measurement: a survey

Niu Xuesong1,2, Han Hu1,3, Shan Shiguang1,2,4(1.Key Laboratory of Intelligent Information Processing of Chinese Academy of Sciences(CAS), Institute of Computing Technology, CAS, Beijing 100190, China;2.University of Chinese Academy of Sciences, Beijing 100049, China;3.Peng Cheng Laboratory, Shenzhen 518066, China;4.Chinese Academy of Sciences(CAS) Center for Excellence in Brain Science and Intelligence Technology, CAS, Shanghai 200031, China)

Physiological signals, such as heart rate (HR), respiration frequency (RF), and heart rate variability (HRV), are important clues to analyze a person's health and affective status. Traditional measurements of physiological signals are based on the electrocardiography (ECG) or contact photoplethysmography (cPPG) technology. However, both technologies require professional equipment, which may cause inconvenience and discomfort for subjects. Remote photoplethysmography (rPPG) technology for remote measurement of physiological signals has progressed considerably and recently attracted considerable research attention. The rPPG technology, which is based on skin color variations due to the periodical optical absorption of skin tissue caused by cardiac activity, demonstrates high potential in many applications, such as healthcare, sleep monitoring, and defection detection. The process for rPPG-based physiological measurement can be divided into three steps. First, regions of interest (ROIs) are extracted from the face video. Second, blood volume pulse(BVP) signal is reconstructed from signals generated from the ROIs. Finally, the reconstructed BVP signal is used for physiological measurements. The reconstruction of the BVP signal is the key step for rPPG-based remote physiological measurements. A detailed review of methods for rPPG-based remote physiological measurement is presented in this study from the aspect of assumptions they use, which can be categorized into three kinds, i.e., methods based on the skin reflection model, methods based on the BVP signal's physical characteristics, and data-driven methods. Studies on the skin reflection model-based methods can be further categorized into spatial skin and skin reflection models of different color channels. Studies on methods that using the BVP signal's physical characteristics can be further categorized into blind signal separation, manifold projection, low rank factorization, and frequency domain constraint. Studies on data-driven methods can be further categorized into methods based on hand-crafted features and deep learning. A detailed review of evaluations of different rPPG-based physiological measurement methods is also presented from the aspects of tasks, databases, metrics, and protocols. Evaluation tasks used for remote physiological measurement include average heart rate measurement, respiration frequency measurement, and heart rate variability analysis. Databases of rPPG-based physiological measurements are summarized according to database scale and variations. Evaluation metrics for remote physiological measurement can be categorized into statistics of error, correlation, and signal quality. Evaluation protocols for data-driven methods are summarized into fixed partition, subject-independent division, subject-exclusive division, and cross-database protocols. Finally, we discuss the challenges of the rPPG-based remote physiological measurement and put forward the potential research directions for future investigations. Challenges include video quality (i.e., video compression and pre-processing of frames), influence of subject's head movements, variations of lighting conditions, and lacking data. Future research trends include designing hand-crafted methods for different challenge scenarios and exploring technologies, such as self-supervised, semi-supervised, and weakly-supervised learning, for data-driven methods.