Current Issue Cover
非规律运动伪迹干扰鲁棒的人脸视频心率检测研究

成娟1, 殷辰楚1, 宋仁成1, 付静1, 刘羽2(1.合肥工业大学生物医学工程系;2.合肥工业大学)

摘 要
目的 基于远程光电容积脉搏波描记法(remote photoplethysmograph,rPPG)的非接触人脸视频心率检测广泛应用于移动健康监护领域,由于其携带的生理参数信息幅值微弱,容易受到运动伪迹干扰。据此,该文提出了一种结合非负矩阵分解(nonnegative matrix factorization,NMF)和独立向量分析(independent vector analysis,IVA)的非规律运动伪迹去除的视频心率检测方法,记为NMF-IVA。方法 首先,将面部感兴趣区域(region of interest,ROI)分为多个子区域(sub ROIs,SROIs),利用平均光照强度、光照强度变化、信噪比这三个指标筛选出3个最优质的SROIs,并获取每个SROI的绿色通道时间序列。其次,将3个绿色通道时间序列去趋势、带通滤波后送入NMF-IVA进行盲源分离。然后,对分离后的源信号进行功率谱密度分析,并且将峰值信噪比最高且主频落在心率感兴趣范围内的源信号确定为血容量脉冲(blood volume pulse,BVP)信号。最后,将BVP信号的主频确定为所测量心率的主频,从而计算出心率值。结果 实验在2个公开数据集UBFC-RPPG和UBFC-PHYS,及1个真实场景自采数据集上与最相关的7种典型的rPPG方法进行了比较,在UBFC-RPPG数据集上,相比于性能第2的单通道滤波(single channel filtering,SCF)方法,均方根误差提升了1.39 次/分(beat per minute,bpm)、平均绝对误差提升了1.25 bpm、皮尔逊相关系数提升了0.02;在UBFC-PHYS数据集上的T2情况下,其性能提升最为显著,相比于性能第2的独立向量分析(independent vector analysis,IVA)方法,均方根误差提升了16.42 bpm、平均绝对误差提升了9.91 bpm、皮尔逊相关系数提升了0.64;在自采数据集上,除了低于深度学习方法性能之外,所提NMF-IVA方法在传统方法中取得了最好的结果。结论 所提NMF-IVA方法对规律信号提取具有敏感性,即便是在头部存在剧烈非规律运动情况下,相比于传统方法亦能取得最优的结果,该结果能够媲美基于深度学习的方法。
关键词
Research on Facial-video-based Heart Rate Measurement against Irregular Motion Artifacts

(合肥工业大学)

Abstract
Objective Heart rate (HR) is one of the most important physiological parameters that can reflect both physical and mental status of individuals. Currently, various methods have been developed to estimate HR values using contact and noncontact sensors. The advantage of all the noncontact methods is to provide a more comfortable and unobtrusive way to estimate HR and avoid discomfort or skin allergy caused by the conventional contact methods. Among them, since the pulse-induced subtle color variations of facial skins can be measured from consumer-level cameras, the camera-based non-contact heart rate (HR) detection technology, also called remote photoplethysmograph (rPPG), has been widely used in the fields of mobile health monitoring, driving safety, emotion awareness, etc. The principle of camera-based rPPG measurement is similar to that of traditional PPG measurement, that is, the pulsatile blood propagating in cardiovascular systems changes blood volumes in microvascular tissue beds beneath skins within each heartbeat and thereby a fluctuation is accordingly produced. However, such a kind of technology is susceptible to motion artifacts due to weak amplitudes of physiological parameter information it carries. For instance, subjects" heads may move involuntarily during interviews, presentations, and other socially stressful situations, resulting in the degradation of rPPG-based HR detection performance. Accordingly, this paper proposes a novel motion-robust rPPG method through combining nonnegative matrix factorization (NMF) and independent vector analysis (IVA), termed as NMF-IVA, to remove irregular motion artifacts. Method First, the whole facial region of interest (ROI) is divided into several sub ROIs (SROIs), among which three optimal SROIs are selected on the basis of three indicators including average light intensity, and light intensity variation of a certain SROI, as well as signal to noise ratio (SNR) of the green-channel signal derived from the SROI. Afterwards, three green-channel time series are derived from the corresponding three optimal SROIs. Second, the three channels of time series are detrended, bandpass filtered and then sent to the proposed NMF-IVA as input. After the NMF-IVA operation, three source signals are extracted and then processed by power spectral density analysis. The one with the highest peak SNR and the corresponding dominant frequency falling within the interested HR range will be identified as the blood volume pulse (BVP) signal, whose dominant frequency is identified as that of the estimated HR. Result We compare the proposed NMF-IVA method with seven typical rPPG methods on two publicly available datasets, UBFC-RPPG and UBFC-PHYS, as well as one house-in dataset. On the UBFC-RPPG dataset, compared to the second-best performance of single channel filtering (SCF) method, the proposed NMF-IVA achieves the best performance, with an improved root mean square error (RMSE) of HR measurement by 1.39 beat per minute (bpm), an improved mean absolute error (MAE) by 1.25 bpm, and a higher pearson"s correlation coefficient (PCC) by 0.02. Although both the MAE and the RMSE achieved by the proposed NMF-IVA method are lower than those of deep-learning-based methods, the PCC of the NMF-IVA is comparable to that of deep-learning-based ones, which demonstrates the effectiveness of the proposed NMF-IVA method. As for the UBFC-PHYS dataset when compared to traditional rPPG methods, when during the T1 condition, the performance of the proposed NMF-IVA method is better than that of the second-best SCF method, with an improved RMSE by 6.45 bpm, an improved MAE by 2.53 bpm, and a higher PCC by 0.18. Besides, when compared to deep-learning-based ones, the proposed NMF-IVA method can achieve the second-best performance. The performance improvement of the proposed NMF-IVA is most noticeable during T2 condition on the UBFC-PHYS dataset. Specifically, when compared to the second-best performance of independent vector analysis (IVA) method, the above three metrices are improved by 16.42 bpm, 9.91 bpm and 0.64, respectively. As for the UBFC-PHYS dataset, when during the T3 condition, the best performance is still achieved by the proposed NMF-IVA method. When compared to the second-best performance of independent component analysis (ICA) method, the corresponding three metrices are improved by 8.54 bpm, 6.14 bpm and 0.37, respectively. Besides, the performance of the proposed NMF-IVA method can be comparable to that of deep-learning-based ones both in T2 and T3 conditions. As for the house-in dataset, except that lower than deep learning-based methods, the proposed NMF-IVA method achieves the best performance when compared to those of the traditional methods. Conclusion The results show that the proposed NMF-IVA method achieves the best results on all the three datasets when compared to traditional rPPG methods, and the performance improvement is most noticeable during irregular motion artifact conditions involving head motions with large amplitudes. On the other hand, when compared to deep-learning-based methods, the performance of the proposed NMF-IVA method is a little bit poor. The possible reason is that deep learning technology has excellent abilities in learning and extracting effective features. However, sufficient training samples and generalization should be considered when adopting deep-learning-based methods. Additionally, before deriving the BVP source of high-quality, upsampling technique is employed, which leads to a relatively large time consumption. In the future, the HR estimation performance and the upsampling rate should be traded off. Due to that the proposed NMF-IVA method has the advantage in extracting regular signals, our study can provide a new solution for promoting the practical application ability of rPPG technology.
Keywords

订阅号|日报