Heart rate is one of the important indicators that can directly reflect the health of the human body. Heart rate detection has been applied to many aspects of the medical field

such as physical examination

major surgery

and postoperative treatment. Heart rate detection based on face video processing has recently been performed through a noncontact manner without complex operations and sense of restraint. However

the existing methods cannot predict well in complex realistic scenes

including shaking target. If face detection in video processing is accompanied with face shaking

the facial region of interest is selected inaccurately. Such methods also disregard spatial scale features

which are significant to extract blood volume pulse (BVP) signal. The results of current methods are consequently inadequate. To this end

a new non-contact heart rate detection method based on face video processing is proposed to reduce the influence of face shake and improve precision. Our method consists of three major steps. First

we deal with video through a robust face detecting and tracking model to obtain a refined face video in which facial shake is eliminated. Considering that the universal Viola-Jones face detection model generates an incorrect face area when a face is tilted along consecutive frames

discriminative response map fitting is used to detect important feature points for tracking the right face area. For the first frame image

we mark 66 landmark points on the facial organ (eyes

nose

mouth

and facial shape) and four vertexes of facial rectangle. These feature points are then entered into the Kanade-Lucas-Tomasi tracking model to calculate the facial rectangle of subsequent frames. According to the oblique angle of each facial rectangle

the corresponding face image is rotated to a vertical position. Second

the modified face video is handled by a space-time processing algorithm for amplifying the video color variations to separate the spatial scale characteristics of the video and intercept the frequency range of blood volume changes. We average the chrominance of skins under the eyes as clean BVP. Finally

for the BVP signal that belongs to a small sample

frequency domain analysis and iterative Fourier coefficient interpolation are combined to estimate heart rate. Iteration is performed 1 000 times for improved accuracy. The proposed method is tested on two different types of face video libraries comprising still and shaking face videos. Each video library contains 60 10-second videos from 20 participants

including twelve men and eight women. We conduct a quantitative analysis for the typical method provided by Poh

the up-to-date method provided by Liu

and our method. Statistically

the overall accuracies of our method in still and shaking face videos are 97.84% and 97.30%

respectively. The accuracy is increased by more than 1% in still face videos and more than 7% in shaking face videos. Video-based heart rate detection in complex realistic scenes is affected by facial shaking

which leads to significantly reduced accuracy. Neglecting spatial scale characteristics and the small sample affect detection performance. Hence

this study proposes a novel heart rate detection method applied to complex realistic scenes. We detect and track important facial feature points to effectively analyze the state of facial shaking and adjust the facial slope. After space-time processing for selecting a proper spatial scale

a clean BVP signal is extracted to calculate heart rate iteratively. Experimental results indicate that our method has high accuracy and preferable adaptive performance to cases involving facial shaking.