结合混合池化的双流人脸活体检测网络

汪亚航; 宋晓宁; 吴小俊

doi:10.11834/jig.190419

图像分析和识别 | 浏览量 : 0 下载量: 19 CSCD: 6

PDF
导出
分享
收藏
专辑

结合混合池化的双流人脸活体检测网络
Two-stream face spoofing detection network combined with hybrid pooling
2020年25卷第7期页码：1408-1420
收稿：2019-08-20，

修回：2019-12-17，

录用：2019-12-24，

纸质出版：2020-07-16
DOI： 10.11834/jig.190419
稿件说明：

移动端阅览

汪亚航, 宋晓宁, 吴小俊. 结合混合池化的双流人脸活体检测网络[J]. 中国图象图形学报, 2020,25(7):1408-1420. DOI： 10.11834/jig.190419.

Yahang Wang, Xiaoning Song, Xiaojun Wu. Two-stream face spoofing detection network combined with hybrid pooling[J]. Journal of Image and Graphics, 2020, 25(7): 1408-1420. DOI： 10.11834/jig.190419.

摘要

目的

人脸识别技术在很多领域起着重要作用，但大量的欺诈攻击对人脸识别产生了威胁，比如打印攻击和重放攻击。传统的活体检测方法是以手工方式提取特征且缺乏对时间维度的考虑，导致检测效果不佳。针对以上问题，提出一种结合混合池化的双流活体检测网络。

方法

对数据集提取光流图像并进行面部检测，得到双流网络的两个输入；在双流网络末端加入空间金字塔和全局平均混合池化，利用全连接层对池化后的特征进行分类并进行分数层面的融合；对空间流网络和时间流网络进行融合得到一个最优结果，同时考虑了不同颜色空间对检测性能的影响。

结果

在CASIA-FASD（CASIA face anti-spoofing database）和replay-attack两个数据集上做了多组对比实验，在CASIA-FASD数据集上，等错误率（equal error rate，EER）为1.701%；在replay-attack数据集上，等错误率和半错误率（half total error rate，HTER）分别为0.091%和0.082%。

结论

结合混合池化的双流活体检测网络充分考虑时间维度，提出的空间金字塔和全局平均混合池化策略能有效地利用特征。针对包含多种攻击类型、图像质量差异较大的数据集，本文提出的网络模型均能取得较低的错误率。

Abstract

Objective

The convenient and efficient technique of biometric feature identification triggers the wide and profound research on face recognition

which draws extensive concern and has been widely used in various authorization scenarios

including mobile phone unlocking

access control

and face payment. The rise of Internet of Things application terminals including mobile phone cameras make much private facial information of user easily and instantaneously available

which lead to serious threat for the traditional face identification systems. Therefore

more efficient and accurate identifying of face spoofing attacks

including replay and print attacks

reducing threats

and ensuring system security have become an urgent issue to be addressed. Face anti-spoofing mainly ensures that the real entity is displayed in front of the camera

rather than a video or a photo. Many techniques

including convolutional neural network (CNN) methods

have appeared to address this issue and aroused a great concern constantly in recent years. Typically

the hand-crafted features including LBP (local binary pattern)

HOG (histogram of oriented gradient) jointly using a kind of classifier such as SVM (support vector machine)

is to be the most commonly used technique for face anti-spoofing. However

the diversity of spoofing means still causes great difficulties in manually extracting effective features. Most CNN methods cannot fully exploit the valuable information in the case of temporal dimensionality and thus result in poor detection. To address this issue

this study presents a two-stream spoofing detection network based on a scheme of hybrid pooling.

Method

Those methods that only use spatial information to solve the face anti-spoofing problem are not generalized. The face anti-spoofing framework proposed in this paper includes spatial stream

temporal stream and fusion modules. The entire design process of experiment is divided into two aspects

one is to use spatial net to achieve spatial information

and another is to apply the temporal net to acquire temporal information. The algorithm implementation is detailed as following. First

optical flow pictures are extracted from the dataset

on which face detection is performed

the optical flow pictures are utilized as the input of the temporal stream to learn temporal information

and the original face pictures are used as the input of the spatial stream to learn spatial information. Second

a shallow network is adopted for the spatial stream network

in which spatial pyramid pooling is combined with global average pooling at the end of the network to obtain hybrid pooling features. We then perform classification using a fully connected layer. For the temporal stream network

we adopt a deep network using residual blocks given that effective temporal features are difficult to extract. Spatial pyramid pooling and global average pooling are also synthesized at the end of the network for the classification. Results obtained from the spatial and temporal stream networks are used for the final score fusion for improved classification. The choice of different color spaces may affect the final result; thus

we compare the effectiveness of different color spaces by experiment and determine the optimal color space of the presented model.

Result

The proposed method is demonstrated on two public benchmark datasets

namely

CASIA-FASD (CASIA face anti-spoofing database) and replay-attack. We obtain the results of 2.141% EER (equal error rate) in the spatial stream

9.005% EER in the temporal stream

and 1.701% EER in fusion on CASIA-FASD; and 0.071% EER and 0.109% HTER(half total error rate) in the spatial stream

17.045% EER and 21.781% HTER in the temporal stream

and 0.091% EER and 0.082% HTER in fusion on replay-attack.

Conclusion

The two-stream spoofing detection network combined with the hybrid pooling scheme achieves promising results on different datasets. The proposed method utilizes the effective information in temporal dimensionality. The hybrid of spatial pyramid and global average pooling means can exploit the effective information in multiple scales. We can also relieve the burden of the high dimensionality of datasets. Experimental results obtained from popular datasets demonstrate the merits of the proposed method

especially its robustness to the diversity of spoofing means and the large differences in the quality of image datasets. Our promising results will encourage further work on synthesizing an informative stream and lead to successful solutions for other application domains in the future.

关键词

Keywords

references

Arashloo S R, Kittler J and Christmas W. 2017. An anomaly detection approach to face spoofing detection:a new formulation and evaluation protocol. IEEE Access, 5:13868-13882[DOI:10.1109/ACCESS.2017.2729161]

Arnheim R and Gibson J J. 1952. The perception of the visual world. Journal of Aesthetics and Art Criticism, 11(2):172-173[DOI:10.2307/426044]

Asim M, Ming Z and Javed M Y. 2017. CNN based spatio-temporal feature extraction for face anti-spoofing//Proceedings of the 2nd International Conference on Image, Vision and Computing (ICIVC). Chengdu, China: IEEE: 234-238[ DOI: 10.1109/ICIVC.2017.7984552 http://dx.doi.org/10.1109/ICIVC.2017.7984552 ]

Boulkenafet Z, Komulainen J and Hadid A. 2016. Face spoofing detection using colour texture analysis. IEEE Transactions on Information Forensics and Security, 11(8):1818-1830[DOI:10.1109/tifs.2016.2555286]

Boulkenafet Z, Komulainen J and Hadid A. 2017. Face anti-spoofing using speeded-up robust features and fisher vector encoding. IEEE Signal Processing Letters, 24(2):141-145[DOI:10.1109/LSP.2016.2630740]

Chingovska I, Anjos A and Marcel S. 2012. On the effectiveness of local binary patterns in face anti-spoofing//Proceedings of 2012 International Conference of Biometrics Special Interest Group (BIOSIG). Darmstadt, Germany: IEEE: 1-7

Dalal N and Triggs B. 2005. Histograms of oriented gradients for human detection//Proceedings of 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR). San Diego, CA, USA: IEEE: 886-893[ DOI: 10.1109/CVPR.2005.177 http://dx.doi.org/10.1109/CVPR.2005.177 ]

de Souza G B, Papa J P and Marana A N. 2018. On the learning of deep local features for robust face spoofing detection[EB/OL].[2019-08-15] . https://arxiv.org/pdf/1806.07492.pdf https://arxiv.org/pdf/1806.07492.pdf

Duc N M and Minh B Q. 2009. Your face is not your password//Black Hat Conference.[EB/OL].[2009-08-15] . https://www.blackhat.com/presentations/bh-dc-09/Nguyen/BlackHat-DC-09-Nguyen-Face-not-your-password-slides.pdf https://www.blackhat.com/presentations/bh-dc-09/Nguyen/BlackHat-DC-09-Nguyen-Face-not-your-password-slides.pdf

Erdem C E, Ulukaya S, Karaali A and Erdem A T. 2011.Combining haar feature and skin color based classifiers for face detection//Proceedings of 2011 IEEE International Conference on Acoustics, Speech and Signal Processing. Prague, Czech Republic: IEEE: 1497-1500[ DOI: 10.1109/ICASSP.2011.5946777 http://dx.doi.org/10.1109/ICASSP.2011.5946777 ]

Erdogmus N and Marcel S. 2014. Spoofing face recognition with 3D masks. IEEE Transactions on Information Forensics and Security, 9(7):1084-1097[DOI:10.1109/TIFS.2014.2322255]

Galbally J and Marcel S. 2014. Face anti-spoofing based on general image quality assessment//Proceedings of the 22nd International Conference on Pattern Recognition. Stockholm, Sweden: IEEE: 1173-1178[ DOI: 10.1109/ICPR.2014.211 http://dx.doi.org/10.1109/ICPR.2014.211 ]

Gan J Y, Li S L, Zhai Y K and Liu C Y. 2017. 3D convolutional neural network based on face anti-spoofing//Proceedings of the 2nd International Conference on Multimedia and Image Processing (ICMIP). Wuhan, China: IEEE: 1-5[ DOI: 10.1109/ICMIP.2017.9 http://dx.doi.org/10.1109/ICMIP.2017.9 ]

He K M, Zhang X Y, Ren S Q and Sun J. 2014. Spatial pyramid pooling in deep convolutional networks for visual recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 37(9):1904-1916[DOI:10.1109/TPAMI.2015.2389824]

Jourabloo A, Liu Y J and Liu X M. 2018. Face de-spoofing: anti-spoofing via noise modeling//Proceedings of the 15th European Conference Computer Vision ECCV 2018. Munich, Germany: Springer: 297-315[ DOI: 10.1007/978-3-030-01261-8_18 http://dx.doi.org/10.1007/978-3-030-01261-8_18 ]

Karthik K and Katika B R. 2017. Image quality assessment based outlier detection for face anti-spoofing//Proceedings of the 2nd International Conference on Communication Systems, Computing and IT Applications (CSCITA). Mumbai, India: IEEE: 72-77[ DOI: 10.1109/CSCITA.2017.8066527 http://dx.doi.org/10.1109/CSCITA.2017.8066527 ]

Kollreider K, Fronthaler H, Faraj M I and Bigun J. 2007. Real-time face detection and motion analysis with application in "Liveness" assessment. IEEE Transactions on Information Forensics and Security, 2(3):548-558[DOI:10.1109/TIFS.2007.902037]

Li L, Feng X Y, Boulkenafet Z, Xia Z Q, Li M M and Hadid A. 2016. An original face anti-spoofing approach using partial convolutional neural network//Proceedings of the 6th International Conference on Image Processing Theory, Tools and Applications (IPTA). Oulu, Finland: IEEE: 1-6[ DOI: 10.1109/IPTA.2016.7821013 http://dx.doi.org/10.1109/IPTA.2016.7821013 ]

Lin M, Chen Q and Yan S C. 2014. Network in network[EB/OL].[2019-08-15] . https://arxiv.org/pdf/1312.4400.pdf https://arxiv.org/pdf/1312.4400.pdf

Liu Y J, Jourabloo A and Liu X M. 2018. Learning deep models for face anti-spoofing: binary or auxiliary supervision//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, USA: IEEE: 389-398[ DOI: 10.1109/CVPR.2018.00048 http://dx.doi.org/10.1109/CVPR.2018.00048 ]

Liu Y J, Stehouwer J, Jourabloo A and Liu X M. 2019. Deep tree learning for zero-shot face anti-spoofing[EB/OL].[2019-08-15] . https://arxiv.org/pdf/1904.02860.pdf https://arxiv.org/pdf/1904.02860.pdf

Lucas B D and Kanade T. 1997. An iterative image registration technique with an application to stereo vision//Proceedings of the 7th International Joint Conference on Artificial Intelligence. San Francisco: ACM: 674-679

Lucena O, Junior A, Moia V, Souza R, Valle E and Lotufo R. 2017. Transfer learning using convolutional neural networks for face anti-spoofing//Proceedings of the 14th International Conference on Image Analysis and Recognition. Montreal, QC, Canada: Springer: 27-34[ DOI: 10.1007/978-3-319-59876-5_4 http://dx.doi.org/10.1007/978-3-319-59876-5_4 ]

Nikisins O, Mohammadi A, Anjos A and Marcel S. 2018. On effectiveness of anomaly detection approaches against unseen presentation attacks in face anti-spoofing//Proceedings of 2018 International Conference on Biometrics (ICB). Gold Coast, QLD: IEEE: 75-81[ DOI: 10.1109/ICB2018.2018.00022 http://dx.doi.org/10.1109/ICB2018.2018.00022 ]

Pan G, Sun L, Wu Z H and Lao S H. 2007. Eyeblink-based anti-spoofing in face recognition from a generic webcamera//Proceedings of the 11th IEEE International Conference on Computer Vision. Rio de Janeiro: Brazil: IEEE: 1-8[ DOI: 10.1109/ICCV.2007.4409068 http://dx.doi.org/10.1109/ICCV.2007.4409068 ]

Patel K, Han H and Jain A K. 2016. Secure face unlock:spoof detection on smartphones. IEEE Transactions on Information Forensics and Security, 11(10):2268-2283[DOI:10.1109/TIFS.2016.2578288]

Pérez-Cabo D, Jiménez-Cabello D and Costa-Pazo A and López-Sastre R J. 2019. Deep anomaly detection for generalized face anti-spoofing[EB/OL].[2019-08-15] . https://arxiv.org/pdf/1904.08241.pdf https://arxiv.org/pdf/1904.08241.pdf

Rehman Y A U, Po L M and Liu M Y. 2018. LiveNet:improving features generalization for face liveness detection using convolution neural networks. Expert Systems with Applications, 108:159-169[DOI:10.1016/j.eswa.2018.05.004]

Schwartz W R, Rocha A and Pedrini H. 2011. Face spoofing detection through partial least squares and low-level descriptors//Proceedings of 2011 International Joint Conference on Biometrics (IJCB). Washington DC, USA: IEEE: 1-8[ DOI: 10.1109/IJCB.2011.6117592 http://dx.doi.org/10.1109/IJCB.2011.6117592 ]

Taigman Y, Yang M, Ranzato M A and Wolf L. 2014. DeepFace: closing the gap to human-level performance in face verification//Proceedings of 2014 IEEE Conference on Computer Vision and Pattern Recognition. Columbus, OH: IEEE: 1701-1708[ DOI: 10.1109/CVPR.2014.220 http://dx.doi.org/10.1109/CVPR.2014.220 ]

Tan X Y, Li Y, Liu J and Jiang L. 2010. Face liveness detection from a single image with sparse low rank bilinear discriminative model//Proceedings of the 11th European Conference on Computer Vision. Heraklion, Crete, Greece: Springer: 504-517[ DOI: 10.1007/978-3-642-15567-3_37 http://dx.doi.org/10.1007/978-3-642-15567-3_37 ]

Wen D, Han H and Jain A K. 2015. Face spoof detection with image distortion analysis. IEEE Transactions on Information Forensics and Security, 10(4): 746-761[DOI: 10.1109/tifs.2015.2400395 http://dx.doi.org/10.1109/tifs.2015.2400395 ]

Xu Z Q, Li S and Deng W H. 2015. Learning temporal features using LSTM-CNN architecture for face anti-spoofing//Proceedings of the 3rd IAPR Asian Conference on Pattern Recognition (ACPR). Kuala Lumpur, Malaysia: IEEE: 141-145[ DOI: 10.1109/ACPR.2015.7486482 http://dx.doi.org/10.1109/ACPR.2015.7486482 ]

Yang J W, Lei Z and Li S Z. 2014. Learn convolutional neural network for face anti-spoofing[EB/OL].[2019-08-15] . https://arxiv.org/pdf/1408.5601.pdf https://arxiv.org/pdf/1408.5601.pdf

Zhang K P, Zhang Z P, Li Z F and Qiao Y. 2016. Joint face detection and alignment using multitask cascaded convolutional networks. IEEE Signal Processing Letters, 23(10):1499-1503[DOI:10.1109/lsp.2016.2603342]

Zhang P, Zou F H, Wu Z W, Dai N L, Mark S, Fu M, Zhao J and Li K. 2019. FeatherNets: convolutional neural networks as light as feather for face anti-spoofing[EB/OL].[2019-08-15] . https://arxiv.org/pdf/1904.09290.pdf https://arxiv.org/pdf/1904.09290.pdf

Zhang Z W, Yan J J, Liu S F, Lei Z, Yi D and Li S A. 2012. A face antispoofing database with diverse attacks//Proceedings of the 5th IAPR International Conference on Biometrics (ICB). New Delhi, India: IEEE: 26-31[ DOI: 10.1109/ICB.2012.6199754 http://dx.doi.org/10.1109/ICB.2012.6199754 ]

Zhou B L, Khosla A, Lapedriza A, Oliva A and Torralba A. 2016. Learning deep features for discriminative localization//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas, NV, USA: IEEE: 2921-2929[ DOI: 10.1109/CVPR.2016.319 http://dx.doi.org/10.1109/CVPR.2016.319 ]