浏览全部资源
扫码关注微信
纸质出版日期： 2023-03-16 ，
录用日期： 2022-08-23
移动端阅览
朱建秋, 华阳, 宋晓宁, 吴小俊, 冯振华. 结合面部纹理和前/背景差异的人脸活体检测[J]. 中国图象图形学报, 2023,28(3):789-803.
Jianqiu Zhu, Yang Hua, Xiaoning Song, Xiaojun Wu, Zhenhua Feng. Human facial texture and fore-background differences-relevant anti-spoofing detection[J]. Journal of Image and Graphics, 2023,28(3):789-803.
目的
2
人脸活体检测是人脸识别技术安全的重要保障，而背景环境的负面干扰是限制检测系统性能的关键问题。针对这一问题，提出了一种基于面部纹理和前/背景差异分析的人脸活体检测模型。
方法
2
首先，基于新型骨干网络构建面部纹理分析模块，为模型提供充足的面部纹理信息，减缓模型受背景差异的负面干扰；接着，结合边缘检测算法思想对前/背景差异分析模块的卷积核重新改写，以此突出面部边缘的细节特征；最后，设计注意力特征融合模块将两个分流模块有机结合，提升模型在各种复杂环境下的可靠性。
结果
2
在CASIA-MFSD(CASIA-mobile face spoofing dataset)、Replay-Attack和OULU-NPU等3个数据集上做了多组对比实验，在CASIA-MFSD数据集上的等错误率(equal error rate，EER)为0.19%，在Replay-Attack数据集上的等错误率和半错误率(half total error rate，HTER)均为0.00%和0.00%，在OULU-NPU数据集4个协议上的平均分类错误率(average classification error rate，ACER)分别达到0.6%、1.9%、1.9±1.2%和3.7±1.1%。
结论
2
在人脸反欺诈模型中融合前/背景差异信息和面部纹理信息，可有效减缓背景环境带来的负面干扰，提升模型的鲁棒性。所提模型在背景复杂、欺诈攻击方式尖锐的数据集上均取得较低的错误率，较同类先进算法具有明显优势。
Objective
2
Human-relevant face recognition is often vulnerable for a spoofing attacks-involved system. In this scenario
face information-verified is projected for that. To optimize face recognition system
face anti-spoofing (FAS) has been concerned in recent years. It can provide a security barrier for face recognition systems in practice. Conventional FAS methods are still restricted by handcrafted features and shallow classifiers
and end-to-end trained deep neural networks. Specific attacks are dealt with only because they are sensitive and vulnerable to facial appearance variations in pose
expression
illumination
makeup
occlusion
etc.. In contrast
deep neural network (DNN) based face anti-spoofing algorithms have the capability in distinguishing real and spoofing faces in unconstrained scenarios. Nevertheless
the existing deep learning based FAS methods also have two problems to be challenged: 1) a universal convolution layer and computational complexity and 2) complex background distractions. Although the diversity of background is beneficial to improve the robustness of a trained deep neural network
the performance may degrade when the background contour and exposure degree become the key for face anti-spoofing. For example
a trained model may rely on background features severely and pay less attention to the facial area
resulting in poor generalization for unclear scenarios.
Method
2
To resolve the issues mentioned above
we develop a novel facial texture information-based face anti-spoofing model and its relevant fore-background difference analysis. The proposed method has two main modules: 1) facial texture analysis (FTA) and 2) fore-background difference analysis (FBDA). First
the FTA is used to extract rich facial information for FAS
and a mask is used to process an input image in terms of face region-activated only. Next
the ConvNeXt is used to extract facial features-masked. It is required to be coordinated with FBDA although FTA-extracted facial information can alleviate background differences-introduced interference effectively. The challenge is that FTA may overfit to the training data due to the lack of background information. Third
to reduce computational complexity of the convolution kernel and eliminate the redundancy information of the extracted features
the convolution kernel of a backbone network is redesigned in terms of the mechanism of edge detection in the FBDA module. Our convolution kernels proposed consists of 1) sobel horizontal (vertical) convolution kernels for detecting horizontal (vertical) edge information and 2) a convex convolution kernel for detecting face contours. To improve the efficiency of a deep network
the proposed kernels have less parameter compared to the universal convolution kernel. Furthermore
the network can capture more fine-grained information of an input image in terms of the proposed convolution kernels-cascaded. Finally
to highlight the potentials of these two modules
an attention fusion module is designed and used to balance the contribution of the extracted facial texture information for spoofing detection. In particular
to improve the reliability and robustness of the proposed method
a multiple scenario-oriented attention fusion module is developed. To validate the proposed method
three kinds of datasets of are used
including CASIA-MFSD dataset
Replay-Attack dataset
and OULU-NPU dataset. In our experiments
comparative analysis is carried out based on the same evaluation metrics of those are 1) equal error rate (EER)
2) half total error rate (HTER)
and 3) average classification error rate (ACER).
Result
2
Our optimization achieves 0.19% in terms of EER on the CASIA-MFSD dataset
0.00% in EER and 0.00% in HTER on the Replay-Attack dataset
and 0.6%
1.9%
1.9±1.2%
and 3.7±1.1% in terms of ACER on the four subsets of the OULU-NPU dataset. To evaluate the performance of our method for unclear environments
the cross-dataset evaluations are employed on CASIA-MFSD and Replay-Attack as well. Trained on CASIA-MFSD
it achieves 17.1% in HTER on the Replay-Attack test set. Trained on Replay-Attack
it achieves 27.4% on the CASIA-MFSD test set in terms of HTER. The comparative analysis is demonstrated that our optimization has its lower error rates on several complex data sets and performs better for unclear environments.
Conclusion
2
A novel face anti-spoofing model is developed using the front/background difference information and facial texture information
which can effectively alleviate the difficulties posed by complex background
resulting in better robustness of a trained deep network. To harness a new strategy for practical applications of face spoofing detection
our algorithm is proposed. It can improve the generalization capability of the trained model under unclear scenarios in accordance with more attention weight of the extracted facial texture features. Limitation the accuracy of our method is still not optimal for the datasets for a single background.
人脸活体检测(FAS)边缘检测纹理特征注意力特征融合人脸识别
face anti-spoofing detection(FAS)edge detectiontexture featureattention feature fusionface recognition
Atoum Y, Liu Y J, Jourabloo A and Liu X M. 2017. Face anti-spoofing using patch and depth-based CNNs//Proceedings of 2017 IEEE International Joint Conference on Biometrics. Denver, USA: IEEE: 319-328 [DOI: 10.1109/BTAS.2017.8272713http://dx.doi.org/10.1109/BTAS.2017.8272713]
Boulkenafet Z, Komulainen J and Hadid A. 2015. Face anti-spoofing based on color texture analysis//Proceedings of 2015 IEEE International Conference on Image Processing. Quebec City, Canada: IEEE: 2636-2640 [DOI: 10.1109/ICIP.2015.7351280http://dx.doi.org/10.1109/ICIP.2015.7351280]
Boulkenafet Z, Komulainen J and Hadid A. 2017a. Face antispoofing using speeded-up robust features and fisher vector encoding. IEEE Signal Processing Letters, 24(2): 141-145 [DOI: 10.1109/LSP.2016.2630740]
Boulkenafet Z, Komulainen J, Li L, Feng X Y and Hadid A. 2017b. OULU-NPU: a mobile face presentation attack database with real-world variations//Proceedings of the 12th IEEE International Conference on Automatic Face and Gesture Recognition. Washington, USA: IEEE: 1-7 [DOI: 10.1109/FG.2017.77http://dx.doi.org/10.1109/FG.2017.77]
Chen B L, Yang W H, Li H L, Wang S Q and Kwong S. 2021. Camera invariant feature learning for generalized face anti-spoofing. IEEE Transactions on Information Forensics and Security, 16: 2477-2492 [DOI: 10.1109/TIFS.2021.3055018]
Chen H N, Chen Y W, Tian X and Jiang R X. 2019a. A cascade face spoofing detector based on face anti-spoofing R-CNN and improved retinex LBP. IEEE Access, 7: 170116-170133 [DOI: 10.1109/access.2019.2955383]
Chen H N, Hu G S, Lei Z, Chen Y W, Robertson N M and Li S Z. 2019b. Attention-based two-stream convolutional networks for face spoofing detection. IEEE Transactions on Information Forensics and Security, 15: 578-593 [DOI: 10.1109/TIFS.2019.2922241]
Chingovska I, Anjos A and Marcel S. 2012. On the effectiveness of local binary patterns in face anti-spoofing//Proceedings of 2012 BIOSIG-International Conference of Biometrics Special Interest Group (BIOSIG). Darmstadt, Germany: IEEE: 1-7
Dai J F, Qi H Z, Xiong Y W, Li Y, Zhang G D, Hu H and Wei Y C. 2017. Deformable convolutional networks//Proceedings of 2017 IEEE International Conference on Computer Vision. Venice, Italy: IEEE: 764-773 [DOI: 10.1109/ICCV.2017.89http://dx.doi.org/10.1109/ICCV.2017.89]
Feng L T, Po L M, Li Y M, Xu X Y, Yuan F, Cheung T C H and Cheung K W. 2016. Integration of image quality and motion cues for face anti-spoofing: a neural network approach. Journal of Visual Communication and Image Representation, 38: 451-160 [DOI: 10.1016/j.jvcir.2016.03.019]
Feng Y, Wu F, Shao X H, Wang F F and Zhou X. 2018. Joint 3D face reconstruction and dense alignment with position map regression network//Proceedings of the 15th European Conference on Computer Vision. Munich, Germany: Springer: 557-574 [DOI: 10.1007/978-3-030-01264-9_33http://dx.doi.org/10.1007/978-3-030-01264-9_33]
Girshick R. 2015. Fast R-CNN//Proceedings of 2015 IEEE International Conference on Computer Vision. Santiago, Chile: IEEE: 1440-1448 [DOI: 10.1109/ICCV.2015.169http://dx.doi.org/10.1109/ICCV.2015.169]
He K M, Zhang X Y, Ren S Q and Sun J. 2016. Deep residual learning for image recognition//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, USA: IEEE: 770-778 [DOI: 10.1109/CVPR.2016.90http://dx.doi.org/10.1109/CVPR.2016.90]
Hendrycks D and Gimpel K. 2016. Gaussian error linear units (GELUS) [EB/OL]. [2022-03-12].https://arxiv.org/pdf/1606.08415v4.pdfhttps://arxiv.org/pdf/1606.08415v4.pdf
Juefei-Xu F, Boddeti V N and Savvides M. 2017. Local binary convolutional neural networks//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, USA: IEEE: 19-28 [DOI: 10.1109/CVPR.2017.456http://dx.doi.org/10.1109/CVPR.2017.456]
Kanopoulos N, Vasanthavada N and Baker R L. 1988. Design of an image edge detection filter using the Sobel operator. IEEE Journal of Solid-State Circuits, 23(2): 358-367 [DOI: 10.1109/4.996]
Komulainen J, Hadid A and Pietiköinen M. 2013. Context based face anti-spoofing//Proceedings of the 6th IEEE International Conference on Biometrics: Theory, Applications and Systems. Arlington, USA: IEEE: 1-8 [DOI: 10.1109/BTAS.2013.6712690http://dx.doi.org/10.1109/BTAS.2013.6712690]
Li L, Feng X Y, Boulkenafet Z, Xia Z Q, Li M M and Hadid A. 2016a. An original face anti-spoofing approach using partial convolutional neural network//Proceedings of the 6th International Conference on Image Processing Theory, Tools and Applications. Oulu, Finland: IEEE: 1-6 [DOI: 10.1109/IPTA.2016.7821013http://dx.doi.org/10.1109/IPTA.2016.7821013]
Li X B, Komulainen J, Zhao G Y, Yuen P C and Pietiköinen M. 2016b. Generalized face anti-spoofing by detecting pulse from face videos//Proceedings of the 23rd International Conference on Pattern Recognition. Cancun, Mexico: IEEE: 4244-4249 [DOI: 10.1109/ICPR.2016.7900300http://dx.doi.org/10.1109/ICPR.2016.7900300]
Liu S B, Zhang K Y, Yao T P, Bi M W, Ding S H, Li J L, Huang F Y and Ma L Z. 2021a. Adaptive normalized representation learning for generalizable face anti-spoofing//Proceedings of the 29th ACM International Conference on Multimedia. Lisboa, Portugal: ACM: 1469-1477 [DOI: 10.1145/3474085.3475279http://dx.doi.org/10.1145/3474085.3475279]
Liu Y C, Shao Z R and Hoffmann N. 2021b. Global attention mechanism: retain information to enhance channel-spatial interactions [EB/OL]. [2022-03-12].https://arxiv.org/pdf/2112.05561.pdfhttps://arxiv.org/pdf/2112.05561.pdf
Liu Y J, Jourabloo A and Liu X M. 2018. learning deep models for face anti-spoofing: binary or auxiliary supervision//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, USA: IEEE: 638-642 [DOI: 10.1109/CVPR.2018.00048http://dx.doi.org/10.1109/CVPR.2018.00048]
Liu Z, Lin Y T, Cao Y, Hu H, Wei Y X, Zhang Z, Lin S and Guo B N. 2021c. Swin transformer: hierarchical vision transformer using shifted windows//Proceedings of 2021 IEEE/CVF International Conference on Computer Vision. Montreal, Canada: IEEE: 10012-10022 [DOI: 10.1109/ICCV48922.2021.00986http://dx.doi.org/10.1109/ICCV48922.2021.00986]
Liu Z, Mao H Z, Wu C Y, Feichtenhofer C, Darrell T and Xie S N. 2022. A ConvNet for the 2020s [EB/OL]. [2022-03-12].https://arxiv.org/pdf/2201.03545.pdfhttps://arxiv.org/pdf/2201.03545.pdf
Milletari F, Navab N and Ahmadi S A. 2016. V-Net: fully convolutional neural networks for volumetric medical image segmentation//Proceedings of the 4th International Conference on 3D Vision. Stanford, USA: IEEE: 565-571 [DOI: 10.1109/3DV.2016.79http://dx.doi.org/10.1109/3DV.2016.79]
Mohammadi A, Bhattacharjee S and Marcel S. 2020. Domain adaptation for generalization of face presentation attack detection in mobile settengs with minimal information//Proceedings of the ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing. Barcelona, Spain: IEEE: 1001-1005 [DOI: 10.1109/ICASSP40776.2020.9053685http://dx.doi.org/10.1109/ICASSP40776.2020.9053685]
Patel K, Han H and Jain A K. 2016. Secure face unlock: spoof detection on smartphones. IEEE Transactions on Information Forensics and Security, 11(10): 2268-2283 [DOI: 10.1109/TIFS.2016.2578288]
Qin Y X, Zhao C X, Zhu X Y, Wang Z Z, Yu Z T, Fu T Y, Zhou F, Shi J P and Lei Z. 2020. Learning meta model for zero-and few-shot face anti-spoofing. Proceedings of the AAAI Conference on Artificial Intelligence, 34(7): 11916-11923 [DOI: 10.1609/aaai.v34i07.6866]
Shao R, Lan X Y, Li J W and Yuen P C. 2019. Multi-adversarial discriminative deep domain generalization for face presentation attack detection//Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach, USA: IEEE: 10023-10031 [DOI: 10.1109/CVPR.2019.01026http://dx.doi.org/10.1109/CVPR.2019.01026]
Wang Y H, Song X N and Wu X J. 2020. Two-stream face spoofing detection network combined with hybrid pooling. Journal of Image and Graphics, 25(7): 1408-1420
汪亚航, 宋晓宁, 吴小俊. 2020. 结合混合池化的双流人脸活体检测网络. 中国图象图形学报, 25(7): 1408-1420 [DOI: 10.11834/jig.190419]
Wang Z Z, Yu Z T, Zhao C X, Zhu X Y, Qin Y X, Zhou Q S, Zhou F and Lei Z. 2020. Deep spatial gradient and temporal depth learning for face anti-spoofing//Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle, USA: IEEE: 5042-5051 [DOI: 10.1109/CVPR42600.2020.00509http://dx.doi.org/10.1109/CVPR42600.2020.00509]
Xie X H, Bian J T and Lai J H. 2022. Review on face liveness detection. Journal of Image and Graphics, 27(1): 63-87
谢晓华, 卞锦堂, 赖剑煌. 2022. 人脸活体检测综述. 中国图象图形学报, 27(1): 63-87 [DOI: 10.11834/jig.210470]
Xu Y W, Wu L F, Jian M, Zheng W S, Ma Y K and Wang Z M. 2021. Identity-constrained noise modeling with metric learning for face anti-spoofing. Neurocomputing, 434: 149-164 [DOI: 10.1016/j.neucom.2020.12.095]
Xu Z Q, Li S and Deng W H. 2015. Learning temporal features using LSTM-CNN architecture for face anti-spoofing//Proceedings of the 3rd IAPR Asian Conference on Pattern Recognition. Kuala Lumpur, Malaysia: IEEE: 141-145 [DOI: 10.1109/ACPR.2015.7486482http://dx.doi.org/10.1109/ACPR.2015.7486482]
Yang J W, Lei Z and Li S Z. 2014. Learn convolutional neural network for face anti-spoofing [EB/OL]. [2022-03-12].https://arxiv.org/pdf/1408.5601.pdfhttps://arxiv.org/pdf/1408.5601.pdf
Yang X, Luo W H, Bao L C, Gao Y, Gong D H, Zheng S B, Li Z F and Liu W. 2019. Face anti-spoofing: model matters, so does data//Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach, USA: IEEE: 3507-3516 [DOI: 10.1109/CVPR.2019.00362http://dx.doi.org/10.1109/CVPR.2019.00362]
Yu F and Koltun V. 2016. Multi-scale context aggregation by dilated convolutions//Proceedings of the 4th International Conference on Learning Representations. San Juan, USA: [s. n.]
Yu Z T, Li X B, Niu X S, Shi J G and Zhao G Y. 2020a. Face anti-spoofing with human material perception//Proceedings of the 16th European Conference on Computer Vision. Glasgow, UK: Springer: 557-575 [DOI: 10.1007/978-3-030-58571-6_33http://dx.doi.org/10.1007/978-3-030-58571-6_33]
Yu Z T, Zhao C X, Wang Z Z, Qin Y X, Su Z, Li X B, Zhou F and Zhao G Y. 2020b. Searching central difference convolutional networks for face anti-spoofing//Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle, USA: IEEE: 5295-5305 [DOI: 10.1109/CVPR42600.2020.00534http://dx.doi.org/10.1109/CVPR42600.2020.00534]
Zhang K Y, Yao T P, Zhang J, Tai Y, Ding S H, Li J L, Huang F Y, SongH C and Ma L Z. 2020. Face anti-spoofing via disentangled representation learning//Proceedings of the 16th European Conference on Computer Vision. Glasgow, UK: Springer: 641-657 [DOI: 10.1007/978-3-030-58529-7_38http://dx.doi.org/10.1007/978-3-030-58529-7_38]
Zhang Z W, Yan J J, Liu S F, Lei Z, Yi D and Li S Z. 2012. A face antispoofing database with diverse attacks//Proceedings of the 5th IAPR International Conference on Biometrics. New Delhi, India: IEEE: 26-31 [DOI: 10.1109/ICB.2012.6199754http://dx.doi.org/10.1109/ICB.2012.6199754]
Zheng W, Yue M Y, Zhao S H and Liu S Q. 2021. Attention-based spatial-temporal multi-scale network for face anti-spoofing. IEEE Transactions on Biometrics, Behavior, and Identity Science, 3(3): 296-307 [DOI: 10.1109/TBIOM.2021.3066983]
Zhou F S, Gao C Q, Chen F, Li C Y, Li X D, Yang F and Zhao Y. 2019. Face anti-spoofing based on multi-layer domain adaptation//Proceedings of 2019 IEEE International Conference on Multimedia and Expo Workshops. Shanghai, China: IEEE: 192-197 [DOI: 10.1109/ICMEW.2019.00-88http://dx.doi.org/10.1109/ICMEW.2019.00-88]
Zhou L F, Luo J, Gao X B, Li W S, Lei B J and Leng J. 2021. Selective domain-invariant feature alignment network for face anti-spoofing. IEEE Transactions on Information Forensics and Security, 16: 5352-5365 [DOI: 10.1109/TIFS.2021.3125603]
Zhu C R and Wang R S. 2012. Local multiple patterns based multiresolution gray-scale and rotation invariant texture classification. Information Sciences, 187: 93-108 [DOI: 0.1016/j.ins.2011.10.014]
相关作者
相关机构
京公网安备11010802024621