Occlusion expression variation face recognition based on auxiliary dictionary and low rank decomposition

Xiaofeng Fu; Yu Zhang; Jun Wu

doi:10.11834/jig.170407

Column of CACIS 2017 | Views : 0 下载量: 4 CSCD: 4

PDF
Export
Share
Collection
Album

Occlusion expression variation face recognition based on auxiliary dictionary and low rank decomposition
Vol. 23, Issue 3, Pages: 399-409(2018)
Received：07 July 2017，

Revised：2017-9-14，

Published：16 March 2018
DOI： 10.11834/jig.170407
稿件说明：

移动端阅览

DOI：

Xiaofeng Fu, Yu Zhang, Jun Wu. Occlusion expression variation face recognition based on auxiliary dictionary and low rank decomposition[J]. Journal of Image and Graphics, 2018, 23(3): 399-409. DOI： 10.11834/jig.170407.

摘要

目的

从真实环境中采集到的人脸图片通常伴随遮挡、光照和表情变化等因素，对识别结果产生干扰。在许多特殊环境下，训练样本的采集数量也无法得到保证，容易产生训练样本远小于测试样本的不利条件。基于以上情况，如何排除复杂的环境变化和较少的训练样本等多重因素对识别效果的影响逐渐成为了人脸识别方向需要攻克的难题。

方法

以低秩矩阵分解为基础，分别使用非凸秩近似范数和核范数进行两次低秩矩阵分解，以达到去除遮挡干扰的

目的

。首先通过非凸稳健主成分分析分解得到去除了光照、遮挡等变化的低秩字典。为消除不同人脸类的五官等共通部分的影响，加快算法收敛效率，将得到的低秩字典用作初始化，进行基于核范数的第二次秩近似分解，以获得去除了类间不相关判别性的低秩字典用于分类。最后针对训练样本较少和遮挡样本占比过大等问题，选用同一数据库中不用做训练和测试的辅助数据作为辅助字典模拟可能出现的遮挡、光照等影响，通过最小化稀疏表示重构误差进行分类识别。

结果

选用AR库和CK+库分别进行实验。在AR库的实验中，通过调整训练图片中遮挡、光照和表情变化的样本比例来检测算法性能。其中，在遮挡图片占比分别为1/7和3/7的训练集中，无遮挡图片由无干扰和光照表情干扰图片联合组成。在遮挡图片占比为2/7的训练集中，无遮挡图片全由光照表情变化图片组成。实验结果表明，在多种实验情况下均获得较高识别率。其中根据不同遮挡比例，分别获得97.75%、92%、95.25%和97.75%、90%、95.25%等识别率。与同类算法对比提高3%~5%。选用的外部数据从10类人脸至40类依次增加，获得的识别结果为96.75%~98%，与同类算法相比提高了2%~3%。在CK+表情库中，选用同伦算法配合分类求解，获得的识别结果为95.25%。

结论

本文提出了一种在克服复杂环境变化和训练样本不足两个方面具有高效性和鲁棒性的人脸识别算法，实验结果表明，本文算法在不同数据库中都具有高效性，未来的研究方向包括将算法应用于联立人脸和表情识别，模拟更为复杂的噪声状况，以期达到更为优异的结果。

Abstract

Objective

Face recognition has an important role in our daily life. Applications are increasingly using face recognition as a useful interactive function. Real-life environments often introduce complex effects on data collection. Thus

face recognition always encounters challenges from several variations and occlusions. Face images collected from the real-world environment are usually accompanied with factors

such as occlusion

illumination

and expression changes

that interfere with recognition results. The number of collected training samples cannot be guaranteed in many special circumstances. Sometimes

the number of training samples is much smaller than the number of test samples. This paper aims to address difficult face recognition problems

such as eliminating complex environmental changes

undersampled training sets

and other recognition factors.

Method

This paper proposes an efficient face recognition algorithm under occlusion and the expressed variable

which is based on low rank approximation decomposition and auxiliary dictionary learning. It utilizes non-convex rank approximation norm and nuclear norm to perform matrix decomposition twice. The algorithm efficiently eliminates gross sparse errors on occlusion and other factors. First

we obtain the initial low-rank dictionary without illumination and occluded variation based on the non-convex robust principal component analysis algorithm to increase the convergence efficiency of the algorithm. Second

we perform the second low-rank decomposition based on a nuclear norm to obtain discriminant and incoherent low-rank dictionary and eliminate the influence of common facial parts through different classes. Finally

we choose the auxiliary data from the same database as an auxiliary dictionary that can simulate possible interference to overcome the problem of insufficient training samples and the large number of occlusion samples. The testing samples can be classified by minimizing the reconstruction error. This study considered the variation of noise

occlusion

illumination

and expression in the natural environment. The proposed efficient algorithm extracts low-complexity components that correspond to facial attributes

which are mutually incoherent among different classes (e.g.

identity and expression) from training data

even in the presence of gross sparse errors. The lack of training data

joint low-rank dictionary

and auxiliary dictionary is addressed

and the weighted reconstruction model is established to complete the classification.

Result

We test our algorithm with the AR and CK+ databases. Experiments on both databases show that the proposed algorithm can achieve outstanding recognition accuracy. We divided the experiments on the AR database into three different training scales. We choose different numbers of images occluded by a scarf or sunglasses with an occlusion percent of 20%-40%. The number of occluded images that contain a scarf or sunglasses increased from one to three

and the unoccluded images decreased from six to four. The training set on each person consists of seven images. With one or three occluded images in the training set

the uncovered images consist of natural images with no extra interface factors

such as expression and illumination. However

when the proportion of the covered images is two in seven

the other five uncovered images contain expression and illumination variables. Experimental results demonstrate that the proposed algorithm has a higher recognition rate in a variety of experimental cases. According to the cover types

such as scarves and sunglasses

the recognition rates are 97.75%

92%

95.25%

97.75%

90%

and 95.25%. Compared with the recognition accuracy of recent algorithms

such as sparse representation classification

robust principal component analysis (RPCA)

and non-convex RPCA

the recognition accuracy of the proposed method improved by 3% to 5%. The recognition results are 96.75%-98% when the amount of selected external data increased from 10 to 40. The recognition rates of the proposed method's recognition rates are 2%-3% higher than those of the compared algorithms.

Conclusion

The algorithm proposed in this paper can overcome the complex environmental variation and insufficient training sample problems effectively and robustly. We obtained state-of-the-art performance after testing on different databases. This study attempts to deal with data with multiple occlusions. The proposed algorithm aims to adapt to complex interference factors and achieves high recognition accuracy. However

this algorithm does not effectively deal with joint expression and face recognition. Thus

further research is needed to understand figure joint expression and face recognition.

关键词

Keywords

references

Turk M A, Pentland A P. Face recognition using eigenfaces[C]//Proceedings of 1991 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Maui, HI:IEEE, 1991:586-591.[ DOI:10.1109/CVPR.1991.139758 http://dx.doi.org/10.1109/CVPR.1991.139758 ]

Belhumeur P N, Hespanha J P, Kriegman D J. Eigenfaces vs. Fisherfaces:recognition using class specific linear projection[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1997, 19(7):711-720.[DOI:10.1109/34.598228]

He X F, Yan S C, Hu Y X, et al. Facerecognition using laplacianfaces[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2005, 27(3):328-340.[DOI:10.1109/TPAMI.2005.55]

Shan C F, Gong S G, McOwan P W. Robust facial expression recognition using local binary patterns[C]//Proceedings of IEEE International Conference on Image Processing. Genova, Italy:IEEE, 2005:Ⅱ-370-3.[ DOI:10.1109/ICIP.2005.1530069 http://dx.doi.org/10.1109/ICIP.2005.1530069 ]

Jiang B H, Valstar M, Martinez B, et al. A dynamic appearance descriptor approach to facial actions temporal modeling[J]. IEEE Transactions on Cybernetics, 2014, 44(2):161-174.[DOI:10.1109/TCYB.2013.2249063]

Li L S, Ying Z L, Yang T R. Facial expression recognition by fusion ofgabor texture features and local phase quantization[C]//Proceedings of the 201412th International Conference on Signal Processing. Hangzhou, China:IEEE, 2015:1781-1784.[ DOI:10.1109/ICOSP.2014.7015300 http://dx.doi.org/10.1109/ICOSP.2014.7015300 ]

Lazebnik S, Schmid C, Ponce J. Beyond bags of features:spatial pyramid matching for recognizing natural scene categories[C]//Proceedings of 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. New York, NY, USA:IEEE, 2006:2169-2178.[ DOI:10.1109/CVPR.2006.68 http://dx.doi.org/10.1109/CVPR.2006.68 ]

Candès E J, Li X D, Ma Y, etal. Robust principal component analysis?[J]. Journal of the ACM (JACM), 2011, 58(3):11.[DOI:10.1145/1970392.1970395]

Kang Z, Peng C, Cheng Q.Robust PCA via nonconvex rank approximation[C]//Proceedings of 2015 IEEE International Conference on Data Mining. Atlantic City, NJ, USA:IEEE, 2015:211-220.[ DOI:10.1109/ICDM.2015.15 http://dx.doi.org/10.1109/ICDM.2015.15 ]

Georgakis C, Panagakis Y, Pantic M. Discriminant incoherent component analysis[J]. IEEE Transactions on Image Processing, 2016, 25(5):2021-2034.[DOI:10.1109/TIP.2016.2539502]

Jolliffe I T. Principal component analysis and factor analysis[M]//Principal Component Analysis. New York, NY:Springer New York, 1986:115-128.[DOI:10.1007/978-1-4757-1904-8_7]

Wright J, Yang A Y, Ganesh A, et al. Robust face recognition via sparserepresentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2009, 31(2):210-227.[DOI:10.1109/TPAMI.2008.79]

Deng W H, Hu J N, Guo J. Extended SRC:undersampled face recognition via intraclass variant dictionary[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2012, 34(9):1864-1870.[DOI:10.1109/TPAMI.2012.30]

Bertsekas D P, Rheinboldt W. Constrained Optimization and Lagrange Multiplier Methods[M]. New York:Academic press, 2014.

Lin Z C, Chen MM, Ma Y. The augmented lagrange multiplier method for exact recovery of corrupted low-rank matrices[J]. Eprint Arxiv, 2010, 09

Wei C P, Chen C F, Wang Y C F. Robust face recognition with structurally incoherent low-rank matrix decomposition[J]. IEEE Transactions on Image Processing, 2014, 23(8):3294-3307.[DOI:10.1109/TIP.2014.2329451]

Cai J F, Candès E J, Shen Z W. A singular value thresholding algorithm for matrix completion[J]. Siam Journal on Optimization, 2010, 20(4):1956-1982.[DOI:10.1137/080738970]

Zou H, Hastie T, Tibshirani R. Sparse principal component analysis[M]//Everitt B, Hothorn T. An Introduction to Applied Multivariate Analysis with R. New York:Springer, 2006:231-235.[ DOI:10.1007/978-1-4419-9650-3 http://dx.doi.org/10.1007/978-1-4419-9650-3 ]

Wei C P, Wang Y C F.Undersampled face recognition via robust auxiliary dictionary learning[J]. IEEE Transactions on Image Processing, 2015, 24(6):1722-1734.[DOI:10.1109/TIP.2015.2409738]

Martínez A M, Benavente R. The AR face database[R]. CVC Technical Report 24. Barcelona, Spain:Computer Vision Center, 1998.

Lucey P, Cohn J F, Kanade T, et al. The Extended Cohn-Kanade Dataset (CK+):A complete dataset for action unit and emotion-specified expression[C]//Proceedings of 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition-Workshops. San Francisco, CA, USA:IEEE, 2010:94-101.[ DOI:10.1109/CVPRW.2010.5543262 http://dx.doi.org/10.1109/CVPRW.2010.5543262 ]

YangA Y, Sastry S S, Ganesh A, et al. Fastℓ1-minimization algorithms and an application in robust face recognition:A review[C]//Proceedings of 2010 IEEE International Conference on Image Processing. Hong Kong, China:IEEE, 2010:1849-1852.[ DOI:10.1109/ICIP.2010.5651522 http://dx.doi.org/10.1109/ICIP.2010.5651522 ]

Alert me when the article has been cited

提交

Face recognition with superposed linear sparse representation based on discriminative nonconvex low-rank matrix decomposition

Structured low-rank dictionary learning for face recognition

Human facial texture and fore-background differences-relevant anti-spoofing detection

Hyperspectral and multispectral image fusion focused on error compensation

Mask-fused human face image quality assessment method