浏览全部资源
扫码关注微信
Received：12 July 2018，
Revised：2018-8-10，
Published：16 February 2019
移动端阅览
目的
2
人脸2维图像反映出来的纹理并非是3维人脸曲面真实的纹理，并且受光照和妆容的影响很大，因此探索3维局部纹理特征对于人脸识别任务有着重要的意义。为此详细分析了一种新颖的3维局部纹理特征mesh-LBP对于人脸纹理的描述能力。
方法
2
首先，在特征提取和识别任务之前，进行一系列的预处理：人脸分割、离群点移除和孔洞填补；接着，在预处理后的人脸曲面上，提取原始mesh-LBP特征，以及基于阈值化策略的3种改进特征：mesh-tLBP、mesh-MBP和mesh-LTP；然后，对于上述提取的4种特征，采用不同的统计方法，包括整体直方图、局部分块直方图和整体编码图像，用做人脸纹理的特征描述。最后，针对CASIA3D数据集中不同表情和姿态变化的人脸，采用余弦相似度进行人脸的识别任务。
结果
2
通过对比人脸曲面和普通物体曲面的纹理特征，发现人脸纹理完全不同于普通纹理，不规则并且难以描述；通过对比mesh-LBP两种变体，发现mesh-LBP（
$$α_1$$
）适用于姿态变化，而mesh-LBP（
$$α_2$$
）适用于表情变化；通过对比原始mesh-LBP及其3种改进，发现mesh-tLBP对于人脸不同表情变化下的识别准确率最高有0.5%的提升；通过对比3种不同的统计方法，发现采用整体编码图像进行统计的特征尽管弱于局部分块直方图，但相比整体直方图，识别率在不同表情变化下最高有46.8%的提升。
结论
2
mesh-LBP特征是一种优良的3维局部纹理特征，未来将会在3维医学处理、3维地形起伏检测以及3维人脸识别中得到更多的应用。
Objective
2
The texture reflected by 2D facial image is different for a 3D face surface
and this 2D texture is considerably affected by the variations of illumination and make-up. These issues make the investigation on 3D local texture features important for face recognition tasks. The concept of 3D texture is completely different from 2D texture
which reflects the repeatable patterns of a 3D facial surface. Aside from the geometric information
3D texture preserves the photometric information of the same individual due to the flexibility of 3D mesh. Therefore
two original 3D textures
namely
3D geometric texture and 3D photometric texture
should be investigated.
Method
2
In this study
we investigate a novel framework called mesh-LBP in representing 3D facial texture in detail. Here
we mainly focus on the improvement and statistic of this operator rather than the comparisons on final face recognition rate with state-of-the-art methods. First
a set of general preprocessing operations
including face detection
outlier removal
and hole filling
are performed before feature extraction and classification because raw 3D facial data contain spikes and holes and a large background area. Specifically
a facial surface is initially cropped by using a common scheme
that is
the point sets of a raw face model located on a sphere that are constructed by nose tip and fixed radius
are extracted as the detected facial area. Then
we define the outlier of raw data as the point whose number of neighborhood points are lower than that of a threshold. A mean filter is used to smooth the facial surface when these outliers are detected. The outlier removal operation usually results in holes in 3D facial data. Thus
we adopt bicubic interpolation to solve this problem. Second
the construction procedure of original mesh-LBP operator and three improved operators based on thresholding scheme
which we called mesh-tLBP
mesh-MBP
and mesh-LTP
are developed. For the mesh-tLBP
a small threshold is added to the calculation process of the mesh-LBP. For the mesh-MBP
the value of a center facet on the mesh is replaced by the mean value of its neighborhood. For the mesh-LTP
an additional coding unit is added for the subtle capture of code changes of the mesh-LBP. The first two improvements are designed for the robustness of the mesh-LBP to noise or face changes
whereas the last one improves the power of the mesh-LBP in capturing facial details. Third
different statistical methods
including naïve holistic histogram
spatially enhanced histogram
and holistic coded image
are employed to form the final facial representation. For the naïve holistic histogram
we do not use any processing method and directly perform frequency statistics on the calculated LBP pattern. For the spatially enhanced histogram
we initially block a 3D facial surface
perform frequency statistics for each block
and concatenate them to form the entire description of the face. For the holistic coded image
we directly use the calculated LBP pattern. However
the number of patterns from different faces is different; thus
we initially normalize them to the same size. Finally
we employ 615 neutral scans under different illumination condition from CASIA3D face database as the training set and evaluate the recognition performance on 615 scans of expression variation and 1 230 scans of pose on the basis of a simple minimum distance classifier.
Result
2
Comparison of the texture features of facial surface and common object surface show that the facial texture is completely different from ordinary texture and is irregular and difficult to describe. In addition
the texture variations of 3D faces are smaller than that of 2D faces
and this finding shows the superiority of 3D data. Experiments on the two variants of mesh-LBP show that the mesh-LBP(
$$α_1$$
) is more robust to pose variations
whereas the mesh-LBP(
$$α_2$$
) is more robust to express variations. Experiments on the two variants of mesh-LBP and its three improvements indicate that only mesh-tLBP causes a 0.5% improvement of recognition accuracy on different facial expression variations in the best case. Results of the mesh-LTP are basically the same as those of the mesh-LBP
whereas the results of the mesh-MBP are worse than those of the mesh-LBP. These improvements do not enhance the representation of 3D facial textures
and pre-processing and parameter selection schemes should be conducted to obtain improved results. Comparison of the results of the three statistical methods show that features based on the spatially enhanced histogram obtain the best recognition in two experimental scenarios. The description power of features based on holistic coded image is weaker than that of the spatially enhanced histogram; however
its recognition rate is increased by 46.8% compared with the features based on the naive holistic histogram on different expression variations in the best case. In addition
the results of features based on the holistic coded image on pose variations are the worst among all the statistical methods
which is mainly due to the limitation of the image.
Conclusion
2
In comparison with other 3D local feature descriptors
the mesh-LBP is an elegant and efficient framework that allows the direct extraction of 3D local textures from a mesh manifold. The calculated patterns of mesh-LBP can use different statistical methods for the 3D texture analysis of different types of object. For example
the simple mesh-hLBPH is suitable for ordinary 3D objects
whereas the mesh-eLBPH is applicable for 3D face analysis. The mesh-LBP can be used extensively in 3D medical imaging
3D terrain relief inspection
and 3D face recognition in the near future. Several aspects will be investigated in our next work. First
the fusion of 3D geometry and 3D photometric appearance based on the mesh-LBP framework will be evaluated to improve recognition. Second
the size of the mesh-LBP(
$$α_2$$
) will be optimized
and its discrimination power will be increased. Third
other schemes of 2D LBP
such as neighborhood topology and sampling to mesh-LBP for different applications of 3D texture
will be extended. Finally
the integration of the mesh-LBP with a robust matching algorithm will be investigated.
Elaiwat S, Bennamoun M, Boussaid F, et al. A curvelet-based approach for textured 3D face recognition[J]. Pattern Recognition, 2015, 48(4):1235-1246.[DOI:10.1016/j.patcog.2014.10.013]
Li H B, Huang D, Morvan J M, et al. Towards 3D face recognition in the real:a registration-free approach using fine-grained matching of 3D keypoint descriptors[J]. International Journal of Computer Vision, 2015, 113(2):128-142.[DOI:10.1007/s11263-014-0785-6]
Lei Y J, Guo Y L, Hayat M, et al. A two-phase weighted collaborative representation for 3D partial face recognition with single sample[J]. Pattern Recognition, 2016, 52:218-237.[DOI:10.1016/j.patcog.2015.09.035]
Yu X, Gao Y S, Zhou J. 3D face recognition under partial occlusions using radial strings[C]//Proceedings of 2016 IEEE International Conference on Image Processing. Phoenix, AZ, USA: IEEE, 2016: 3016-3020.[ DOI: 10.1109/ICIP.2016.7532913 http://dx.doi.org/10.1109/ICIP.2016.7532913 ]
Yu X, Gao Y S, Zhou J. Sparse 3D directional vertices vs continuous 3D curves:efficient 3D surface matching and its application for single model face recognition[J]. Pattern Recognition, 2017, 65:296-306.[DOI:10.1016/j.patcog.2016.12.009]
Emambakhsh M, Evans A. Nasal patches and curves for expression-robust3D face recognition[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(5):995-1007.[DOI:10.1109/TPAMI.2016.2565473]
Li Y, Wang Y H, Liu J, et al. Expression-insensitive 3D face recognition by the fusion of multiple subject-specific curves[J]. Neurocomputing, 2018, 275:1295-1307.[DOI:10.1016/j.neucom.2017.09.070]
Huang D, Ardabilian M, Wang Y H, et al. 3-D face recognition using eLBP-based facial description and local feature hybrid matching[J]. IEEE Transactions on Information Forensics and Security, 2012, 7(5):1551-1565.[DOI:10.1109/TIFS.2012.2206807]
Soltanpour S, Wu J Q M. High-order local normal derivative pattern (LNDP) for 3D face recognition[C]//Proceedings of 2017 IEEE International Conference on Image Processing. Beijing, China: IEEE, 2017.[ DOI: 10.1109/ICIP.2017.8296795 http://dx.doi.org/10.1109/ICIP.2017.8296795 ]
Fehr J, Burkhardt H. 3D rotation invariant local binary patterns[C]//Proceedings of the 19th International Conference on Pattern Recognition. Tampa, FL, USA: IEEE, 2008: 1-4.[ DOI: 10.1109/ICPR.2008.4761098 http://dx.doi.org/10.1109/ICPR.2008.4761098 ]
Tang H L, YinB C, Sun Y F, et al. 3D face recognition using local binary patterns[J]. Signal Processing, 2013, 93(8):2190-2198.[DOI:10.1016/j.sigpro.2012.04.002]
Werghi N, Berretti S, del Bimbo A. The Mesh-LBP:a framework for extracting local binary patterns from discrete manifolds[J]. IEEE Transactions on Image Processing, 2015, 24(1):220-235.[DOI:10.1109/TIP.2014.2370253]
Yang B, Chen S C. A comparative study on local binary pattern (LBP) based face recognition:LBP histogram versus LBP image[J]. Neurocomputing, 2013, 120:365-379.[DOI:10.1016/j.neucom.2012.10.032]
Ojala T, Pietikainen M, Maenpaa T. Multiresolution gray-scale and rotation invariant texture classification with local binary patterns[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2002, 24(7):971-987.[DOI:10.1109/TPAMI.2002.1017623]
Huang X S, Li S Z, Wang Y S. Shape localization based on statistical method using extended local binary pattern[C]//Proceedings of the Third International Conference on Image and Graphics. Hong Kong, China: IEEE, 2004.[ DOI: 10.1109/ICIG.2004.127 http://dx.doi.org/10.1109/ICIG.2004.127 ]
Hafiane A, Seetharaman G, Zavidovique B. Median binary pattern for textures classification[C]//Proceedings of 2007 International Conference Image Analysis and Recognition. Berlin, Heidelberg: Springer, 2007: 387-398.[ DOI: 10.1007/978-3-540-74260-9_35 http://dx.doi.org/10.1007/978-3-540-74260-9_35 ]
Tan X Y, Triggs B. Enhanced local texture feature sets for face recognition under difficult lighting conditions[J]. IEEE Transactions on Image Processing, 2010, 19(6):1635-1650.[DOI:10.1109/TIP.2010.2042645]
Werghi N, Tortorici C, Berretti S, et al. Boosting 3D LBP-based face recognition by fusing shape and texture descriptors on the mesh[J]. IEEE Transactions on Information Forensics and Security, 2016, 11(5):964-979.[DOI:10.1109/TIFS.2016.2515505]
Lv S W. Research of 3D face recognition based on extended LBP features[D]. Nanjing: Southeast University, 2016. http://www.wanfangdata.com.cn/details/detail.do?_type=degree&id=Y3142873 .
吕士文.基于改进LBP特征的三维人脸识别方法研究[D].南京: 东南大学, 2016.
Smeets D, Keustermans J, Vandermeulen D, et al. meshSIFT:Local surface features for 3D face recognition under expression variations and partial data[J]. Computer Vision and Image Understanding, 2013, 117(2):158-169.[DOI:10.1016/j.cviu.2012.10.002]
Berretti S, Werghi N, del Bimbo A, et al. Selecting stable keypoints and local descriptors for person identification using 3D face scans[J]. The Visual Computer, 2014, 30(11):1275-1292.[DOI:10.1007/s00371-014-0932-7]
Pietikäinen M, Zhao G Y. Two decades of local binary patterns: A survey[M]//Bingham E, Kaski S, Laaksonen J, et al. Advances in Independent Component Analysis and Learning Machines. London: Academic Press, 2015: 175-210.[ DOI: 10.1016/B978-0-12-802806-3.00009-9 http://dx.doi.org/10.1016/B978-0-12-802806-3.00009-9 ]
Tang L L, Cai S Y, Da F P, et al. 3D Face recognition method based on the local binary pattern from vertical and horizontal on the mesh[J]. Chinese Journal of Scientific Instrument, 2016, 37(6):1413-1420.
汤兰兰, 盖绍彦, 达飞鹏, 等.基于网格纵横局部二值模式的三维人脸识别[J].仪器仪表学报, 2016, 37(6):1413-1420. [DOI:10.3969/j.issn.0254-3087.2016.06.027]
相关文章
相关作者
相关机构
京公网安备11010802024621
