Kinship verification combined with deep metric learning

Daoqing Qu; Lin Ma; Zhuhong Shao

doi:10.11834/jig.180004

Image Understanding and Computer Vision | Views : 0 下载量: 12 CSCD: 0

PDF
Export
Share
Collection
Album

Kinship verification combined with deep metric learning
Vol. 23, Issue 10, Pages: 1540-1548(2018)
Received：05 January 2018，

Revised：2018-4-8，

Published：16 October 2018
DOI： 10.11834/jig.180004
稿件说明：

移动端阅览

DOI：

Daoqing Qu, Lin Ma, Zhuhong Shao. Kinship verification combined with deep metric learning[J]. Journal of Image and Graphics, 2018, 23(10): 1540-1548. DOI： 10.11834/jig.180004.

摘要

目的

人脸图像分析是计算机视觉和模式识别领域的重要研究方向之一，基于人脸图像的血缘关系识别是对给定的一对或一组人脸图像，判断其是否存在某种血缘关系。人脸血缘关系识别不仅在生物特征识别领域有着重要研究价值，而且在社交媒体挖掘、失散家庭成员寻找等社会生活领域中有重要的应用价值。针对当前大多数算法都是基于传统机器学习方法，提出一种采用深度度量学习进行人脸图像血缘关系研究的新方法。

方法

目前深度学习算法能很好地理解单张人脸图像，但是多个主体间的关系探究仍然是计算机视觉领域富有挑战性的问题之一。为此，提出一种基于深度度量学习的父母与子女的血缘关系识别方法。首先使用超过5 000 000张人脸图像的样本集训练一个深度卷积神经网络FaceCNN并提取父母与子女的人脸图像深度特征，之后引入判别性度量学习方法，使得具有血缘关系的特征尽可能地靠近，反之则尽可能地远离。然后对特征进行分层非线性变换使其具有更强判别特性。最后根据余弦相似度分别计算父亲、母亲和孩子的相似度并利用相似概率值得到双亲和孩子的综合相似度得分。

结果

算法在TSKinFace数据集上验证了FaceCNN提取特征与深度度量学习结合进行血缘关系识别的有效性，最终在该数据集上父母与儿子和女儿的血缘关系识别准确率分别达到87.71%和89.18%，同时算法在进行血缘度量学习和双亲相似度计算仅需要3.616 s。

结论

提出的血缘关系识别方法，充分利用深度学习网络良好的表征和学习能力，不仅耗时少，而且有效地提高了识别准确率。

Abstract

Objective

Facial image analysis has been an important and active subfield of computer vision and pattern recognition.Kinship verification refers to the tasks of training a machine based on features extracted from facial images to determine the existence

type

and degree of kinship of two or more people.Moreover

kinship verification has extensive and promising applications in many technical and social fields

such as family history research and photo management in social network services.In addition

the study on kinship verification is of great theoretical significance to related fields.Several commonalities can be observed among kinship verification algorithms

which are based on traditional machine learning.This paper proposes a new method for studying kinship verification based on face images through deep metric learning.

Method

Currently

deep learning algorithm can effectively understand a single face image

but the relationship between multiple subjects is still one of the challenging problems in the field of computer vision.This paper presents an approach to kinship verification between parents and children.First

a deep convolution neural network called "FaceCNN" is trained by using a dataset that contains more than 5 000 000 face images from which deep features are extracted.The training set of the deep convolutional neural network contains Internet-sourced datasets of thousands of people

including public kinship datasets

Microsoft face datasets

some films and television dramas

and public figures.Each dataset contains several face images.These face images have complex backgrounds

large differences in image quality

and various face postures

expressions

ages

and genders.Before training the deep network

the face images in the dataset must be preprocessed

including face filtering

face detection

and key point alignment.Then

all images are cut into 144×144 pixels.Then

we divide the father-mother-son and father-mother-daughter samples into father-son

mother-son

father-daughter

and mother-daughter samples.However

we cannot use the Mahalanobis distance to perform metric learning for kinship verification because the traditional Mahalanobis distance measurement learning method only seeks a linear transformation and cannot easily capture the non-linear manifold of the face image.A discriminative metric learning algorithm called DDML is adopted to render the kinship features as near as possible and vice versa.Hierarchical non-linear transformation is employed to achieve a more discriminative learning.Finally

we propose a kinship verification method for parents and children where the cosine similarity is calculated between parents and children.The children may be more similar to one of the parents

and thus we must calculate the similarity probability of the children with their father and mother.The final kinship score is the weighted average of the similarity between children and parents.

Result

A nine-layer deep model is trained to determine whether the given images have a relationship.The effectiveness of features extraction by using FaceCNN combined with the deep metric learning for kinship verification is validated in the experiment in which the recognition accuracy between parents and son performed on the TSKinFace dataset is 87.71%

and that between parents and daughter is 89.18%

which indicates that the recognition rate of the model increased by 1.31% and 4.87%

respectively.Meanwhile

the proposed approach only requires 3.616 s for metric learning and parental similarity calculation

reducing the time consumed greatly compared with other algorithms.

Conclusion

The proposed kinship verification takes full advantage of the characterization and learning ability of the deep neural network

and it does not only consume less time but also improves the recognition accuracy.

关键词

Keywords

references

Fang R G, Tang K D, Snavely N, et al.Towards computational models of kinship verification[C]//Proceedings of 2010 IEEE International Conference on Image Processing.Hong Kong, China: IEEE, 2010: 1577-1580.[ DOI:DOI:10.1109/ICIP.2010.5652590 http://dx.doi.org/10.1109/ICIP.2010.5652590 ]

Xia S Y, Shao M, Luo J B, et al.Understanding kin relationships in a photo[J].IEEE Transactions on Multimedia, 2012, 14(4):1046-1056.[DOI:10.1109/TMM.2012.2187436]

Qin X Q, Liu D K, Wang D.Heterogeneous similarity learning for more practical kinship verification[J].Neural Processing Letters, 2018, 47(3):1253-1269.[DOI:10.1007/s11063-017-9694-3]

Zhao Y G, Song Z, Zheng F, et al.Learning a multiple kernel similarity metric for kinship verification[J].Information Sciences, 2018, 430-431:247-260.[DOI:10.1016/j.ins.2017.11.048]

Sun Y, Chen Y H, Wang X G, et al.Deep learning face representation by joint identification-verification[C]//Proceedings of the 27th International Conference on Neural Information Processing Systems.Montreal, Canada: MIT Press, 2014: 1988-1996.

Somanath G, Kambhamettu C.Can faces verify blood-relations?[C]//Proceedings of the IEEE 5th International Conference on Biometrics: Theory, Applications and Systems.Arlington, VA, USA: IEEE Press, 2012: 105-112.[ DOI:10.1109/BTAS.2012.6374564 http://dx.doi.org/10.1109/BTAS.2012.6374564 ]

Ghahramani M, Yau W Y, Teoh E K.Family verification based on similarity of individual family member's facial segments[J].Machine Vision and Applications, 2014, 25(4):919-930.[DOI:10.1007/s00138-013-0566-1]

Dibeklioglu H, Salah A A, Gevers T.Like father, like son: facial expression dynamics for kinship verification[C]//Proceedings of 2013 IEEE International Conference on Computer Vision.Sydney, Australia: IEEE, 2013: 1497-1504.[ DOI:10.1109/ICCV.2013.189 http://dx.doi.org/10.1109/ICCV.2013.189 ]

Vieira T F, Bottino A, Islam I U.Automatic verification of parent-child pairs from face images[C]//Proceedings of the 18th Iberoamerican Congress Progress in Pattern Recognition, Image Analysis, Computer Vision, and Applications.Havana, Cuba: Springer-Verlag, 2013: 326-333.[ DOI:10.1007/978-3-642-41827-3_41 http://dx.doi.org/10.1007/978-3-642-41827-3_41 ]

Vieira T F, Bottino A, Laurentini A, et al.Detecting siblings in image pairs[J].The Visual Computer, 2014, 30(12):1333-1345.[DOI:10.1007/s00371-013-0884-3]

Fang R G, Gallagher A C, Chen T, et al.Kinship classification by modeling facial feature heredity[C]//Proceedings of 2013 IEEE International Conference on Image Processing.Melbourne, VIC, Australia: IEEE Press, 2014: 2983-2987.[ DOI:10.1109/ICIP.2013.6738614 http://dx.doi.org/10.1109/ICIP.2013.6738614 ]

Lu J W, Zhou X Z, Tan Y P, et al.Neighborhood repulsed metric learning for kinship verification[J].IEEE Transactions on Pattern Analysis and Machine Intelligence, 2014, 36(2):331-345.[DOI:10.1109/TPAMI.2013.134]

Zhang K P, Zhang Z P, Li Z F, et al.Joint face detection and alignment using multitask cascaded convolutional networks[J].IEEE Signal Processing Letters, 2016, 23(10):1499-1503.[DOI:10.1109/LSP.2016.2603342]

Goodfellow I, Warde-Farley D, Mirza M, et al. Maxout networks[C]//Proceddings of the 30th Ineternational Conference on Machine Learning. PMLR, 2013: 1319-1327..

Wen Y D, Zhang K P, Li Z F, et al.A discriminative feature learning approach for deep face recognition[C]//Proceedings of the 14th European Conference on Computer Vision.Amsterdam, The Netherlands: Springer International Publishing, 2016: 499-515.[ DOI:10.1007/978-3-319-46478-7_31 http://dx.doi.org/10.1007/978-3-319-46478-7_31 ]

van der Maaten L, Hinton G.Visualizing data using t-SNE[J].Journal of Machine Learning Research, 2008, 9:2579-2605.

Hu J L, Lu J W, Tan Y P.Discriminative deep metric learning for face verification in the wild[C]//Proceedings of 2014 IEEE Conference on Computer Vision and Pattern Recognition.Columbus, OH, USA: IEEE, 2014: 1875-1882.[ DOI:10.1109/CVPR.2014.242 http://dx.doi.org/10.1109/CVPR.2014.242 ]

Qin X Q, Tan X Y, Chen S C.Tri-subject kinship verification:understanding the core of a family[J].IEEE Transactions onMultimedia, 2015, 17(10):1855-1867.[DOI:10.1109/TMM.2015.2461462]

Dehshibi M M, Shanbehzadeh J.Cubic norm and kernel-based bi-directional PCA:toward age-aware facial kinship verification[J].The Visual Computer, 2017:1-18.[DOI:10.1007/s00371-017-1442-1]

Dehghan A, Ortiz E G, Villegas R, et al.Who do i look like? determining parent-offspring resemblance via gated autoencoders[C]//Proceedings of 2014 IEEE Conference on Computer Vision and Pattern Recognition.Columbus, OH, USA: IEEE, 2014: 1757-1764.[ DOI:10.1109/CVPR.2014.227 http://dx.doi.org/10.1109/CVPR.2014.227 ]

Kohli N, Vatsa M, Singh R, et al.Hierarchical representation learning for kinship verification[J].IEEE Transactions on Image Processing, 2017, 26(1):289-302.[DOI:10.1109/TIP.2016.2609811]

Alert me when the article has been cited

提交

A summary on group re-identification

Indoor scene segmentation based on fully convolutional neural networks

Performance of convolutional neural network and deep belief network in sea ice-water classification using SAR imagery

Automated classification of diabetic retinal images by using deep learning method

Image super-resolution reconstruction based on intermediate supervision convolutional neural networks