浏览全部资源
扫码关注微信
Received：06 June 2019，
Revised：2019-9-17，
Accepted：24 September 2019，
Published：16 April 2020
移动端阅览
目的
2
为解决真实环境中由类内差距引起的面部表情识别率低及室内外复杂环境对类内差距较大的面部表情识别难度大等问题，提出一种利用生成对抗网络（generative adversarial network，GAN）识别面部表情的方法。
方法
2
在GAN生成对抗的思想下，构建一种IC-GAN（intra-class gap GAN）网络结构，使用卷积组建编码器、解码器对自制混合表情图像进行更深层次的特征提取，使用基于动量的Adam（adaptive moment estimation）优化算法进行网络权重更新，重点针对真实环境面部表情识别过程中的类内差距较大的表情进行识别，使其更好地适应类内差异较大的任务。
结果
2
基于Pytorch环境，在自制的面部表情数据集上进行训练，在面部表情验证集上进行测试，并与深度置信网络（deep belief network，DBN）和GoogLeNet网络进行对比实验，最终IC-GAN网络的识别结果比DBN网络和GoogLeNet网络分别提高11%和8.3%。
结论
2
实验验证了IC-GAN在类内差距较大的面部表情识别中的精度，降低了面部表情在类内差距较大情况下的误识率，提高了系统鲁棒性，为面部表情的生成工作打下了坚实的基础。
Objective
2
China has a large population of 1.3 billion in 2005
accounting for 19% of the world's population. This number is equivalent to the population of Europe or Africa added to the populations of Australia
North America
and Central America. It is one of the few populous countries in the world
and its huge population size has brought many problems. With the rapid development of the economy
the number of people working outside of their homes is increasing
the population is moving frequently
and the safety of the floating population is even difficult to control. The huge mobile population provides the city's infrastructure and public services tremendous pressure. Thus
to conduct a comprehensive check on the area of adult traffic
which is time-consuming and labor-intensive
is difficult for security and related staff. Particularly
complex environmental safety problems
such as subways
railway stations
and airports
are becoming increasingly serious. Unstable events occur frequently
security situation is receiving much attention
and urban management and service systems are seriously lagging behind. These conditions need to be improved
especially after the September 11 incident in the United States. The situation has aroused widespread concern in the international community. Meanwhile
expression is the most intuitive way for humans to express emotions. In addition to language communication
expressions are extremely effective means of communication. People usually express their inner feelings through specific expressions. Expression can be used to judge other person's thoughts. For expressions used to express information
psychologist Mehrabian summed up a formula:emotional expression=7% of words + 38% of sound + 55% of facial expressions. Expression is one of the most important features of human emotion recognition. Expression is the emotional state expressed by facial muscle changes. Through the facial expression of the person's face
to evaluate abnormal psychological state
speculate on extreme emotions
and observe the facial expressions of pedestrians in the subway
railway station
and airport to further judge the psychology of the person is possible. We provide technical support to determine who is suspicious and prevent certain criminal activities in a timely manner. Strengthening urban surveillance and identifying the facial expressions of criminals are especially important. Expression plays an important role in human emotion cognition. However
factors affecting facial expression recognition in safety screening are extremely large
and the large intra-class gap seriously inflluences the accuracy of facial expression recognition. The problem of large gaps in facial expression recognition in a real environment is solved by identifying suspected molecules to be monitored should be identified
and the security personnel should prepare in advance to accurately identify them. Facial expressions are also particularly important for preventing security problems. The era of large data has arrived. Meanwhile
with the advancement of computer hardware
deep learning continues to develop. The traditional facial expression recognition method cannot meet the needs of the development of the times
and a new algorithm based on deep learning facial expression recognition is coming soon. Learning methods are widely used in facial expression recognition. Although facial recognition intelligent recognition technology has a long history of research
a large number of research methods have been proposed. However
due to the large facial expression gap
the expression is complex
and the influencing factors are many. The current intelligent recognition effect of facial expression results is not ideal. Considering the deep learning because of its powerful expressive ability
this study introduces the model structure of traditional neural network and carries out corresponding experiments and analysis in the context of real-life facial expression recognition and proposes real-world facial expression recognition research based on deep learning. In the next period
real-world facial expression recognition will make considerable progress. This work further studies the realistic facial expression recognition based on deep learning.
Method
2
This study constructs a new IC-GAN(intra-class gap GAN(generative adversarial network)) recognition network model
providing good adaptability to the facial expression recognition task with large gap within the class. The network consists of a convolutional layer
a fully connected layer
an active layer
a BathNorm layer
and a Softmax layer
in which a convolutional assembly encoder and a decoder are used to perform deep feature extraction on facial expression images
and download and parse from the network. The video self-made mixed facial expression data set is based on the real environment
the image is expanded
and the facial expression data are normalized. The complexity of the facial expression features with large differences within the class also increased the network training and network recognition. The momentum-based Adam is used to update the network weight
adjust the network parameters
and optimize the network structure based on this factor. In this study
the facial expression category data are trained based on the Pytorch platform in deep learning and tested on the verification set of the self-made mixed facial expression data set.
Result
2
When the input image is 256×256 pixels
the IC-GAN network model can reduce the false positive rate of the expression in large difference in the class
image blur
and facial expression incompleteness
and improve the system robustness. Compared with deep belief network(DBN) deep trust network and GoogLeNet network
the recognition result of IC-GAN network is 11% higher than that of DBN network and 8.3% higher than that of GoogLeNet network.
Conclusion
2
The IC-GAN accuracy in facial expression recognition with large gaps in the class is verified by experiments. This condition reduces the misunderstanding rate of facial expressions in large intra-class differences
improves the system robustness
and lays down the solid foundation for facial expression generation.
Bowles C, Chen L, Guerrero R, Bentley P, Gunn R, Hammers A, Dickie D A, Hernández M V, Wardlaw J and Rueckert D. 2018. GAN augmentation: augmenting training data using generative adversarial networks[EB/OL]. (2018-11-27)[2019-08-29] . https://arxiv.org/pdf/1810.10863v1.pdf https://arxiv.org/pdf/1810.10863v1.pdf
Cao J M, Ni R R and Yang B. 2018. Binary-channel convolutional neutral network for facial expression recognition. Journal of Nanjing Normal University (Engineering Technology Edition), 18(3):1-9
曹金梦, 倪蓉蓉, 杨彪. 2018.面向面部表情识别的双通道卷积神经网络.南京师范大学学报(工程技术版), 18(3):1-9)[DOI:10.3969/j.issn.1672-1292.2018.03.001]
Chen Y Z and Hu H F. 2019. An improved method for semantic image inpainting with GANs:progressive inpainting. Neural Processing Letters, 49(3):1355-1367[DOI:10.1007/s11063-018-9877-6]
Cherry J M, Adler C, Ball C, Chervitz S A, Dwight S S, Hester E T, Jia Y K, Juvik G, Roe T Y, Schroeder M, Weng S and Botstein D. 1998. SGD:saccharomyces genome database. Nucleic Acids Research, 26(1):73-79[DOI:10.1093/nar/26.1.73]
Cui C. 2017. Design and Research on Face Recognition System Based on Deep Convolutional Neural Network. Harbin:Harbin Institute of Technology
崔成. 2017.基于深度卷积神经网络人脸识别系统设计与研究.哈尔滨:哈尔滨工业大学
Hadgu A T, Nigam A and Diaz-Aviles E. 2015. Large-scale learning with AdaGrad on spark//Proceedings of 2015 IEEE International Conference on Big Data. Santa Clara, United States: IEEE: 2828-2830[ DOI: 10.1109/BigData.2015.7364091 http://dx.doi.org/10.1109/BigData.2015.7364091 ]
Hinton G. 2011. Deep belief nets//Sammut C, Webb G I, eds. Encyclopedia of Machine Learning. Boston: Springer: 1527-1554[ DOI: 10.1007/978-0-387-30164-8_208 http://dx.doi.org/10.1007/978-0-387-30164-8_208 ]
Ioffe S and Szegedy C. 2015. Batch normalization: accelerating deep network training by reducing internal covariate shift[EB/OL]. (2015-05-02)[2019-08-29] . https://arxiv.org/pdf/1502.03167.pdf https://arxiv.org/pdf/1502.03167.pdf
Kingma D P and Ba J. 2014. Adam: a method for stochastic optimization[EB/OL]. (2014-01-30)[2019-08-29] . https://arxiv.org/pdf/1412.6980.pdf https://arxiv.org/pdf/1412.6980.pdf
Liu M Y, Li S X, Shan S G and Chen X L. 2015. AU-inspired deep networks for facial expression feature learning. Neurocomputing, 159:126-136[DOI:10.1016/j.neucom.2015.02.011]
Simonyan K and Zisserman A. 2014. Very deep convolutional networks for large-scale image recognition[EB/OL]. (2014-09-04)[2019-08-29] . https://arxiv.org/pdf/1409.1556.pdf https://arxiv.org/pdf/1409.1556.pdf
Suárez P L, Sappa A D and Vintimilla B X. 2017. Infrred image colorization based on a triplet DCGAN architecture//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops. Honolulu, HI, USA: IEEE: 212-217[ DOI: 10.1109/CVPRW.2017.32 http://dx.doi.org/10.1109/CVPRW.2017.32 ]
Szegedy C, Liu W, Jia Y Q, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V and Rabinovich A. 2015. Going deeper with convolutions//Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition. Boston,MA, USA: IEEE: 1-9[ DOI: 10.1109/CVPR.2015.7298594 http://dx.doi.org/10.1109/CVPR.2015.7298594 ]
Szegedy C, Vanhoucke V, Ioffe S, Shlens J and Wojna Z. 2016. Rethinking the inception architecture for computer vision//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, NV, USA: IEEE: 2818-2826[ DOI: 10.1109/CVPR.2016.308 http://dx.doi.org/10.1109/CVPR.2016.308 ]
Tang S J and Li P C. 2014. Image feature extraction in face recognition and research on matching technology. Computer CD-ROM Software and Application, 17(14):133-133, 135
唐守军, 李鹏程. 2014.人脸识别中图像特征提取与匹配技术研究.计算机光盘软件与应用, 17(14):133-133, 135
Xiao R Q. 2017. The Influence of Facial Expression and Body Gestures on Peak Emotion Perception. Shanghai:East China Normal University
肖瑞琪. 2017.极端情绪下面部表情与身体姿势对表情识别的影响.上海:华东师范大学
Xu L L, Zhang S M and Zhao J L. 2017. Summary of facial expression recognition methods based on image. Journal of Computer Applications, 37(12):3509-3516
徐琳琳, 张树美, 赵俊莉. 2017.基于图像的面部表情识别方法综述.计算机应用, 37(12):3509-3516)[DOI:10.11772/j.issn.1001-9081.2017.12.3509]
Yao X. 2010. The Affecting Factors of Facial Expression Recognition:Expression Intensity and the Presentation. Jilin:Jilin University
姚雪. 2010.面部表情识别的影响因素:表情强度和呈现方式.吉林:吉林大学
Yu Z D and Zhang C. 2015. Image based static facial expression recognition with multiple deep network learning//Proceedings of 2015 ACM on International Conference on Multimodal Interaction. Seattle: ACM: 435-442[ DOI: 10.1145/2818346.2830595 http://dx.doi.org/10.1145/2818346.2830595 ]
Zhang Y M, Diao Q, Huang S and Hu W. 2003. DBN based multi-stream models for speech//Proceedings of 2003 IEEE International Conference on Acoustics, Speech, and Signal Processing. Hong Kong, China: IEEE: 1520-6149[ DOI: 10.1109/I-CASSP.2003.1198911 http://dx.doi.org/10.1109/I-CASSP.2003.1198911 ]
相关文章
相关作者
相关机构
京公网安备11010802024621
