浏览全部资源
扫码关注微信
Received：26 August 2020，
Revised：2021-1-7，
Accepted：14 January 2021，
Published：16 April 2022
移动端阅览
目的
2
针对目前基于生成式的步态识别方法采用特定视角的步态模板转换、识别率随视角跨度增大而不断下降的问题，本文提出融合自注意力机制的生成对抗网络的跨视角步态识别方法。
方法
2
该方法的网络结构由生成器、视角判别器和身份保持器构成，建立可实现任意视角间步态转换的网络模型。生成网络采用编码器—解码器结构将输入的步态特征和视角指示器连接，进而实现不同视角域的转换，并通过对抗训练和像素级损失使生成的目标视角步态模板与真实的步态模板相似。在判别网络中，利用视角判别器来约束生成视角与目标视角相一致，并使用联合困难三元组损失的身份保持器以最大化保留输入模板的身份信息。同时，在生成网络和判别网络中加入自注意力机制，以捕捉特征的全局依赖关系，从而提高生成图像的质量，并引入谱规范化使网络稳定训练。
结果
2
在CASIA-B(Chinese Academy of Sciences' Institute of Automation gait database——dataset B)和OU-MVLP(OU-ISIR gait database-multi-view large population dataset)数据集上进行实验，当引入自注意力模块和身份保留损失训练网络时，在CASIA-B数据集上的识别率有显著提升，平均rank-1准确率比GaitGAN(gait generative adversarial network)方法高15%。所提方法在OU-MVLP大规模的跨视角步态数据库中仍具有较好的适用性，可以达到65.9%的平均识别精度。
结论
2
本文方法提升了生成步态模板的质量，提取的视角不变特征更具判别力，识别精度较现有方法有一定提升，能较好地解决跨视角步态识别问题。
Objective
2
Gait is a sort of human behavioral biometric feature
which is clarified as a style of person walks. Compared with other biometric features like human face
fingerprint and iris
the feature of gait is that it can be captured at a long-distance without the cooperation of the subjects. Gait recognition has its potential in surveillance security
criminal investigation and medical diagnosis. However
gait recognition is changed clearly in the context of clothing
carrying status
view variation and other factors
resulting in strong intra gradient changes in the extracted gait features. The relevant view change is a challenging issue as appearance differences are introduced for different views
which leads to the significant decline of cross view recognition performance. The existing generative gait recognition methods focus on transforming gait templates to a specific view
which may decline the recognition rate in a large variation of multi-views. A cross-view gait recognition analysis is demonstrated based on generative adversarial networks (GANs) derived of self-attention mechanism.
Method
2
Our network structure analysis is composed of generator
G
view discriminator
D
and identity preserver
Φ
. Gait energy images (GEI) is used as the input of network to achieve view transformation of gaits across two various views for cross view gait recognition task. The generator is based on the encoder-decoder structure. First
the input GEI image is disentangled from the view information and the identity information derived of the encoder
G
enc
which is encoded into the identity feature representation
f
(
x
) in the latent space. Next
it is concatenated with the view indicator
v
which is composed of the one-hot coding with the target view assigned 1. To achieve different views of transformation
the concatenated vector as input is melted into the decoder
G
dec
to generate the GEI image from the target view. In order to generate a more accurate gait template in the target view for view transformation task
pixel-wise loss is introduced to constrain the generated image at the end of decoder. In the discriminant network
the view discriminator learning distinguishes the true or false of the input images and classifies them to its corresponding view domain. It is composed of four Conv-LeakyReLU blocks and in-situ two convolution layers those are real/fake discrimination and view classification each. For the constraint of the generated images inheriting identity information in the process of gait template view transformation
an identity preserver is introduced to bridge the gap between the target and generated gait templates. The input of identity preserver are three generated images
which are composed of anchor samples
positive samples from other views with the same identity as anchor samples
and negative samples are from the same view in related to different identities. The following Tri-Hard loss is used to enhance the discriminability of the generated image. The GAN-based gait recognition method can achieve the view transformation of gait template but it cannot capture the global
long-range dependency within features in the process of view transformation effectively. The details of the generated image are not clear result in blurred artifacts. The self-attention mechanism can efficiently sort the long-range dependencies out within internal representations of images. We yield the self-attention mechanism into the generator and discriminator network
and the self-attention module is integrated into the up-sampling area of the generator
which can involve the global and local spatial information. The self-attention module derived discriminator can clarify the real image originated from the generated. We update parameters of one module while keeping parameters of the other two modules fixed
and spectral normalization is used to increase the stable training of the network.
Result
2
In order to verify the effectiveness of the proposed method for cross-view gait recognition
several groups of comparative experiments are conducted on Chinese Academy of Sciences' Institute of Automation gait database——dataset B (CASIA-B) as mentioned below: 1) To clarify the influence of self-attention module plus to identify positions of the generator on recognition performance
the demonstrated results show that it is prior to add self-attention module to the feature map following de-convolution in the second layer of decoder; 2) Ablation experiment on self-attention module and identity preserving loss illustrates that the recognition rate is 15% higher than that of GaitGAN method when self-attention module and identity preserving loss are introduced simultaneously; 3) The frame-shifting method is used to enhance the GEI dataset on CASIA-B
and the improved recognition accuracy of the method is significantly harnessed following GEI data enhancement. Our illustration is derived of the OU-MVLP (OU-ISIR gait database-multi-view large population dataset) large-scale cross-view gait database
which has a rank-1 average recognition rate of 65.9%. The demonstrated results based on OU-MVLP are quantitatively analyzed
and the gait templates synthesized at four views (0°
30°
60°
and 90°) are visualized in the dataset. The results show that the generated gait images are highly similar to the gait images with real and target views even when the difference of views is large.
Conclusion
2
A generative adversarial network framework derived of self-attention mechanism is implemented
which can achieve view transformation of gait templates across two optioned views using one uniform model
and retain the gait feature information in the process of view transformation while improving the quality of generated images. The effective self-attention module demonstrates that the generated gait templates in the target view is incomplete
and improves the matching of the generated images; The identity preserver based on Tri-Hard loss constrains the generated gait templates inheriting identity information via input gaits and the discrimination of the generated images is enhanced. The integration of the self-attention module and the Tri-Hard loss identity preserver improves the effect and quality of transformation of gaits
and the recognition accuracy is improved for qualified cross-view gait recognition. As the GEI input of the model
the quality of pedestrian detection and segmentation will intend to the quality loss of the synthesized GEI images straightforward in real scenarios. The further research will focus on problem solving of cross-view gait recognition in complex scenarios.
Chao H Q, He Y W, Zhang J P and Feng J F. 2019. GaitSet: regarding gait as a set for cross-view gait recognition//Proceedings of 2019 AAAI Conference on Artificial Intelligence. Honolulu, USA: AAAI Press: 8126-8133 [ DOI: 10.1609/aaai.v33i01.33018126 http://dx.doi.org/10.1609/aaai.v33i01.33018126 ]
Goodfellow I J, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A and Bengio Y. 2014. Generative adversarial nets//Proceedings of the 27th International Conference on Neural Information Processing Systems. Montreal, Canada: ACM: 2672-2680 [ DOI: 10.5555/2969033.2969125 http://dx.doi.org/10.5555/2969033.2969125 ]
Han J and Bhanu B. 2006. Individual recognition using gait energy image. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28(2): 316-322 [DOI: 10.1109/TPAMI.2006.38]
He Y W, Zhang J P, Shan H M and Wang L. 2019. Multi-task GANs for view-specific feature learning in gait recognition. IEEE Transactions on Information Forensics and Security, 14(1): 102-113 [DOI: 10.1109/TIFS.2018.2844819]
Hermans A, Beyer L and Leibe B. 2017. In defense of the triplet loss for person re-identification [EB/OL]. [2020-08-10] . https://arxiv.org/pdf/1703.07737.pdf https://arxiv.org/pdf/1703.07737.pdf
Hu M D, Wang Y H, Zhang Z X, Little J J and Huang D. 2013. View-invariant discriminative projection for multi-view gait-based human identification. IEEE Transactions on Information Forensics and Security, 8(12): 2034-2045 [DOI: 10.1109/TIFS.2013.2287605]
Liao R J, An W Z, Yu S Q, Li Z and Huang Y Z. 2020. Dense-view GEIs set: view space covering for gait recognition based on dense-view GAN//Proceedings of 2020 IEEE International Joint Conference on Biometrics. Houston, USA: IEEE: 1-9 [ DOI: 10.1109/IJCB48548.2020.9304910 http://dx.doi.org/10.1109/IJCB48548.2020.9304910 ]
Makihara Y, Sagawa R, Mukaigawa Y, Echigo T and Yagi Y. 2006. Gait recognition using a view transformation model in the frequency domain//Proceedings of the 9th European Conference on Computer Vision. Graz, Austria: ACM: 151-163 [ DOI: 10.1007/11744078_12 http://dx.doi.org/10.1007/11744078_12 ]
Miyato T, Kataoka T, Koyama M and Yoshida Y. 2018. Spectral normalization for generative adversarial networks [EB/OL]. [2020-08-10] . https://arxiv.org/pdf/1802.05957.pdf https://arxiv.org/pdf/1802.05957.pdf
Ouyang N, Liang T and Lin L P. 2019. Self-attention network based image super-resolution. Journal of Computer Applications, 39(8): 2391-2395
欧阳宁, 梁婷, 林乐平. 2019. 基于自注意力网络的图像超分辨率重建. 计算机应用, 39(8): 2391-2395 [DOI: 10.11772/j.issn.1001-9081.2019010158]
Shiraga K, Makihara Y, Muramatsu D, Echigo T and Yagi Y. 2016. GEINet: view-invariant gait recognition using a convolutional neural network//Proceedings of 2016 International Conference on Biometrics (ICB). Halmstad, Sweden: IEEE: 1-8 [ DOI: 10.1109/ICB.2016.7550060 http://dx.doi.org/10.1109/ICB.2016.7550060 ]
Takemura N, Makihara Y, Muramatsu D, Echigo T and Yagi Y. 2018. Multi-view large population gait dataset and its performance evaluation for cross-view gait recognition. IPSJ Transactions on Computer Vision and Applications, 10(4): #4 [DOI: 10.1186/s41074-018-0039-6]
Takemura N, Makihara Y, Muramatsu D, Echigo T and Yagi Y. 2019. On input/output architectures for convolutional neural network-based cross-view gait recognition. IEEE Transactions on Circuits and Systems for Video Technology, 29(9): 2708-2719 [DOI: 10.1109/TCSVT.2017.2760835]
Tran L, Yin X and Liu X M. 2017. Disentangled representation learning GAN for pose-invariant face recognition//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, USA: IEEE: 1283-1292 [ DOI: 10.1109/CVPR.2017.141 http://dx.doi.org/10.1109/CVPR.2017.141 ]
Wang K J, Ding X N, Xing X L and Liu M C. 2019. A survey of multi-view gait recognition. Acta Automatica Sinica, 45(5): 841-852
王科俊, 丁欣楠, 邢向磊, 刘美辰. 2019. 多视角步态识别综述. 自动化学报, 45(5): 841-852 [DOI: 10.16383/j.aas.2018.c170599]
Wang Y Y, Song C F, Huang Y, Wang Z Y and Wang L. 2019. Learning view invariant gait features with Two-Stream GAN. Neurocomputing, 339: 245-254 [DOI: 10.1016/j.neucom.2019.02.025]
Wu Z F, Huang Y Z, Wang L, Wang X G and Tan T N. 2017. A comprehensive study on cross-view gait based human identification with deep CNNs. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(2): 209-226 [DOI: 10.1109/TPAMI.2016.2545669]
Xing X L, Wang K J, Yan T and Lyu Z W. 2016. Complete canonical correlation analysis with application to multi-view gait recognition. Pattern Recognition, 50: 107-117 [DOI: 10.1016/j.patcog.2015.08.011]
Yu S Q, Chen H F, Reyes E B G and Poh N. 2017a. GaitGAN: invariant gait feature extraction using generative adversarial networks//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops. Honolulu, USA: IEEE: 532-539 [ DOI: 10.1109/CVPRW.2017.80 http://dx.doi.org/10.1109/CVPRW.2017.80 ]
Yu S Q, Chen H F, Wang Q, Shen L L and Huang Y Z. 2017b. Invariant feature extraction for gait recognition using only one uniform model. Neurocomputing, 239: 81-93 [DOI: 10.1016/j.neucom.2017.02.006]
Yu S Q, Tan D L and Tan T N. 2006. A framework for evaluating the effect of view angle, clothing and carrying condition on gait recognition//Proceedings of the 18th International Conference on Pattern Recognition. Hong Kong, China: IEEE: 441-444 [ DOI: 10.1109/ICPR.2006.67 http://dx.doi.org/10.1109/ICPR.2006.67 ]
Zhang H, Goodfellow I, Metaxas D and Odena A. 2018. Self-attention Generative Adversarial Networks [EB/OL]. [2020-08-10] . https://arxiv.org/pdf/1805.08318.pdf https://arxiv.org/pdf/1805.08318.pdf
Zhi S S, Zhao Q H and Tang J. 2019. Semantic segmentation of gait body with multilayer semantic fusion convolutional neural network. Journal of Image and Graphics, 24(8): 1302-1314
支双双, 赵庆会, 唐琎. 2019. 多层语义融合CNN的步态人体语义分割. 中国图象图形学报, 24(8): 1302-1314 [DOI: 10.11834/jig.180597]
Zhu J Y, Park T, Isola P and Efros A A. 2017. Unpaired image-to-image translation using cycle-consistent adversarial networks//Proceedings of 2017 IEEE International Conference on Computer Vision. Venice, Italy: IEEE: 2242-2251 [ DOI: 10.1109/ICCV.2017.244 http://dx.doi.org/10.1109/ICCV.2017.244 ]
相关文章
相关作者
相关机构
京公网安备11010802024621
