Objective

2

Gait is a sort of human behavioral biometric feature

which is clarified as a style of person walks. Compared with other biometric features like human face

fingerprint and iris

the feature of gait is that it can be captured at a long-distance without the cooperation of the subjects. Gait recognition has its potential in surveillance security

criminal investigation and medical diagnosis. However

gait recognition is changed clearly in the context of clothing

carrying status

view variation and other factors

resulting in strong intra gradient changes in the extracted gait features. The relevant view change is a challenging issue as appearance differences are introduced for different views

which leads to the significant decline of cross view recognition performance. The existing generative gait recognition methods focus on transforming gait templates to a specific view

which may decline the recognition rate in a large variation of multi-views. A cross-view gait recognition analysis is demonstrated based on generative adversarial networks (GANs) derived of self-attention mechanism.

Method

2

Our network structure analysis is composed of generator

G

view discriminator

D

and identity preserver

Φ

. Gait energy images (GEI) is used as the input of network to achieve view transformation of gaits across two various views for cross view gait recognition task. The generator is based on the encoder-decoder structure. First

the input GEI image is disentangled from the view information and the identity information derived of the encoder

G

enc

which is encoded into the identity feature representation

f

(

x

) in the latent space. Next

it is concatenated with the view indicator

v

which is composed of the one-hot coding with the target view assigned 1. To achieve different views of transformation

the concatenated vector as input is melted into the decoder

G

dec

to generate the GEI image from the target view. In order to generate a more accurate gait template in the target view for view transformation task

pixel-wise loss is introduced to constrain the generated image at the end of decoder. In the discriminant network

the view discriminator learning distinguishes the true or false of the input images and classifies them to its corresponding view domain. It is composed of four Conv-LeakyReLU blocks and in-situ two convolution layers those are real/fake discrimination and view classification each. For the constraint of the generated images inheriting identity information in the process of gait template view transformation

an identity preserver is introduced to bridge the gap between the target and generated gait templates. The input of identity preserver are three generated images

which are composed of anchor samples

positive samples from other views with the same identity as anchor samples

and negative samples are from the same view in related to different identities. The following Tri-Hard loss is used to enhance the discriminability of the generated image. The GAN-based gait recognition method can achieve the view transformation of gait template but it cannot capture the global

long-range dependency within features in the process of view transformation effectively. The details of the generated image are not clear result in blurred artifacts. The self-attention mechanism can efficiently sort the long-range dependencies out within internal representations of images. We yield the self-attention mechanism into the generator and discriminator network

and the self-attention module is integrated into the up-sampling area of the generator

which can involve the global and local spatial information. The self-attention module derived discriminator can clarify the real image originated from the generated. We update parameters of one module while keeping parameters of the other two modules fixed

and spectral normalization is used to increase the stable training of the network.

Result

2

In order to verify the effectiveness of the proposed method for cross-view gait recognition

several groups of comparative experiments are conducted on Chinese Academy of Sciences' Institute of Automation gait database——dataset B (CASIA-B) as mentioned below: 1) To clarify the influence of self-attention module plus to identify positions of the generator on recognition performance

the demonstrated results show that it is prior to add self-attention module to the feature map following de-convolution in the second layer of decoder; 2) Ablation experiment on self-attention module and identity preserving loss illustrates that the recognition rate is 15% higher than that of GaitGAN method when self-attention module and identity preserving loss are introduced simultaneously; 3) The frame-shifting method is used to enhance the GEI dataset on CASIA-B

and the improved recognition accuracy of the method is significantly harnessed following GEI data enhancement. Our illustration is derived of the OU-MVLP (OU-ISIR gait database-multi-view large population dataset) large-scale cross-view gait database

which has a rank-1 average recognition rate of 65.9%. The demonstrated results based on OU-MVLP are quantitatively analyzed

and the gait templates synthesized at four views (0°

30°

60°

and 90°) are visualized in the dataset. The results show that the generated gait images are highly similar to the gait images with real and target views even when the difference of views is large.

Conclusion

2

A generative adversarial network framework derived of self-attention mechanism is implemented

which can achieve view transformation of gait templates across two optioned views using one uniform model

and retain the gait feature information in the process of view transformation while improving the quality of generated images. The effective self-attention module demonstrates that the generated gait templates in the target view is incomplete

and improves the matching of the generated images; The identity preserver based on Tri-Hard loss constrains the generated gait templates inheriting identity information via input gaits and the discrimination of the generated images is enhanced. The integration of the self-attention module and the Tri-Hard loss identity preserver improves the effect and quality of transformation of gaits

and the recognition accuracy is improved for qualified cross-view gait recognition. As the GEI input of the model

the quality of pedestrian detection and segmentation will intend to the quality loss of the synthesized GEI images straightforward in real scenarios. The further research will focus on problem solving of cross-view gait recognition in complex scenarios.