Objective

2

Gait has become a popular research topic that is currently investigated by using visual and machine learning methods. However

most of these studies are concentrated in the field of human identification and use 2D RGB images. In contrast to these studies

this paper investigates abnormal gait recognition by using 3D data. A method based on 3D point cloud data and the semantic body model is then proposed for view-invariant abnormal gait recognition. Compared with traditional 2D abnormal gait recognition approaches

the proposed 3D-based method can easily deal with many obstacles in abnormal gait modelling and recognition processes

including view-invariant problems and interference from external items.

Method

2

The point cloud data of human gait are obtained by using an infrared structured light sensor

which is a 3D depth camera that uses a structure projector and reflecting light receiver to gain the depth information of an object and calculate its point cloud data. Although the point cloud data of the human body are also in 3D

they are generally unstructured

thereby influencing the 3D representation of the human body and posture. To deal with this problem

a 3D parametric human body learned from the 3D body dataset by using a statistic method is introduced in this paper. The parameterized human body model refers to the description and construction of the corresponding visual human body mesh through abstract high-order semantic features

such as height

weight

age

gender

and skeletal joints. The parameters are determined by using statistical learning methods. The human body is embedded into the model

and the 3D parametric model can be deformed both in shapes and poses. Unlike traditional methods that directly model the 3D body from point cloud data via the point cloud reduction algorithm and triangle mesh grid method

the related 3D parameterized body model is deformed to fit the point cloud data in both shape and posture. The standard 3D human model proposed in this paper is constructed based on the body shape PCA (principal component analysis) analysis and skin method. An observation function that measures the similarity of the deformed 3D model with the raw point cloud data of the human body is also introduced. An accurate deformation of the 3D body is ensured by iteratively minimizing the observation function. After the 3D model estimation process

the features of the raw point cloud data of the human body are converted into a high-level structured representation of the human body. This process not only abstracts the unstructured data to a high-order semantic description but also effectively reduces the dimensionality of the original data. After 3D modelling and structured feature representation

a convolution gated recurrent unit (ConvGRU) recurrent neural network is applied to extract the temporal-spatial features of the projected depth gait images. ConvGRU has the advantages of both convolutional and recurrent neural networks

the latter of which is based on the gate structure. The tow gates (i.e.

reset and update gates) help the model memorize useful information and forget useless data. In the final classification process

the samples are divided into positive

negative

and anchor samples. The anchor sample is the sample itself

the positive samples are same-category samples that belong to different objects

and the negative samples are those that belong to opposite categories. Training the classifier by using the triples elements strategy can improve its ability to discriminate small feature differences of different categories. At the same time

a virtual 3D sample synthesizing method based on body

pose

and view deformation is proposed to deal with the data shortage problem of abnormal gait. Compared with normal gait datasets

abnormal gait data

especially 3D abnormal datasets

are rare and difficult to obtain. Moreover

given the limited amount of ground truth data

most of the abnormal data are imitated by the experimental participates. As a result

the virtual synthesizing method can help extend the training data and improve the generalization ability of the abnormal gait classification model.

Result

2

Experiments were performed by using the CSU(Central South University) abnormal 3D gait database and the depth-included human action video (DHA) dataset

and different abnormal gait or action recognition methods were compared with the proposed approach. In the CSU abnormal gait database

the rank-1 mean detection and recognition rate of abnormal gait is 96.6% at the 0°

45°

and 90° views. In the 90°-0° cross view recognition experiment

the proposed method outperforms the other approaches that use DMHI(difference motion history image) or DMM-CNN(depth motion map-convolutional neural network) as feature representation by at least 25%. Meanwhile

in the DHA dataset

the proposed method result has a rank-1 mean detection and recognition rate of near 98%

which is 2% to 3% higher than that of novel approaches

including DMM based methods.

Conclusion

2

Based on the feature extraction method of the 3D parameterized human body model

abnormal gait image data can be abstracted to high-order descriptions and effectively complete the feature extraction and dimensionality reduction of the original data. ConvGRU can extract the spatial and temporal features of the abnormal gait data well. The virtual sample synthesis and triple classification methods can be combined to classify and recognize abnormal gait data from different views. The proposed method not only improves the recognition accuracy of abnormal gait under various view angles but also provides a new approach for the detection and recognition of abnormal gait.