Human action recognition is a widely studied area in computer vision and machine learning; it has many potential applications including human computer interfaces

video surveillance

and health care. In the past decade

extensive research efforts focused on recognizing human action from monocular video sequences. Since human motion is articulated

capturing human joint characters accurately from video is a very difficult task. The recent introduction of real time depth cameras such as the Kinect sensor

give us the opportunity to use 3D depth data of a scene instead of pictures. In this paper

we present a manifold-based framework for human action recognition using depth image data captured from depth camera. With the recent release of Kinect sensor and the technology assessing skeleton joint position from depth image matured

recent research used 3D skeleton joint position information as human body representation and achieved good recognition performance. As we know

human action is composed of ordered posture set

and the difference between postures is only a few changes of 3D joints pairwise

most of the 3D information changes only little. In this paper

we estimated the 3D joint locations from Kinect depth images and use pairwise relative positions as the representation of human features. In the training phase

the LE (Lalpacian eigenmaps) is used to build action model in low dimensional space. In test phase

the nearest-neighbor interpolation technique is used to map test sequence to the manifold space

then measure the distance with the test sequence and the training data. A novel modified Hausdorff distance is used to measure similarity and fitness of the test sequence and the training data in the matching process. The recognition performance of the proposed method was evaluated from Kinect sensor dataset and the result confirmed the proposed method can work well in several experiments. We also tested the proposed method on the MSR Action3D dataset and achieved state of the art accuracy in our comparison with related work when the training set has many samples. Manifold learning is an effective nonlinear dimensionality reduction method and low-dimensional motion models can be trained well when training sample size is large. We propose a novel human action recognition based on manifold learning in this paper. The experimental results show the effectiveness of the proposed method for human action recognition based on depth image sequence.