Multimodal emotion recognition has been a very important research topic in affect computing. This paper mainly focuses on the methods of bimodal emotion recognition based on body gesture and facial expression and presents a new bimodal emotion recognition method based on bilateral sparse partial least squares(BSPLS). First

the spatio-temporal feature is extracted as the emotion feature vector for video-based body gesture and facial expression respectively. Then we propose a new bilateral sparse partial least squares(BSPLS) method to extract emotion feature and fuse facial expression and body gestures as new emotion feature. Finally

we use two classifiers in emotional classification. We compared the BSPLS method with some subspace methods including PCA

CCA and PLSR based on the data from the FABO database. The experimental results show that the fusion feature methods are all better than the monomodal emotion recognition and our BSPLS feature fusion provides the best recognition performance.