Current Issue Cover
面向运动想象脑电图识别的镜卷积神经网络

罗靖1,2, 王耀杰1,2, 刘光明1,2, 王晓帆1,2, 鲁晓锋1,2, 黑新宏1,2(1. 西安理工大学计算机科学与工程学院, 西安 710048;2.
2. 陕西省网络计算与安全技术重点实验室, 西安 710048)

摘 要
目的 脑电图(electroencephalogram,EEG)是一种灵活、无创、非侵入式的大脑监测方法,广泛应用于运动想象脑机接口系统中,运动想象脑电图识别精度是决定系统性能的关键因素。然而由于脑电图采集时间长、个体差异大等原因,导致单个受试者可用于模型训练的样本数量少,严重影响了卷积神经网络在脑电图识别任务中的表现。为此,本文提出一种镜卷积神经网络(mirror convolutional neural network,MCNN)模型,使用集成学习与数据扩增方法提高运动想象脑电图识别精度。方法 在训练阶段,基于源脑电通过互换左右侧脑电通道构造镜像脑电,并与源脑电一起用于源卷积网络训练,有效扩增了训练样本;在预测阶段,复制已训练源卷积网络作为镜像卷积网络,将测试集中的源脑电输入源卷积网络,构造的镜像脑电输入镜像卷积网络,集成源卷积网络与镜像卷积网络输出的类别预测概率,形成最终类别预测。结果 为了验证模型的有效性和通用性,基于3种不同运动想象脑电图识别卷积网络模型分别构造镜卷积网络,并在第4届脑机接口大赛2a与2b数据集上进行实验验证。实验结果与原始模型相比,运动想象四分类和二分类准确率分别平均提高了4.83%和4.61%,显著提高了识别精度。结论 本文面向运动想象脑电图识别,提出了镜卷积神经网络模型,通过集成学习与数据扩增方法提高运动想象识别精度,有效改善了运动想象脑机接口性能。
关键词
Mirror convolutional neural network for motor imagery electroencephalogram recognition

Luo Jing1,2, Wang Yaojie1,2, Liu Guangming1,2, Wang Xiaofan1,2, Lu Xiaofeng1,2, Hei Xinhong1,2(1. School of Computer Science and Engineering, Xi'an University of Technology, Xi'an 710048, China;2.
2. Shaanxi Key Laboratory for Network Computing and Security Technology, Xi'an 710048, China)

Abstract
Objective Apart from the neuromuscular system, the brain-computer interface (BCI) has provided an alternative way to convey the intention of the brain. For patients who have lost their ability to control their bodies, the BCI technique can recognize the state of the brain and send control order to external assistive devices,assisting patients in their daily lives. For healthy people, the BCI technique can greatly improve multimedia and video game experience and has promising commercial value. Electroencephalogram (EEG) is a flexible and noninvasive brain monitoring method that has been widely used in BCI systems based on motor imagery. The performance of such systems mainly depends on the classification accuracy of motor imagery EEG. In a motor imagery BCI system, EEG signals are recorded when a subject is imaging a specific movement, such as tongue, hand, or foot movement. Motor imagery is classified according to EEG signals. However, owing to long EEG collection time and obvious individual differences, the number of training samples belonging to one subject is small, seriously affecting the performance of convolutional neural network models in EEG recognition tasks. This paper proposes a mirror convolutional neural network (MCNN) that uses ensemble learning and data augmentation methods to improve the recognition accuracy of motor imagery EEG. Method The proposed MCNN model can be built on the basis of any motor imagery EEG recognition convolutional neural network (CNN) model. At the training stage, a sufficient number of samples ensure the successful training of an EEG recognition model. For a CNN-based method, training a CNN model needs large number of training samples to have good performance because of the numerous parameters that must be trained. However, the number of training samples in an EEG recognition task is usually small compared with that in a natural image classification task. Therefore, enlarging the number of EEG in a training set is a simple but efficient way to improve the effect of CNN model training. In this paper, we first proposed a mirror EEG construction method to enlarge a training set. A mirror EEG was constructed according to a source EEG by exchanging the left-side and right-side channels of the source EEG. For the left/right hand motor imagery-based EEG, the label of mirror EEG was set opposite to the label of the source EEG because event-related desynchronization and synchronization occur on the contralateral side of the brain. For other types of motor imagery EEG, for instance, feet or tongue motor imagery, the label of the mirror EEG was set in the same manner as the label of the source EEG. At the training stage, the source EEG and mirror EEG constructed on the basis of the source EEG were combined into an enlarged training set to train the source CNN model. This data augmentation method effectively expanded the training samples. At the prediction stage, MCNN improved the EEG recognition performance with the ensemble learning method. Specifically, the trained source CNN model is copied as the mirror CNN model. The source EEG is input into the source CNN model and the mirror EEG was input into the mirror CNN model. The average output category prediction probability of source CNN model and mirror CNN model was the final category prediction probability. In this way, the ensemble learning idea was applied without extra training session. Result To verify the effectiveness and universality of the proposed MCNN model, we constructed it according to three different motor imagery recognition CNN models, namely, the EEGNet, Shallow ConvNet, and Deep ConvNet. Experimental verification was performed on the Brain-Computer Interface Competition IV datasets 2a and 2b. Compared with the results obtained using the original model,the experimental results indicated that the accuracy of the four-category and two-category classification schemes for motor imagery increased by 4.83% and 4.59%, respectively, showing significant improvements. Enhanced performance was observed in different source CNN models, filters, and datasets. Conclusion The MCNN model was proposed for the EEG recognition of motor imagery BCI. The ideas of ensemble learning and data augmentation were applied to the design of the MCNN. The result showed that the MCNN considerably improved four-category and two-category motor imagery classification performance. Therefore, the proposed MCNN can greatly improve the performance of motor imagery-based BCIs.
Keywords

订阅号|日报