浏览全部资源
扫码关注微信
收稿日期：2021-02-07，
修回日期：2021-06-01，
录用日期：2021-6-8，
纸质出版日期：2021-09-16
移动端阅览
目的
2
脑电图（electroencephalogram，EEG）是一种灵活、无创、非侵入式的大脑监测方法，广泛应用于运动想象脑机接口系统中，运动想象脑电图识别精度是决定系统性能的关键因素。然而由于脑电图采集时间长、个体差异大等原因，导致单个受试者可用于模型训练的样本数量少，严重影响了卷积神经网络在脑电图识别任务中的表现。为此，本文提出一种镜卷积神经网络（mirror convolutional neural network，MCNN）模型，使用集成学习与数据扩增方法提高运动想象脑电图识别精度。
方法
2
在训练阶段，基于源脑电通过互换左右侧脑电通道构造镜像脑电，并与源脑电一起用于源卷积网络训练，有效扩增了训练样本；在预测阶段，复制已训练源卷积网络作为镜像卷积网络，将测试集中的源脑电输入源卷积网络，构造的镜像脑电输入镜像卷积网络，集成源卷积网络与镜像卷积网络输出的类别预测概率，形成最终类别预测。
结果
2
为了验证模型的有效性和通用性，基于3种不同运动想象脑电图识别卷积网络模型分别构造镜卷积网络，并在第4届脑机接口大赛2a与2b数据集上进行实验验证。实验结果与原始模型相比，运动想象四分类和二分类准确率分别平均提高了4.83%和4.61%，显著提高了识别精度。
结论
2
本文面向运动想象脑电图识别，提出了镜卷积神经网络模型，通过集成学习与数据扩增方法提高运动想象识别精度，有效改善了运动想象脑机接口性能。
Objective
2
Apart from the neuromuscular system
the brain-computer interface (BCI) has provided an alternative way to convey the intention of the brain. For patients who have lost their ability to control their bodies
the BCI technique can recognize the state of the brain and send control order to external assistive devices
assisting patients in their daily lives. For healthy people
the BCI technique can greatly improve multimedia and video game experience and has promising commercial value. Electroencephalogram (EEG) is a flexible and noninvasive brain monitoring method that has been widely used in BCI systems based on motor imagery. The performance of such systems mainly depends on the classification accuracy of motor imagery EEG. In a motor imagery BCI system
EEG signals are recorded when a subject is imaging a specific movement
such as tongue
hand
or foot movement. Motor imagery is classified according to EEG signals. However
owing to long EEG collection time and obvious individual differences
the number of training samples belonging to one subject is small
seriously affecting the performance of convolutional neural network models in EEG recognition tasks. This paper proposes a mirror convolutional neural network (MCNN) that uses ensemble learning and data augmentation methods to improve the recognition accuracy of motor imagery EEG.
Method
2
The proposed MCNN model can be built on the basis of any motor imagery EEG recognition convolutional neural network (CNN) model. At the training stage
a sufficient number of samples ensure the successful training of an EEG recognition model. For a CNN-based method
training a CNN model needs large number of training samples to have good performance because of the numerous parameters that must be trained. However
the number of training samples in an EEG recognition task is usually small compared with that in a natural image classification task. Therefore
enlarging the number of EEG in a training set is a simple but efficient way to improve the effect of CNN model training. In this paper
we first proposed a mirror EEG construction method to enlarge a training set. A mirror EEG was constructed according to a source EEG by exchanging the left-side and right-side channels of the source EEG. For the left/right hand motor imagery-based EEG
the label of mirror EEG was set opposite to the label of the source EEG because event-related desynchronization and synchronization occur on the contralateral side of the brain. For other types of motor imagery EEG
for instance
feet or tongue motor imagery
the label of the mirror EEG was set in the same manner as the label of the source EEG. At the training stage
the source EEG and mirror EEG constructed on the basis of the source EEG were combined into an enlarged training set to train the source CNN model. This data augmentation method effectively expanded the training samples. At the prediction stage
MCNN improved the EEG recognition performance with the ensemble learning method. Specifically
the trained source CNN model is copied as the mirror CNN model. The source EEG is input into the source CNN model and the mirror EEG was input into the mirror CNN model. The average output category prediction probability of source CNN model and mirror CNN model was the final category prediction probability. In this way
the ensemble learning idea was applied without extra training session.
Result
2
To verify the effectiveness and universality of the proposed MCNN model
we constructed it according to three different motor imagery recognition CNN models
namely
the EEGNet
Shallow ConvNet
and Deep ConvNet. Experimental verification was performed on the Brain-Computer Interface Competition Ⅳ datasets 2a and 2b. Compared with the results obtained using the original model
the experimental results indicated that the accuracy of the four-category and two-category classification schemes for motor imagery increased by 4.83% and 4.59%
respectively
showing significant improvements. Enhanced performance was observed in different source CNN models
filters
and datasets.
Conclusion
2
The MCNN model was proposed for the EEG recognition of motor imagery BCI. The ideas of ensemble learning and data augmentation were applied to the design of the MCNN. The result showed that the MCNN considerably improved four-category and two-category motor imagery classification performance. Therefore
the proposed MCNN can greatly improve the performance of motor imagery-based BCIs.
Ang K K, Chin Z Y, Wang C C, Guan C T and Zhang H H. 2012. Filter bank common spatial pattern algorithm on BCI competition Ⅳ datasets 2a and 2b. Frontiers in Neuroscience, 6: #39[DOI: 10.3389/fnins.2012.00039]
Clevert D A, Unterthiner T and Hochreiter S. 2015. Fast and accurate deep network learning by exponential linear units (ELUs)[EB/OL]. [2021-01-07] . https://arxiv.org/pdf/1511.07289.pdf https://arxiv.org/pdf/1511.07289.pdf
Dai G H, Zhou J, Huang J H and Wang N. 2020. HS-CNN: a CNN with hybrid convolution scale for EEG motor imagery classification. Journal of Neural Engineering, 17(1): #016025[DOI: 10.1088/1741-2552/ab405f]
Edelman B J, Meng J, Suma D, Zurn C, Nagarajan E, Baxter B S, Cline C C and He B. 2019. Noninvasive neuroimaging enhances continuous neural tracking for robotic device control. Science Robotics, 4(31): #6844[DOI: 10.1126/scirobotics.aaw6844]
Ehrsson H H, Naito E and Roland P E. 2001. Activation of human motor cortices during mental motor imagery of hand, foot and tongue movements. NeuroImage, 13(S6): #1158[DOI: 10.1016/S1053-8119(01)92479-6]
Ioffe S and Szegedy C. 2015. Batch normalization: accelerating deep network training by reducing internal covariate shift[EB/OL]. [2021-02-11] . https://arxiv.org/pdf/1502.03167.pdf https://arxiv.org/pdf/1502.03167.pdf
Kingma D P and Ba J L. 2014. Adam: a method for stochastic optimization[EB/OL]. [2021-04-23] . https://arxiv.org/pdf/1412.6980.pdf https://arxiv.org/pdf/1412.6980.pdf
Kumar S U and Inbarani H H. 2017. PSO-based feature selection and neighborhood rough set-based classification for BCI multiclass motor imagery task. Neural Computing and Applications, 28(11): 3239-3258[DOI: 10.1007/s00521-016-2236-5]
Lawhern V J, Solon A J, Waytowich N R, Gordon S M, Hung C P and Lance B J. 2018. EEGNet: a compact convolutional neural network for EEG-based brain-computer interfaces. Journal of Neural Engineering, 15(5): #056013[DOI: 10.1088/1741-2552/aace8c]
Lemm S, Blankertz B, Curio G and Muller K R. 2005. Spatio-spectral filters for improving the classification of single trial EEG. IEEE Transactions on Biomedical Engineering, 52(9): 1541-1548[DOI: 10.1109/TBME.2005.851521]
Luo J, Feng Z R, Zhang J and Lu N. 2016. Dynamic frequency feature selection based approach for classification of motor imageries. Computers in Biology and Medicine, 75: 45-53[DOI: 10.1016/j.compbiomed.2016.03.004]
Luo J, Wang J, Xu R and Xu K L. 2019. Class discrepancy-guided sub-band filter-based common spatial pattern for motor imagery classification. Journal of Neuroscience Methods, 323: 98-107[DOI: 10.1016/j.jneumeth.2019.05.011]
Luo J, Gao X, Zhu X B, Wang B, Lu N and Wang J. 2020. Motor imagery EEG classification based on ensemble support vector learning. Computer Methods and Programs in Biomedicine, 193: #105464[DOI: 10.1016/j.cmpb.2020.105464]
Meirovitch Y, Harris H, Dayan E Arieli A and Flash T. 2015. Alpha and beta band event-related desynchronization reflects kinematic regularities. Journal of Neuroscience, 35(4): 1627-1637[DOI: 10.1523/JNEUROSCI.5371-13.2015]
Meng M, Zhu J Q, She Q S, Ma Y L and Luo Z Z. 2016. Two-level feature extraction method for multi-class motor imagery EEG. Acta Automatica Sinica, 42(12): 1915-1922
孟明, 朱俊青, 佘青山, 马玉良, 罗志增. 2016. 多类运动想象脑电信号的两级特征提取方法. 自动化学报, 42(12): 1915-1922)[DOI: 10.16383/j.aas.2016.c160122]
Morioka H, Kanemura A, Hirayama J I, Shikauchi M, Ogawa T, Ikeda S, Kawanabe M and Ishii S. 2015. Learning a common dictionary for subject-transfer decoding with resting calibration. NeuroImage, 111: 167-178[DOI: 10.1016/j.neuroimage.2015.02.015]
O'Shea A, Lightbody G, Boylan G and Temko A. 2017. Neonatal seizure detection using convolutional neural networks//Proceedings of the 27th IEEE International Workshop on Machine Learning for Signal Processing (MLSP). Tokyo, Japan: IEEE: 1-6[ DOI: 10.1109/MLSP.2017.8168193 http://dx.doi.org/10.1109/MLSP.2017.8168193 ]
Paszke A, Gross S, Massa F, Lerer A, Bradbury J, Chanan G, Killeen T, Lin Z M, Gimelshein N, Antiga L, Desmaison A, Köpf A, Yang E, DeVito Z, Raison M, Tejani A, Chilamkurthy S, Steiner B, Fang L, Bai J J and Chintala S. 2019. PyTorch: an imperative style, high-performance deep learning library[EB/OL]. [2021-01-07] . https://arxiv.org/pdf/1912.01703v1.pdf https://arxiv.org/pdf/1912.01703v1.pdf
Penaloza C I and Nishio S. 2018. BMI control of a third arm for multitasking. Science Robotics, 3(20): #1228[DOI: 10.1126/scirobotics.aat1228]
Pfurtscheller G and da Silva F H L. 1999. Event-related EEG/MEG synchronization and desynchronization: basic principles. Clinical Neurophysiology, 110(11): 1842-1857[DOI: 10.1016/S1388-2457(99)00141-8]
Sakhavi S, Guan C T and Yan S C. 2015. Parallel convolutional-linear neural network for motor imagery classification//Proceedings of the 23rd European Signal Processing Conference (EUSIPCO). Nice, France: IEEE: 2736-2740[ DOI: 10.1109/EUSIPCO.2015.7362882 http://dx.doi.org/10.1109/EUSIPCO.2015.7362882 ]
Schirrmeister R T, Springenberg J T, Fiederer L D J, Glasstetter M, Eggensperger K, Tangermann M, Hutter F, Burgard W and Ball T. 2017. Deep learning with convolutional neural networks for EEG decoding and visualization. Human Brain Mapping, 38(11): 5391-5420[DOI: 10.1002/hbm.23730]
Srivastava N, Hinton G, Krizhevsky A, Sutskever I and Salakhutdinov R. 2014. Dropout: a simple way to prevent neural networks from overfitting. The Journal of Machine Learning Research, 15(1): 1929-1958
Tabar Y R and Halici U. 2017. A novel deep learning approach for classification of EEG motor imagery signals. Journal of Neural Engineering, 14(1): #016003[DOI: 10.1088/1741-2560/14/1/016003]
Tangermann M, Müller K R, Aertsen A, Birbaumer N, Braun C, Brunner C, Leeb R, Mehring C, Miller K J, Müller-Putz G R, Nolte G, Pfurtscheller G, Preissl H, Schalk G, Schlögl A, Vidaurre C, Waldert S and Blankertz B. 2012. Review of the BCI competition Ⅳ. Frontiers in Neuroscience, 6: #55[DOI: 10.3389/fnins.2012.00055]
Ullah I, Hussain M, Qazi E U H and Aboalsamh H. 2018. An automated system for epilepsy detection using EEG brain signals based on deep learning approach. Expert Systems with Applications, 107: 61-71[DOI: 10.1016/j.eswa.2018.04.021]
Wang F, Zhong S H, Peng J F, Jiang J M and Liu Y. 2018. Data augmentation for EEG-based emotion recognition with deep convolutional neural networks//Proceedings of the 24th International Conference, MMM 2018. Bangkok, Thailand: MMM: 82-93[ DOI: 10.1007/978-3-319-73600-6_8 http://dx.doi.org/10.1007/978-3-319-73600-6_8 ]
Yang M H, Chen W Z and Li M Y. 2017. Multiple feature extraction based on ensemble empirical mode decomposition for motor imagery EEG recognition tasks. Acta Automatica Sinica, 43(5): 743-752
杨默涵, 陈万忠, 李明阳. 2017. 基于总体经验模态分解的多类特征的运动想象脑电识别方法研究. 自动化学报, 43(5): 743-752[DOI: 10.16383/j.aas.2017.c160175]
Zhang Q Q and Liu Y. 2018. Improving brain computer interface performance by data augmentation with conditional Deep Convolutional Generative Adversarial Networks[EB/OL]. [2021-01-07] . https://arxiv.org/pdf/1806.07108.pdf https://arxiv.org/pdf/1806.07108.pdf
Zheng M M, Yang B H and Xie Y L. 2020. EEG classification across sessions and across subjects through transfer learning in motor imagery-based brain-machine interface system. Medical and Biological Engineering and Computing, 58(7): 1515-1528[DOI: 10.1007/s11517-020-02176-y]
Zheng Y F and Chen B D. 2019. Least p-norm based broad learning system. Pattern Recognition and Artificial Intelligence, 32(1): 51-57
郑云飞, 陈霸东. 2019. 基于最小p-范数的宽度学习系统. 模式识别与人工智能, 32(1): 51-57[DOI: 10.16451/j.cnki.issn1003-6059.201901007]
Zhou Z H. 2012. Ensemble Methods: Foundations and Algorithms. Boca Raton: CRC Press[DOI:10.1201/b12207]
京公网安备11010802024621