浏览全部资源
扫码关注微信
导出
分享
收藏
专辑
收稿：2021-08-10，
修回：2022-1-24，
录用：2022-1-31，
纸质出版：2023-02-16
移动端阅览
目的
2
在脑科学领域，已有研究借助脑功能核磁共振影像数据（functional magnetic resonance imaging，fMRI）探索和区分人类大脑在不同运动任务下的状态，然而传统方法没有充分利用fMRI数据的时序特性。对此，本文提出基于fMRI数据计算的全脑脑区时间信号（time course，TC）的门控循环单元（gated recurrent unit，GRU）方法（TC-GRU）进行运动任务分类。
方法
2
基于HCP（human connectome project）数据集中的100个健康被试者在5种运动任务中分两轮采集的1 000条fMRI数据，对每种运动任务计算每个被试者在各脑区（共360个脑区）的时间信号；使用10折交叉验证方案基于训练集和验证集训练TC-GRU模型，并用构建好的模型对测试集进行测试，考察其对5种运动任务的分类能力，其中TC-GRU在各时刻的输入特征为全脑脑区在对应时刻的TC信号幅值，通过这样的方式提取全脑脑区在整个时间段的时序特征。同时，为了展示使用TC-GRU模型可挖掘fMRI数据中更丰富的信息，设计了多个对比实验进行比较，利用长短期记忆网络（long short-term memory，LSTM）、图卷积网络（graph convolutional network，GCN）和多层感知器（multi-layer perceptron，MLP）基于全脑脑区时间信号进行运动任务分类，以及利用MLP基于由fMRI数据估计的脑功能连接进行运动任务分类。此外，考察了先验的特征选择对分类效果的效应。
结果
2
基于全脑脑区时间信号的TC-GRU模型在运动任务中的分类准确率最高，为94.51%±2.4%，其次是基于全脑脑区时间信号的LSTM模型，准确率为93.73%±2.67%。基于全脑脑区时间信号利用MLP进行分类，有先验和无先验的特征选择准确率分别为92.75%±2.59%和92.04%±7.15%，比基于全脑脑区时间信号的GCN（准确率为87.14%±3.73%）和基于脑功能连接利用MLP进行分类（有先验和无先验的特征选择准确率分别为72.47%±4.47%和61.49%±9.97%）表现更好。
结论
2
TC-GRU模型可挖掘脑fMRI数据中丰富的时序信息，非常有效地对不同的运动任务进行分类。
Objective
2
In the field of neuroscience
there have been studies using functional magnetic resonance imaging (fMRI) data to explore functions of the human brain and distinguish its states under different motor tasks. However
previous studies that focused on the brain state classification using task fMRI did not make full use of temporal characteristics of fMRI data. Here
we propose a method (named TC-GRU) that employs gated recurrent unit (GRU) to capture fine-grained features from time courses (TC) of whole-brain regions estimated from fMRI data for the classification of motor tasks.
Method
2
The fMRI data are gathered from 100 healthy subjects in the human connectome project (HCP) under 5 body-motion tasks (including left hand
right hand
left foot
right foot
and tongue motor) with 2 scanning operations
resulting in 1 000 samples for classifying the 5 motor tasks. First
for each sample
we calculate the average fMRI TC for each brain region as the representative TC of the brain region. The whole brain is divided into 360 brain regions according to the Glasser brain template. Then
using a 10-fold cross-validation framework (8:1:1 for the training set
the validation set
and the testing set) with 100 repetitions
the TC-GRU model is trained and optimized based on the training set and the validation set
and the model-trained is further applied to the testing set to examine its ability in classifying this 5 body motor tasks. In our TC-GRU model
the GRU is used to extract the temporal features in the TCs of the brain regions
and a linear classifier is used for classification based on the temporal features. Specifically
at a certain moment
the inputs of the GRU model are the TC amplitudes of the whole-brain regions at that moment as well as the temporal features of the past moments captured by the GRU model
and the GRU model fuses the inputs and produces the temporal features at the current time. This process continues until the last moment to generate temporal features for the classification. In our work
we also compare the most state-of-the-art methods with the TC-GRU. The long short-term memory (LSTM)
graph convolutional network (GCN)
and multi-layer perceptron (MLP) are used to classify the motor tasks based on TCs of whole-brain regions as well as brain functional connectivity measures estimated by the fMRI data. Furthermore
we examine the effects of prior feature selection and no feature selection on the classification performance. It is noteworthy that a consistent 10-fold cross-validation framework is used for multiple methods and the overall classification accuracy is summarized through 100 cross-validation tests. The overall classification accuracy is the mean classification accuracy
and the performance stability is reflected by the standard deviation of the classification accuracy.
Result
2
The highest ranking of the classification accuracy (accuracy: 94.51%±2.4%) can be achieved via the TC-GRU method
and the second rank is the LSTM using TC information (accuracy: 93.73%±2.67%). Using MLP based on the TCs of whole-brain regions (accuracy from the experiments with prior feature selection and without prior feature selection is 92.75%±2.59% and 92.04%±7.15%
respectively) is better than using GCN (accuracy: 87.14%±3.73%) based on the TCs of whole-brain regions and MLP based on the brain functional connectivity measures (accuracy from experiments with prior feature selection and without prior feature selection is 72.47%±4.47% and 61.49%±9.97%
respectively).
Conclusion
2
To the best of our knowledge
this paper is the first time to distinguish different human brain motor tasks using GRU based on time courses of the whole-brain regions. Our results support that the TC-GRU method outperforms six state-of-the-art methods on human brain motor task classification because that the TC-GRU can mine more useful information in the brain fMRI data. In summary
our finding suggests the importance of utilizing temporal information of fMRI data to decode the complex brain.
Anzellotti S and Coutanche M N. 2018. Beyond functional connectivity: investigating networks of multivariate representations. Trends in Cognitive Sciences, 22(3): 258-269[DOI: 10.1016/j.tics.2017.12.002]
Barch D M, Burgess G C, Harms M P, Petersen S E, Schlaggar B L, Corbetta M, Glasser M F, Curtiss S, Dixit S, Feldt C, Nolan D, Bryant E, Hartley T, Footer O, Bjork J M, Poldrack R, Smith S, Johansen-Berg H, Snyder A Z and van Essen D C. 2013. Function in the human connectome: task-fMRI and individual differences in behavior. NeuroImage, 80: 169-189[DOI: 10.1016/j.neuroimage.2013.05.033]
Cho K, van Merrienboer B, Gulcehre C, Bahdanau D, Bougares F, Schwenk H and Bengio Y. 2014. Learning phrase representations using RNN encoder-decoder for statistical machine translation//Proceedings of 2014 Conference on Empirical Methods in Natural Language Processing. Doha, Qatar: Association for Computational Linguistics: 1724-1734[ DOI: 10.3115/v1/D14-1179 http://dx.doi.org/10.3115/v1/D14-1179 ]
Gao Y F, Zhang Y M, Wang H L, Guo X J and Zhang J C. 2019. Decoding behavior tasks from brain activity using deep transfer learning. IEEE Access, 7: 43222-43232[DOI: 10.1109/access.2019.2907040]
Glasser M F, Coalson T S, Robinson E C, Hacker C D, Harwell J, Yacoub E, Ugurbil K, Andersson J, Beckmann C F, Jenkinson M, Smith S M and van Essen D C. 2016. A multi-modal parcellation of human cerebral cortex. Nature, 536(7615): 171-178[DOI: 10.1038/nature18933]
Glasser M F, Sotiropoulos S N, Wilson J A, Coalson T S, Fischl B, Andersson J L, Xu J Q, Jbabdi S, Webster M, Polimeni J R, van Essen D C and Jenkinson M. 2013. The minimal preprocessing pipelines for the Human Connectome Project. NeuroImage, 80: 105-124[DOI: 10.1016/j.neuroimage.2013.04.127]
Gonzalez-Castillo J, Hoy C W, Handwerker D A, Robinson M E, Buchanan L C, Saad Z S and Bandettini P A. 2015. Tracking ongoing cognition in individuals using brief, whole-brain functional connectivity patterns. Proceedings of the National Academy of Sciences of the United States of America, 112(28): 8762-8767[DOI: 10.1073/pnas.1501242112]
Hebart M N and Baker C I. 2018. Deconstructing multivariate decoding for the study of brain function. NeuroImage, 180: 4-18[DOI: 10.1016/j.neuroimage.2017.08.005]
Liu W W, Su Y L, Wunier and Renqingdaoerji. 2018. Mongolian-Chinese machine translation research based on part of speech tagging with gated unit neural network. Journal of Chinese Information Processing, 32(8): 68-74
刘婉婉, 苏依拉, 乌尼尔, 仁庆道尔吉. 2018. 基于门控循环神经网络词性标注的蒙汉机器翻译研究. 中文信息学报, 32(8): 68-74[DOI: 10.3969/j.issn.1003-0077.2018.08.010]
Mensch A, Mairal J, Bzdok D, Thirion B and Varoquaux G. 2017. Learning neural representations of human cognition across many fMRI studies//Proceedings of the 31st International Conference on Neural Information Processing Systems. Long Beach, USA: Curran Associates Inc. : 5885-5895[ DOI: 10.5555/3295222.3295338 http://dx.doi.org/10.5555/3295222.3295338 ]
Qi Y, Lin H W, Li Y P and Chen J S. 2021. Parameter-free attention in fMRI decoding. IEEE Access, 9: 48704-48712[DOI: 10.1109/ACCESS.2021.3068921]
Sutskever I, Vinyals O and Le Q V. 2014. Sequence to sequence learning with neural networks//Proceedings of the 27th International Conference on Neural Information Processing Systems. Montreal, Canada: MIT Press: 3104-3112[ DOI: 10.5555/2969033.2969173 http://dx.doi.org/10.5555/2969033.2969173 ]
Tan Y M, Liu S W and Lyu X Q. 2018. CNN and BiLSTM based Chinese textual entailment recognition. Journal of Chinese Information Processing, 32(7): 11-19
谭咏梅, 刘姝雯, 吕学强. 2018. 基于CNN与双向LSTM的中文文本蕴含识别方法. 中文信息学报, 32(7): 11-19[DOI: 10.3969/j.issn.1003-0077.2018.07.002]
Thomas A W, Müller K R and Samek W. 2019.Deep transfer learning for whole-brain FMRI analyses//Zhou L P, Sarikaya D, Kia S M, Speidel S, Malpani A, Hashimoto D, Habes M, Löfstedt T, Ritter K and Wang H Z, eds. OR 2.0 Context-Aware Operating Theaters and Machine Learning in Clinical Neuroimaging. Cham, Germany: Springer: 59-67[ DOI: 10.1007/978-3-030-32695-1_7 http://dx.doi.org/10.1007/978-3-030-32695-1_7 ]
Wang D X, Cui P and Zhu W W. 2016. Structural deep network embedding//Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. San Francisco, USA: ACM: 1225-1234[ DOI: 10.1145/2939672.2939753 http://dx.doi.org/10.1145/2939672.2939753 ]
Wang X X, Liang X, Jiang Z F, Nguchu B A, Zhou Y W, Wang Y M, Wang H J, Li Y, Zhu Y Y, Wu F, Gao J H and Qiu B S. 2020. Decoding and mapping task states of the human brain via deep learning. Human Brain Mapping, 41(6): 1505-1519[DOI: 10.1002/hbm.24891]
Yan W Z, Zhao M, Fu Z N, Pearlson G D, Sui J and Calhoun V D. 2022. Mapping relationships among schizophrenia, bipolar and schizoaffective disorders: a deep classification and clustering framework using fMRI time series. Schizophrenia Research, 245: 141-150[DOI: 10.1016/j.schres.2021.02.007]
Yang Z and Zuo X N. 2015. Big neuroimaging data-informed mind-brain association studies: methodology and applications. Chinese Science Bulletin, 60(11): 966-975
杨志, 左西年. 2015. 神经影像大数据与心脑关联: 方法学框架与应用. 科学通报, 60(11): 966-975[DOI: 10.1360/N972014-00806]
Yarkoni T, Poldrack R A, Nichols T E, van Essen D C and Wager T D. 2011. Large-scale automated synthesis of human functional neuroimaging data. Nature Methods, 8(8): 665-670[DOI: 10.1038/nmeth.1635]
Zhang Y, Tetrel L, Thirion B and Bellec P. 2021. Functional annotation of human cognitive states using deep graph convolution. NeuroImage, 231: #117847[DOI: 10.1016/j.neuroimage.2021.117847]
Zhuang L S, Lyu Y, Yang J and Li H Q. 2019. Long term recurrent neural network with state-frequency memory. Journal of Computer Research and Development, 56(12): 2641-2648
庄连生, 吕扬, 杨健, 李厚强. 时频联合长时循环神经网络. 计算机研究与发展, 56(12): 2641-2648[DOI: 10.7544/issn1000-1239.2019.20180474]
相关作者
相关机构
京公网安备11010802024621
