视觉Transformer与多特征融合的脑卒中检测算法
Cerebral stroke detection algorithm for visual Transformer and multi-feature fusion
- 2022年27卷第3期 页码:923-934
纸质出版日期: 2022-03-16 ,
录用日期: 2021-11-18
DOI: 10.11834/jig.210745
移动端阅览
浏览全部资源
扫码关注微信
纸质出版日期: 2022-03-16 ,
录用日期: 2021-11-18
移动端阅览
赵琛琦, 王华虎, 赵涓涓, 冀伦文, 王麒达, 李慧芝, 赵紫娟. 视觉Transformer与多特征融合的脑卒中检测算法[J]. 中国图象图形学报, 2022,27(3):923-934.
Chenqi Zhao, Huahu Wang, Juanjuan Zhao, Lunwen Ji, Qida Wang, Huizhi Li, Zijuan Zhao. Cerebral stroke detection algorithm for visual Transformer and multi-feature fusion[J]. Journal of Image and Graphics, 2022,27(3):923-934.
目的
2
急性缺血性卒中是最常见的脑卒中类型,具有发病率高、死亡率高和致残率高的特点。患者发病前症状不明显、发病急骤以及溶栓治疗时间窗窄等问题导致其成为临床上的高危疾病。中医望诊可以在疾病发展早期,通过观察患者形、色、气和神的变化,对患者病情进行诊断和预测,达到“治未病”的目的,与人工智能技术相结合,可以解决缺乏客观和定量评价标准的问题。因此,通过中医望诊中的脸部和手部图像,充分利用两种图像的颜色、纹理等特征以及二者之间的关系特征,本文提出一种基于序列自注意力网络的急性缺血性卒中辅助诊断方法。
方法
2
对脸部和手部图像进行山根和大鱼际处的感兴趣区域提取。采用$\rm YCbCr$颜色空间和灰度共生矩阵,提取区域图像的颜色和纹理特征,将颜色特征和纹理特征进行融合并将其与原图像特征相结合,得到的特征图序列化地输入到Transformer模型中,进一步学习高层次的空间特征和注意力特征。将模型输出结果输入到多层感知机中,从而实现急性缺血性卒中的检测。
结果
2
在收集的急性缺血性卒中患者数据集上进行实验,结果表明,提出的基于序列自注意力网络的方法取得了83.57%的准确率,获得较高性能,在速度和便携性上具有很大的优势。
结论
2
该方法采用端到端的学习方式,能够有效解决目前临床诊断因医疗资源的差异而受到影响的问题,对于初步判断患者疾病具有指导性的作用,为诊断急性缺血性卒中提供了一种新思路和新方法。
Objective
2
Cerebral ischemic stroke is the most common type of cerebral stroke
which is characterized by high morbidity
mortality and disability. The lack of obvious symptoms before the onset of the disease
the rapid onset of the disease
and the narrow time window for thrombolytic therapy have led to it being a high-risk disease in clinical practice. Although initial progress has been made in cerebral stroke prevention and treatment
it remains a significant cause of disability or death in adults. According to the survey
approximately 75% of stroke patients have varying degrees of functional impairment and loss of work
causing a heavy burden on families and society. With the accelerated aging and urbanization of society
the prevalence of unhealthy lifestyles among the population and the widespread exposure to cerebrovascular disease risk factors
in the disease burden of stroke has greatly increased
with a trend of rapid growth in low-income groups
marked gender and geographical differences and youthfulness. Therefore
effective ways to reduce disability and mortality rates should be developed. The early diagnosis of cerebral stroke is important. Many methods can be used to diagnose cerebral stroke in modern medicine
but the processes are relatively complex. In addition
some tests have certain drawbacks
and the presence of the disease is hard to detect in the early stages of illness
thus requiring advanced equipment and experienced clinicians. How to improve the accuracy of early diagnosis of cerebral stroke has become an important research hotspot for medical aid diagnosis. The characteristics and advantages of traditional Chinese medicine (TCM) are essential in the contemporary medical system of diseases
especially the inspection diagnosis of TCM
which is the most important in TCM diagnosis. Chinese medicine diagnosis is an objective and accurate empirical medicine
which has gradually formed and developed in long-term medical practice and clinically proven
with extremely rich connotations. Based on the basic principles of Chinese medicine diagnosis (the inspection diagnosis of TCM)
and diagnosis can be improved by applying modern scientific knowledge and methods in practice. This method not only provides strong evidence for early diagnosis and treatment
but also has extremely important practical significance in saving medical resources
reducing the medical burden on patients and alleviating the harm caused by cerebral stroke disease.
Method
2
First
feature extraction is performed on the images of the patient's face and hands. The color features are easily affected by light
and the chroma component in ${\rm{YCbCr}}$ color space is used to reduce the effect of luminance. The most important of the texture features are the features of texture length
depth and thickness in the images
and the gray level co-generation matrix (GLCM) was used to extract the image texture features effectively. Then
the higher-order spatial dimensional features further learned from the original image and the attentional features are learned from the different features by designing a reasonable dual Transformer joint classification model. Different transformer modules were cascaded
and multi-layer perception was used for image classification. This method not only considers color and texture features in the image
but also analyzes the spatial features of the image. Based on the differences arising from successive changes in color and texture between different regions in an image
this paper uses transformer to extract the attention features between different regions to improve the performance of the diagnostic model. In addition
the detection model is trained end-to-end. During the training process
the batch size is set to 4
the learning rate is set to 1E-5 and the maximum number of cycles is set to 100. The experiment uses NVIDIA TITAN XP GPU
and the data set was divided into five groups equally for five cross-validations. Finally
the average accuracy of all cross-validated results was taken as the final result of the experiment.
Result
2
When detecting cerebral ischemic stroke
the models with color features (${\rm{YCbCr}}$) and texture features (GLCM) extracted separately achieved accuracies of 79.40% and 80.46% on the dataset
while the model with the fusion of color and texture features achieved an accuracy of 83.53% on the dataset
which was significantly better than the model without feature fusion. Color features and texture features can effectively improve the classification accuracy in classification by using a transformer model
and feature fusion can make the model further improve the detection accuracy. Under the premise of fusion of color and texture features
the accuracy of model classification using a transformer module has dropped by approximately 2%. This finding shows that features from different parts play different roles in the final detection
and the gaps between the same features from different parts can easily disappear in the process of feature fusion into one transformer module. The dual transformer joint classification model uses color
texture
spatial and attention features
and the combination of these features can effectively improve the performance of the model. In addition
the average accuracy of the proposed model on the dataset in this paper outperforms the experimental results of related classification models.
Conclusion
2
In this paper
we proposed an end-to-end joint classification detection method based on the dual Transformer module. High-quality data were acquired using YCbCr color space and GLCM to accelerate the convergence process of the model. In addition
we extracted feature information from the patient's face and hand images. More importantly
the model learning capability was enhanced
and the model performance was improved using a self-attentive mechanism to learn the association between features and assign weights. The proposed model has a good diagnostic effect
and the automatic assisted diagnosis reduced the influence of subjective factors
which is valuable in the study of cerebral ischemic stroke auxiliary diagnosis
provides a reference for clinicians to make decisions on cerebral ischemic stroke disease diagnosis and provides a new method for patients to conduct effective self-screening.
中医望诊特征提取特征融合端到端Transformer
inspection diagnosis of traditional Chinese medicinefeature extractionfeature fusionend-to-endTransformer
Boling B and Keinath K. 2018. Acute ischemic stroke. AACN Advanced Critical Care, 29(2): 152-162[DOI: 10.4037/aacnacc2018483]
Chen Y W. 2019. Face Detection and Facial Landmark Localization Based on Improved MTCNN Model. Shanghai: Donghua University
陈雨薇. 2019. 基于改进MTCNN模型的人脸检测与面部关键点定位. 上海: 东华大学
Cordonnier J B, Loukas A and Jaggi M. 2020. Multi-head attention: collaborate instead of concatenate[EB/OL]. [2020-01-29].https://arxiv.org/pdf/2006.16362v1.pdfhttps://arxiv.org/pdf/2006.16362v1.pdf
Gao J M, Lyu M, Xie W W, Liu X Y, Zhao B C and Zhu Y. 2019. Regularity of traditional Chinese medicine prescriptions for same treatment for cardiovascular and cerebrovascular diseases. China Journal of Chinese Materia Medica, 44(1): 193-198
高佳明, 吕明, 解微微, 刘昕彦, 赵步长, 朱彦. 2019. 中医药心脑血管疾病同治的方剂用药规律分析. 中国中药杂志, 44(1): 193-198[DOI: 10.19540/j.cnki.cjcmm.20181101.007]
Gao L, Wang P P and Li N. 2007. Observation for cause: clinical considerations in a case of stroke. Chinese Journal of Integrated Traditional and Western Medicine in Intensive and Critical Care, 14(4): 252-253
高利, 王平平, 李宁. 2007. 望诊寻因——1例脑卒中患者引发的临床思考. 中国中西医结合急救杂志, 14(4): 252-253[DOI: 10.3321/j.issn:1008-9691.2007.04.026]
Gao Y R, Han X J, Wang L Y, Liu D S and Ren C. 2020. Research progress of thenar inspection. China Journal of Traditional Chinese Medicine and Pharmacy, 35(8): 4052-4054
郜亚茹, 韩学杰, 王丽颖, 刘大胜, 任聪. 2020. 大鱼际望诊法研究进展. 中华中医药杂志, 35(8): 4052-4054
Hao P P, Jiang F, Chen Y G, Yang J M, Zhang K, Zhang M X, Zhang C, Zhao Y X and Zhang Y. 2015. Traditional Chinese medication for cardiovascular disease. Nature Reviews Cardiology, 12(6): #318[DOI: 10.1038/nrcardio.2015.60]
Hsieh M J, Chen Y J, Tang S C, Chen J H, Lin L C, Seak C J, Lee J T, Chang K C, Lien L M, Chan L, Liu C H, Hsieh C Y, Chern C M, Chen J C, Chiu T F, Hung S C, Ng C J and Jeng J S. 2021. 2020 Guideline for Prehospital management, emergency evaluation and treatment of patients with acute ischemic stroke: a guideline for healthcare professionals from the Taiwan society of emergency medicine and Taiwan stroke society. Journal of Acute Medicine, 11(1): 12-17[DOI: 10.6705/j.jacme.202103_11(1).0002]
Karthik R and Menaka R. 2018. Computer-aided detection and characterization of stroke lesion—a short review on the current state-of-the art methods. The Imaging Science Journal, 66(1): 1-22[DOI: 10.1080/13682199.2017.1370879]
Lin Y. 2020. Complexion classification based on convolutional neural network. Journal of Artificial Intelligence Practice, 3(1): 22-30[DOI: 10.23977/jaip.2020.030105]
Liu Z W. 2015. Research on Stroke Recurrence Prediction Based on Machine Learning. Changsha: Hunan University
刘泽文. 2015. 基于机器学习的脑卒中复发预测模型研究. 长沙: 湖南大学
Luo X Z, Wen X P, He J Y, Huang J T and Tang C Z. 2017. Analysis of the influencing factors of post-stroke depression: based on machine learning. Journal of Traditional Chinese Medicine, 58(17): 1478-1481
罗晓舟, 温小鹏, 何家扬, 黄健婷, 唐纯志. 2017. 基于机器学习的卒中后抑郁影响因素分析. 中医杂志, 58(17): 1478-1481[DOI: 10.13288/j.11-2166/r.2017.17.011]
Luo Y S, Shao Y Y and Chen D H. 2020. Diagnosis model of ischemic stroke based on LSTM with multi-feature combination. Intelligent Computer And Applications, 10(10): 74-79
骆轶姝, 邵圆圆, 陈德华. 2020. 基于LSTM多特征联合的缺血性脑卒中诊断模型. 智能计算机与应用, 10(10): 74-79[DOI: 10.3969/j.issn.2095-2163.2020.10.018]
Seshadri S and Wolf P A. 2007. Lifetime risk of stroke and dementia: current concepts, and estimates from the Framingham study. The Lancet Neurology, 6(12): 1106-1114[DOI: 10.1016/S1474-4422(07)70291-0]
Shaik K B, Ganesan P, Kalist V, Sathish B S and Jenitha J M M. 2015. Comparative study of skin color detection and segmentation in HSV and YCbCr color space. Procedia Computer Science, 57: 41-48[DOI: 10.1016/j.procs.2015.07.362]
Strong K, Mathers C and Bonita R. 2007. Preventing stroke: saving lives around the world. The Lancet Neurology, 6(2): 182-187[DOI: 10.1016/S1474-4422(07)70031-5]
Virani S S, Alonso A, Aparicio H J, Benjamin E J, Bittencourt M S, Callaway C W, Carson A P, Chamberlain A M, Cheng S, Delling F N, Elkind M S V, Evenson K R, Ferguson J F, Gupta D K, Khan S S, Kissela B M, Knutson K L, Lee C D, Lewis T T, Liu J X, Loop M S, Lutsey P L, Ma J, Mackey J, Martin S S, Matchar D B, Mussolino M E, Navaneethan S D, Perak A M, Roth G A, Samad Z, Satou G M, Schroeder E B, Shah S H, Shay C M, Stokes A, VanWagner L B, Wang N Y and Tsao C W. 2021. Heart disease and stroke statistics—2021 update: a report from the American heart association. Circulation, 143(8): e254-e743[DOI: 10.1161/CIR.0000000000000950]
Wu M H, Song R R and Liu M. 2017. Video shadow elimination algorithm combining HSV with texture feature. Journal of Image and Graphics, 22(10): 1373-1380
武明虎, 宋冉冉, 刘敏. 2017. 结合HSV与纹理特征的视频阴影消除算法. 中国图象图形学报, 22(10): 1373-1380[DOI: 10.11834/jig.170151].
Xi L. 2016. Classification of Stroke Based on Microwave Experiment Platform. Shanghai: Donghua University
席恋. 2016. 基于微波实验平台的脑中风分类判断研究. 上海: 东华大学
Yu C Y, Wen L F, Yang G and Wang Y T. 2019. Video person reidentification based on BiLSTM and attention mechanism. Journal of Image and Graphics, 24(10): 1703-1710
余晨阳, 温林凤, 杨钢, 王玉涛. 2019. 结合BiLSTM和注意力机制的视频行人再识别. 中国图象图形学报, 24(10): 1703-1710[DOI: 10.11834/jig.190637]
Zhang C, Li W W, Xiao J and Sun C L. 2020. Application of TCM inspection in clinical practice. Henan Traditional Chinese Medicine, 40(6): 839-843
张超, 李唯薇, 肖静, 孙成力. 2020. 中医望诊在临床中的应用. 河南中医, 40(6): 839-843[DOI: 10.16367/j.issn.1003-5028.2020.06.0212]
Zhang J, Wang C, Zhuo L and Yang Y C. 2014. Uniform color space based facial complexion recognition for traditional Chinese medicine//Proceedings of the 13th International Conference on Control Automation Robotics and Vision. Singapore, Singapore: IEEE: 631-636[DOI: 10.1109/ICARCV.2014.7064377http://dx.doi.org/10.1109/ICARCV.2014.7064377]
Zhang W T, Gao K, Liu J W, Zhao H H, Wang J A, Li Y B, Murtaza G, Chen J X and Wang W. 2013. A review of the pharmacological mechanism of traditional Chinese medicine in the intervention of coronary heart disease and stroke. African Journal of Traditional, Complementary and Alternative Medicines, 10(6): 532-537[DOI: 10.4314/ajtcam.v10i6.24]
相关作者
相关机构