浏览全部资源
扫码关注微信
收稿：2021-07-26，
修回：2021-10-14，
录用：2021-10-21，
纸质出版：2022-11-16
移动端阅览
目的
2
高精度图像分割是生物医学图像处理中的一个重要问题。在磁共振成像过程中，噪声和强度不均匀很大程度影响图像分割的精度。因此，提出了一种基于相异性准则熵率超像素的多模态高精度图像分割网络。
方法
2
采用熵率超像素分割算法对多模态图像进行预分割得到超像素块，提出新的融合算法对其重新编号，建立超像素图，该图中的每一个超像素块构成无向图的一个结点；利用每个结点的灰度值提取特征向量，通过相异性权重判断结点间的相关性，构建相邻结点的特征序列；将特征序列作为双向长短期记忆模型（bi-directional long short-term memory，BiLSTM）的输入，经过训练和测试，得到最终的分割结果。
结果
2
本文方法在BrainWeb、MRBrainS和BraTS2017数据集上与主流算法进行了对比。在BrainWeb数据集上，本文方法的像素精度（pixel accuracy，PA）和骰子相似系数（Dice similarity coefficient，DSC）分别为98.93%、97.71%，比LSTM-MA（LSTM method with multi-modality and adjacency constraint）提升了1.28%、2.8%。在MRBrainS数据集上，本文方法的PA为92.46%，DSC为84.74%，比LSTM-MA提升了0.63%、1.44%。在BraTS2017数据集上，本文方法的PA和DSC上分别为98.80%，99.47%，也取得了满意的分割结果。
结论
2
提出的分割网络在多模态图像分割应用中，获得了较好的分割结果，对图像强度不均匀和噪声有较好的鲁棒性。
Objective
2
High-precision image segmentation is a key issue for biomedical image processing. It can aid to understand the anatomical information of biological tissues better. But
the segmentation precision is restricted by the non-uniformity of image intensity and noise-related issues in the process of magnetic resonance imaging (MRI). In addition
more image segmentation effects are constrained by information loss due to multi-modality and spatial neighborhood relations of medical images. Our research is focused on an image segmentation model in combination with dissimilarity criterion and entropy rate super-pixel.
Method
2
Our method is based on a segmentation model in the context of multi-modality feature fusion. This model is composed of three parts as mentioned below: 1) thanks to the entropy rate super-pixel segmentation algorithm (entropy rate super-pixel
ERS)
the multi-modality image is pre-segmented to obtain super-pixel blocks
and a new fusion algorithm is illustrated to renumber them
the super-pixel image is then established. The accurate segmentation of common areas in the tissue area is guaranteed in terms of multi-modality fusion-added
and the boundaries of the tissue area can be divided more accurately
and the overall segmentation accuracy is improved. 2) Each super-pixel block is illustrated by a node of the undirected image
and the feature vector is extracted by the gray value of each node. The correlation between nodes is judged by dissimilarity weight
and the feature sequence of adjacent nodes is constructed. The multi-modality and spatial neighborhood information develop the fineness of the boundary
the robustness of local area noise and intensity non-uniformity. Finally
the feature sequence is used as the input of bi-directional long/short-term memory model. To improve the segmentation accuracy
the cross entropy loss is used for training.
Result
2
Our method is compared to some popular algorithms in the context of BrainWeb
MRbrains and BraTS2017 datasets. The BrainWeb dataset is regarded as a simulation dataset based on brain anatomical structure
which contains MR images of T1
T2 and PD. Compared to LSTM-MA(LSTM method with multi-modality and adjacency constraint)
our pixel accuracy (PA) is 98.93% (1.28% higher) and the Dice similarity coefficient (DSC) is 97.71% (2.8 higher). The MRbrains dataset contains the ground truth of brain-relevant MR images
which consists of MR image of T1
T1IR and FLAIR modalities. Our demonstration achieves 92.46% in relation to PA metric and 84.74% in terms of DSC metric
which are 0.63% and 1.44% higher than LSTM-MA. The dataset of BraTS2017 is related to the four modalities of MR image like T1
T1CE
T2 and FLAIR. We choose three modalities of them (T1CE
T2 and FLAIR). Final PA and DSC metrics are reached to 98.80% and 99.47%. Furthermore
our convergence speed is in comparison with some popular deep learning techniques like convolutional recurrent decoding network(CRDN)
semantic flow network(SFNet) and UNet++. Our competitive convergence results can be obtained when the number of iterations is 40.
Conclusion
2
Our analysis can optimize current multi-modality features further. Multi-modality super-pixel blocks are fused by a new fusion algorithm
and a better feature sequence is constructed via the dissimilarity criterion. The final segmentation result is obtained through training and testing of the bidirectional long/short-term memory network. The experimental results show that our method can optimize the applications of image segmentation
and enhance the robustness of image intensity-oriented non-uniformity and noise-related.
Al-Dmour H and Al-Ani A. 2018. A clustering fusion technique for MR brain tissue segmentation. Neurocomputing, 275: 546-559 [DOI: 10.1016/j.neucom.2017.08.051]
Badrinarayanan V, Kendall A and Cipolla R. 2017. SegNet: a deep convolutional encoder-decoder architecture for image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(12): 2481-2495 [DOI: 10.1109/TPAMI.2016.2644615]
Bai Y Q. 2014. Research on threshold selection method of image segmentation. China CIO News, (2): 146-147
拜颖乾. 2014. 图像分割阈值选取方法的研究. 信息系统工程, (2): 146-147[DOI: 10.3969/j.issn.1001-2362.2014.02.086]
Bakas S, Akbari H, Sotiras A, Bilello M, Rozycki M, Kirby J S, Freymann J B, Farahani K and Davatzikos C. 2017. Advancing the cancer genome Atlas glioma MRI collections with expert segmentation labels and radiomic features. Scientific Data, 4: #170117 [DOI: 10.1038/sdata.2017.117]
Breiman L, Friedman J H, Olshen R A and Stone C J. 1984. Classification and Regression Trees. New York: Routledge [DOI: 10.1201/9781315139470]
Chen H, Qi Y X, Yin Y, Li T X, Liu X Q, Li X L, Gong G Z and Wang L S. 2020. MMFNet: a multi-modality MRI fusion network for segmentation of nasopharyngeal carcinoma. Neurocomputing, 394: 27-40 [DOI: 10.1016/j.neucom.2020.02.002]
Cocosco C A, Kollokian V, Kwan R K S, Pike G B and Evans A C. 1997. BrainWeb: online interface to a 3D MRI simulated brain database. NeuroImage, 5(4): #S425
Dolz J, Gopinath K, Yuan J, Lombaert H, Desrosiers C and Ayed I B. 2019. HyperDense-net: a hyper-densely connected CNN for multi-modal image segmentation. IEEE Transactions on Medical Imaging, 38(5): 1116-1126 [DOI: 10.1109/TMI.2018.2878669]
Dunn J C. 1973. A fuzzy relative of the ISODATA process and its use in detecting compact well-separated clusters. Journal of Cybernetics, 3(3): 32-57 [DOI: 10.1080/01969727308546046]
Girshick R, Donahue J, Darrell T and Malik J. 2014. Rich feature hierarchies for accurate object detection and semantic segmentation//Proceedings of 2014 IEEE Conference on Computer Vision and Pattern Recognition. Columbus, USA: IEEE: 580-587 [ DOI: 10.1109/CVPR.2014.81 http://dx.doi.org/10.1109/CVPR.2014.81 ]
Hastie T and Tibshirani R. 1996. Discriminant adaptive nearest neighbor classification. IEEE Transactions on Pattern Analysis and Machine Intelligence, 18(6): 607-616 [DOI: 10.1109/34.506411]
Hochreiter S and Schmidhuber J. 1997. Long short-term memory. Neural Computation, 9(8): 1735-1780 [DOI: 10.1162/neco.1997.9.8.1735]
Ju M, Xue L J and Zhu J X. 2020. Research on brain MRI Image segmentation algorithm based on hidden Markov random field and conjugate gradient algorithm. China Medical Devices, 35(12): 95-98, 128
居敏, 薛丽君, 朱建新. 2020. 基于隐马尔可夫随机场和共轭梯度算法的脑部MRI图像分割算法研究. 中国医疗设备, 35(12): 95-98, 128[DOI: 10.3969/j.issn.1674-1633.2020.12.024]
Kingma D P and Ba J. 2017. Adam: a method for stochastic optimization[EB/OL]. [2021-07-11] . http://arxiv.org/pdf/1412.6980.pdf http://arxiv.org/pdf/1412.6980.pdf
Li J C, Yu Z L, Gu Z H, Liu H and Li Y Q. 2019. MMAN: multi-modality aggregation network for brain segmentation from MR images. Neurocomputing, 358: 10-19 [DOI: 10.1016/j.neucom.2019.05.025]
Li X T, You A S, Zhu Z, Zhao H L, Yang M K, Yang K Y, Tan S H and Tong Y H. 2020. Semantic flow for fast and accurate scene parsing//Proceedings of the 16th European Conference on Computer Vision. Glasgow, UK: Springer: 775-793 [ DOI: 10.1007/978-3-030-58452-8_45 http://dx.doi.org/10.1007/978-3-030-58452-8_45 ]
Liang X D, Shen X H, Feng J S, Lin L and Yan S C. 2016a. Semantic object parsing with graph LSTM//Proceedings of the 14th European Conference on Computer Vision. Amsterdam, the Netherlands: Springer: 125-143 [ DOI: 10.1007/978-3-319-46448-0_8 http://dx.doi.org/10.1007/978-3-319-46448-0_8 ]
Liang X D, Shen X H, Xiang D L, Feng J S, Lin L and Yan S C. 2016b. Semantic object parsing with local-global long short-term memory//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, USA: IEEE: 3185-3193 [ DOI: 10.1109/CVPR.2016.347 http://dx.doi.org/10.1109/CVPR.2016.347 ]
Litjens G, Kooi T, Bejnordi B E, Setio A A A, Ciompi F, Ghafoorian M, van der Laak J A W M, van Ginneken B and Sánchez C I. 2017. A survey on deep learning in medical image analysis. Medical Image Analysis, 42: 60-88 [DOI: 10.1016/j.media.2017.07.005]
Liu M Y, Tuzel O, Ramalingam S and Chellappa R. 2011. Entropy rate superpixel segmentation//Proceedings of 2011 IEEE Conference on Computer Vision and Pattern Recognition. Colorado Springs, USA: IEEE: 2097-2104 [ DOI: 10.1109/CVPR.2011.5995323 http://dx.doi.org/10.1109/CVPR.2011.5995323 ]
Liu Y, Jiang P T, Petrosyan V, Li S J, Bian J W, Zhang L and Cheng M M. 2018. DEL: deep embedding learning for efficient image segmentation//Proceedings of the 27th International Joint Conference on Artificial Intelligence. Stockholm, Sweden: [s. n.]: 864-870 [ DOI: 10.24963/ijcai.2018/120 http://dx.doi.org/10.24963/ijcai.2018/120 ]
Long J, Shelhamer E and Darrell T. 2015. Fully convolutional networks for semantic segmentation//Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition. Boston, USA: IEEE: 3431-3440 [ DOI: 10.1109/CVPR.2015.7298965 http://dx.doi.org/10.1109/CVPR.2015.7298965 ]
MacQueen J. 1967. Some methods for classification and analysis of multivariate observations//Proceedings of the 5th Berkeley Symposium on Mathematical Statistics and Probability. Berkeley, USA: University of California Press: 281-297
Mendrik A M, Vincken K L, Kuijf H J, Breeuwer M, Bouvy W H, de Bresser J, Alansary A, de Bruijne M, Carass A, El-Baz A, Jog A, Katyal R, Khan A R, van der Lijn F, Mahmood Q, Mukherjee R, van Opbroek A, Paneri S, Pereira S, Persson M, Rajchl M, Sarikaya D, Smedby Ö, Silva C A, Vrooman H A, Vyas S, Wang C L, Zhao L, Biessels G J and Viergever M A. 2015. MRBrainS challenge: online evaluation framework for brain image segmentation in 3T MRI scans. Computational Intelligence and Neuroscience, 2015: #813696 [DOI: 10.1155/2015/813696]
Peng B, Zhang L and Zhang D. 2011. Automatic image segmentation by dynamic region merging. IEEE Transactions on Image Processing, 20(12): 3592-3605 [DOI: 10.1109/TIP.2011.2157512]
Pham D L, Xu C Y and Prince J L. 2000. Current methods in medical image segmentation. Annual Review of Biomedical Engineering, 2(1): 315-337 [DOI: 10.1146/annurev.bioeng.2.1.315]
Ronneberger O, Fischer P and Brox T. 2015. U-Net: convolutional networks for biomedical image segmentation//Proceedings of the 18th International Conference on Medical Image Computing and Computer-Assisted Intervention-MICCAI 2015. Munich, Germany: Springer: 234-241 [ DOI: 10.1007/978-3-319-24574-4_28 http://dx.doi.org/10.1007/978-3-319-24574-4_28 ]
Tseng K L, Lin Y L, Hsu W and Huang C Y. 2017. Joint sequence learning and cross-modality convolution for 3D biomedical segmentation//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, USA: IEEE: 3739-3746 [ DOI: 10.1109/CVPR.2017.398 http://dx.doi.org/10.1109/CVPR.2017.398 ]
Wang L, Shi F, Gao Y Z, Li G, Gilmore J H, Lin W L and Shen D G. 2014. Integration of sparse multi-modality representation and anatomical constraint for isointense infant brain MR image segmentation. NeuroImage, 89: 152-164 [DOI: 10.1016/j.neuroimage.2013.11.040]
Wen Y, Xie K and He L H. 2020. Segmenting medical MRI via recurrent decoding cell. Proceedings of the AAAI Conference on Artificial Intelligence, 34(7): 12452-12459 [DOI: 10.1609/aaai.v34i07.6932]
Xie K and Wen Y. 2019. LSTM-MA: a LSTM method with multi-modality and adjacency constraint for brain image segmentation//Proceedings of 2019 IEEE International Conference on Image Processing. Taipei, China: IEEE: 240-244 [ DOI: 10.1109/ICIP.2019.8802959 http://dx.doi.org/10.1109/ICIP.2019.8802959 ]
Xu X Y, Xu S Z, Jin L H and Song E M. 2011. Characteristic analysis of Otsu threshold and its applications. Pattern Recognition Letters, 32(7): 956-961 [DOI: 10.1016/j.patrec.2011.01.021]
Yu F and Koltun V. 2016. Multi-scale context aggregation by dilated convolutions. [EB/OL]. [2021-07-11] . https://arxiv.org/pdf/1511.07122.pdf https://arxiv.org/pdf/1511.07122.pdf
Zhang J J, Zeng J C, Qin P L and Zhao L J. 2021. Brain tumor segmentation of multi-modality MR images via triple intersecting U-Nets. Neurocomputing, 421: 195-209 [DOI: 10.1016/j.neucom.2020.09.016]
Zhang S, Dong J W and She L H. 2009. The methodology of evaluating segmentation algorithms on medical image. Journal of Image and Graphics, 14(9):1872-1880
张石, 董建威, 佘黎煌. 2009. 医学图像分割算法的评价方法. 中国图象图形学报, 14(9): 1872-1880[DOI: 10.11834/jig.20090926]
Zhang Z, Liu C Y, Ding W B, Wang S H, Pei C H, Yang M J and Huang L Q. 2020. Multi-modality pathology segmentation framework: application to cardiac magnetic resonance images//Myocardial Pathology Segmentation Combining Multi-Sequence Cardiac Magnetic Resonance Images. Lima, Peru: Springer: 37-48 [ DOI: 10.1007/978-3-030-65651-5_4 http://dx.doi.org/10.1007/978-3-030-65651-5_4 ]
Zhou T, Dong Y L, Huo B Q, Liu S and Ma Z J. 2021. U-Net and its applications in medical image segmentation: a review. Journal of Image and Graphics, 26(9): 2058-2077
周涛, 董雅丽, 霍兵强, 刘珊, 马宗军. 2021. U-Net网络医学图像分割应用综述. 中国图象图形学报, 26(9): 2058-2077[DOI: 10.11834/jig.200704]
Zhou Z W, Siddiquee M R, Tajbakhsh N and Liang J M. 2020. UNet++: redesigning skip connections to exploit multiscale features in image segmentation. IEEE Transactions on Medical Imaging, 39(6): 1856-1867 [DOI: 10.1109/TMI.2019.2959609]
相关作者
相关机构
京公网安备11010802024621
