浏览全部资源
扫码关注微信
收稿：2018-02-02，
修回：2018-5-17，
纸质出版：2018-10-16
移动端阅览
目的
2
MRI正逐步代替CT进行骨头与关节的检查，肩关节MRI中骨结构的精确自动分割对于骨损伤和疾病的度量与诊断至关重要，现有骨头分割算法无法做到不用任何先验知识进行自动分割，且通用性和精准度相对较低，为此提出一种基于图像块和全卷积神经网络（PCNN和FCN）相结合的自动分割算法。
方法
2
首先建立4个分割模型，包括3个基于U-Net的骨头分割模型（肱骨分割模型、关节骨分割模型、肱骨头和关节骨作为整体的分割模型）和一个基于块的AlexNet分割模型；然后使用4个分割模型来获取候选的骨头区域，并通过投票的方式准确检测到肱骨和关节骨的位置区域；最后在检测到的骨头区域内进一步使用AlexNet分割模型，从而分割出精确度在像素级别的骨头边缘。
结果
2
实验数据来自美国哈佛医学院/麻省总医院骨科的8组病人，每组扫描序列包括100片左右图像，都已经分割标注。5组病人用于训练和进行五倍的交叉验证，3组病人用于测试实际的分割效果，其中Dice Coefficient、Positive Predicted Value（PPV）和Sensitivity平均准确率分别达到0.92±0.02、0.96±0.03和0.94±0.02。
结论
2
本文方法针对小样本的病人数据集，仅通过2维医学图像上的深度学习，可以得到非常精确的肩关节分割结果。所提算法已经集成到我们开发的医学图像度量分析平台"3DQI"，通过该平台可以展示肩关节骨头3D分割效果，给骨科医生提供临床的诊断指导作用。同时，所提算法框架具有一定的通用性，适应于小样本数据下CT和MRI中特定器官和组织的精确分割。
Objective
2
MRI uses the principle of nuclear magnetic resonance.In addition
MRI is safe to use and has a high soft tissue resolution.CT is being gradually replaced by MRI for checking bones and joints.The automated detection and segmentation of shoulder joint structures in MRI are extremely important for measuring and diagnosing bone injuries and diseases.In MRI
the internal bone region and the air
fat
and some soft tissue in it show similar gray and black features and a low image signal to noise ratio and partial volume effect.Thus
the automatic and accurate segmentation of the clinically valuable glenoid and humeral head in the shoulder joint is difficult.Common conventional bone segmentation algorithms
such as region growing or level set
cannot be implemented without prior knowledge
and their generality and accuracy are relatively low.Although various deep learning algorithms have been applied to the segmentation of medical images
such as MRI and CT
a successful segmentation is almost impossible with only one deep network without post-processing.To the best of our knowledge
few studies on the use of deep learning to segment bones in MRI have been conducted
and no study has been conducted about shoulder segmentation.Two deep learning networks
namely
patch-based and fully convolutional networks (PCNN and FCN)
are employed for the automated detection and segmentation of the shoulder joint structure in MRI.
Method
2
First
four segmentation models are built
including three U-Net based models (glenoid segmentation model
humeral head segmentation model
glenoid and humeral head segmentation model) and one patch-based AlexNet segmentation model.The network depth of U-Net is 3
and the number of features in the first layer is 16.Edge expansion is performed by mirroring to ensure that the resolution of the output segmentation map is consistent with the original image because the resolution of the input image is reduced after passing through U-Net.The traditional AlexNet has a three-channel RGB input
but MRI is a grayscale image
and thus the RGB channel values are the same.Three channels can be added to an image by using the three window-level mappings or three different resolutions.However
no performance improvement is observed relative to the channel test
and thus
we adjust the input to one channel.Then
the four segmentation models are used to obtain the candidate bone regions from which the correct glenoid and humeral head locations and regions are obtained by voting.However
false bone regions still exist.Given that the signal intensity of the bone is near the fat and some soft tissues
which are easily misjudged as glenoid and humeral head in partial shapes and positions
and due to the complex shape and wider variation range of the humeral head
the diversity of morphologies easily leads to the misinterpretation of noise or fat as a humeral head.These false bone regions are filtered by location information
and the missing bone objects are calculated by inter-frame prediction due to the continuity of MRI scanning in the time direction.Finally
the AlexNet model is used to segment the edge of the bone with accuracy at the pixel level.
Result
2
The experimental data are derived from eight groups of patients of Harvard Medical School/Massachusetts General Hospital of the United States.Each scan sequence includes approximately 100 images with marked bone edge labels.Five groups of patients are used for training and five-fold cross-validation
and three groups are used to test the actual segmentation results.The Dice coefficient
positive predicted value
and sensitivity average accuracy are 0.92±0.02
0.96±0.03
and 0.94±0.02
respectively.Experimental results show that the segmentation accuracy is very high
which is basically consistent with the results of artificial segmentation.The segmentation accuracy also exceeds the average artificial annotation in a considerable part of the images from observation.In practical segmentation applications
training and segmentation are generally performed at the service end of the GPU device
and the segmentation result is displayed at the client end.For medical institutions
the operation is usually performed in a local area network.For a slice of a patient's MRI sequence
the overall time from the segmentation request of the client to the segmentation result from the server is approximately 1.2 seconds
meeting the real-time requirements of the application.In many cases
the scanned images of a group of patients are uniformly processed offline on the server side
and the segmentation results are saved.The segmentation results can be retrieved when the client loads the patient data.In this case
no real-time performance requirement exists
which is common in application modes.From the experimental dataset
our sample set is very limited
that is
8 sets of patient use cases.This finding indicates that a highly effective predictive effect can be achieved if the methods for obtaining image blocks and data argument are appropriate.
Conclusion
2
The model ensemble method obtained by voting is used to accurately locate the glenoid and humeral head bone in the shoulder joint.Four types of segmentation modeling are performed.The spatial consistency of the image sequence is used to predict the incorrectly deleted area.Then
PCNN segmentation is employed by the local perception and features in the located bone region of interest.Although the patients' datasets are quite small
accurate shoulder joint segmentation results are obtained.The proposed algorithm has been integrated into 3DQI
a medical image measurement and analysis platform we developed
which can demonstrate 3D segmentation of shoulder bones and provide clinical diagnosis and guidance to orthopedists.With the deepening cooperation with hospitals and the increasing number of MRI samples
we can test and analyze the three-dimensional segmentation based on deep learning and compare the segmentation results with the two-dimensional operation in future studies.
Dalvi R, Abugharbieh R, Wilson D, et al.Multi-contrast MR for enhanced bone imaging and segmentation[C]//Proceedings of the 29th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.Lyon, France: IEEE, 2007: 5620-5623.[ DOI: 10.1109/IEMBS.2007.4353621 http://dx.doi.org/10.1109/IEMBS.2007.4353621 ]
Nguyen N T, Laurendeau D, Branzan-Albu A.A new segmentation method for MRI images of the shoulder joint[C] //Proceedings of the 4th Canadian Conference on Computer and Robot Vision.Montreal, Canada: IEEE, 2007: 329-338.[ DOI: 10.1109/CRV.2007.4 http://dx.doi.org/10.1109/CRV.2007.4 ]
Folkesson J, Dam E B, Olsen O F, et al.Segmenting articular cartilage automatically using a voxel classification approach[J].IEEE Transactions on Medical Imaging, 2007, 26(1):106-115.[DOI:10.1109/TMI.2006.886808]
Seim H, Kainmueller D, Lamecker H, et al.Model-based auto-segmentation of knee bones and cartilage in MRI data[C]//Proceeding of MICCAI Workshop Medical Image Analysis for the Clinic.New York: Springer, 2010: 213-223.
Folkesson J, Carballido-Gamio J, Eckstein F, et al.Local bone enhancement fuzzy clustering for segmentation of MR trabecular bone images[J].Medical Physics, 2010, 37(1):295-302.[DOI:10.1118/1.3264615]
Bindernagel M, Kainmueller D, Seim H, et al.An articulated statistical shape model of the human knee[M]//Handels H, Ehrhardt J, Deserno T H, et al.Bildverarbeitung für die Medizin 2011.Berlin, Heidelberg: Springer, 2011: 59-63.[ DOI: 10.1007/978-3-642-19335-4_14 http://dx.doi.org/10.1007/978-3-642-19335-4_14 ]
Schmid J, Kim J, Magnenat-Thalmann N.Robust statistical shape models for MRI bone segmentation in presence of small field of view[J].Medical Image Analysis, 2011, 15(1):155-168.[DOI:10.1016/j.media.2010.09.001]
Mallikarjuna Swamy M S, Holi M S.Knee joint articular cartilage segmentation, visualization and quantification using image processing techniques:a review[J].International Journal of Computer Applications, 2012, 42(19):36-43.[DOI:10.5120/5803-8151]
Shelhamer E, Long J, Darrell T.Fully convolutional networks for semantic segmentation[J].IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(4):640-651.[DOI:10.1109/TPAMI.2016.2572683]
Ronneberger O, Fischer P, Brox T.U-net: convolutional networks for biomedical image segmentation[C]//Proceedings of the 18th International Conference on Medical Image Computing and Computer-Assisted Intervention.Munich, Germany: Springer, 2015: 234-241.[ DOI: 10.1007/978-3-319-24574-4_28 http://dx.doi.org/10.1007/978-3-319-24574-4_28 ]
Çiçek O, Abdulkadir A, Lienkamp S S, et al.3D U-Net: learning dense volumetric segmentation from sparse annotation[C]//Proceedings of the 19th International Conference on Medical Image Computing and Computer-Assisted Intervention.Athens, Greece: Springer, 2016: 424-432.[ DOI: 10.1007/978-3-319-46723-8_49 http://dx.doi.org/10.1007/978-3-319-46723-8_49 ]
Milletari F, Navab N, Ahmadi S A.V-Net: fully convolutional neural networks for volumetric medical image segmentation[C]//Proceedings of the 4th International Conference on 3D Vision.Stanford, CA, USA: IEEE, 2016: 565-571.[ DOI: 10.1109/3DV.2016.79 http://dx.doi.org/10.1109/3DV.2016.79 ]
Drozdzal M, Vorontsov E, Chartrand G, et al.The importance of skip connections in biomedical image segmentation[C]//Proceedings of the 1st International Workshop, LABELS 2016, and 2nd International Workshop, DLMIA 2016, Held in Conjunction with MICCAI 2016.Athens, Greece: Springer, 2016: 179-187.[ DOI: 10.1007/978-3-319-46976-8_19 http://dx.doi.org/10.1007/978-3-319-46976-8_19 ]
Xie Y P, Zhang Z Z, Sapkota M, et al.Spatial clockwork recurrent neural network for muscle perimysium segmentation[C]//Proceedings of 19th International Conference on Medical Image Computing and Computer-Assisted Intervention. Athens, Greece: Springer, 2016: 185-193.[ DOI: 10.1007/978-3-319-46723-8_22 http://dx.doi.org/10.1007/978-3-319-46723-8_22 ]
Stollenga M F, Byeon W, Liwicki M, et al.Parallel multi-dimensional LSTM, with application to fast biomedical volumetric image segmentation[C]//Advances in Neural Information Processing Systems.Montréal, CANADA: Neural Information Processing Systems Foundation, Inc., 2015: 2998-3006.
Andermatt S, Pezold S, Cattin P.Multi-dimensional gated recurrent units for the segmentation of biomedical 3D-data[C]//Proceedings of the 1st International Workshop,LABELS 2016, and 2nd International Workshop, DLMIA 2016, Held in Conjunction with MICCAI 2016.Athens, Greece: Springer, 2016: 142-151.[ DOI: 10.1007/978-3-319-46976-8_15 http://dx.doi.org/10.1007/978-3-319-46976-8_15 ]
Poudel R P K, Lamata P, Montana G.Recurrent fully convolutional neural networks for multi-slice MRI cardiac segmentation[C]//Proceedings of the 1st International Workshops, RAMBO 2016 and HVSMR 2016, Held in Conjunction with MICCAI 2016.Athens, Greece: Springer, 2016: 83-94.[ DOI: 10.1007/978-3-319-52280-7_8 http://dx.doi.org/10.1007/978-3-319-52280-7_8 ]
Korez R, Likar B, PernušF, et al.Model-based segmentation of vertebral bodies from MR images with 3D CNNs[C]//Proceedings of the 19th International Conference on Medical Image Computing and Computer-Assisted Intervention.Athens, Greece: Springer, 2016: 433-441.[ DOI: 10.1007/978-3-319-46723-8_50 http://dx.doi.org/10.1007/978-3-319-46723-8_50 ]
Zhou X R, Ito T, Takayama R, et al.Three-dimensional CT image segmentation by combining 2D fully convolutional network with 3D majority voting[C]//Proceedings of the 1st International Workshop, LABELS 2016, and 2nd International Workshop, DLMIA 2016, Held in Conjunction with MICCAI 2016.Athens, Greece: Springer, 2016: 111-120.[ DOI: 10.1007/978-3-319-46976-8_12 http://dx.doi.org/10.1007/978-3-319-46976-8_12 ]
Moeskops P, Wolterink J M, van der Velden B H M, et al.Deep learning for multi-task medical image segmentation in multiple modalities[C]//Proceedings of the 19th International Conference on Medical Image Computing and Computer-Assisted Intervention.Athens, Greece: Springer, 2016: 478-486.[ DOI: 10.1007/978-3-319-46723-8_55 http://dx.doi.org/10.1007/978-3-319-46723-8_55 ]
Shakeri M, Tsogkas S, Ferrante E, et al.Sub-cortical brain structure segmentation using F-CNN'S[C]//The 13th International Symposium on Biomedical Imaging.Prague, Czech Republic: IEEE, 2016: 269-272.[ DOI: 10.1109/ISBI.2016.7493261 http://dx.doi.org/10.1109/ISBI.2016.7493261 ]
Song Y Y, Zhang L, Chen S P, et al.Accurate segmentation of cervical cytoplasm and nuclei based on multiscale convolutional network and graph partitioning[J].IEEE Transactions on Biomedical Engineering, 2015, 62(10):2421-2433.[DOI:10.1109/TBME.2015.2430895]
Christ P F, Elshaer M E A, Ettlinger F, et al.Automatic liver and lesion segmentation in CT using cascaded fully convolutional neural networks and 3D conditional random fields[C]//Proceedings of the 19th International Conference on Medical Image Computing and Computer-Assisted Intervention.Athens, Greece: Springer, 2016: 415-423.[ DOI: 10.1007/978-3-319-46723-8_48 http://dx.doi.org/10.1007/978-3-319-46723-8_48 ]
Gao M C, Xu Z Y, Lu L, et al.Segmentation label propagation using deep convolutional neural networks and dense conditional random field[C]//The 13th International Symposium on Biomedical Imaging.Prague, Czech Republic: IEEE, 2016: 1265-1268.[ DOI: 10.1109/ISBI.2016.7493497 http://dx.doi.org/10.1109/ISBI.2016.7493497 ]
Alansary A, Kamnitsas K, Davidson A, et al.Fast fully automatic segmentation of the human placenta from motion corrupted MRI[C]//Proceedings of the 19th International Conference on Medical Image Computing and Computer-Assisted Intervention.Athens, Greece: Springer, 2016: 589-597.[ DOI: 10.1007/978-3-319-46723-8_68 http://dx.doi.org/10.1007/978-3-319-46723-8_68 ]
Zeng G D, Yang X, Li J, et al.3D U-net with multi-level deep supervision: fully automatic segmentation of proximal femur in 3D MR images[C]//Proceedings of the 8th International Workshop on Machine Learning in Medical Imaging.Quebec City, QC, Canada: Springer, 2017: 274-282.[ DOI: 10.1007/978-3-319-67389-9_32 http://dx.doi.org/10.1007/978-3-319-67389-9_32 ]
Nie D, Wang L, Trullo R, et al.Segmentation of craniomaxillofacial bony structures from MRI with a 3D deep-learning based cascade framework[C]//Proceedings of the 8th International Workshop.Quebec City, QC, Canada: Springer, 2017: 266-273.[ DOI: 10.1007/978-3-319-67389-9_31 http://dx.doi.org/10.1007/978-3-319-67389-9_31 ]
相关作者
相关机构
京公网安备11010802024621
