浏览全部资源
扫码关注微信
收稿：2020-05-31，
修回：2020-7-12，
录用：2020-7-19，
纸质出版：2020-10-16
移动端阅览
点云作为一种重要的3维数据，能够直观地模拟生物器官、组织等的3维结构，基于医学点云数据的分类、分割、配准、目标检测等任务可以辅助医生进行更为准确的诊断和治疗，在临床医学以及个性化医疗器械辅助设计与3D打印有着重要的应用价值。随着深度学习的发展，越来越多的点云算法逐步由传统算法扩展到深度学习算法中。本文对点云算法在医学领域的研究及其应用进行综述，旨在总结目前用于医学领域的点云方法，包括医学点云的特点、获取途径以及数据转换方法；医学点云分割中的传统算法和深度学习算法；以及医学点云的配准任务定义、意义，以及基于有/无特征的配准方法。总结了医学点云在临床应用中仍存在的限制和挑战：1）医学图像重建的人体器官点云分布稀疏且包含噪音、误差；2）医学点云数据集标注困难、制作成本高，可用于训练深度学习模型的公开数据集非常稀少；3）前沿的点云处理算法大都基于自然场景点云数据集训练，这些算法在医学点云处理中的鲁棒性和泛化能力还有待验证。随着医学点云数据集质量和数量的提升，医学点云处理算法的研究将会吸引更多的研究者。
A point cloud refers to a set of data points in a three-dimensional space. Each point is composed of a three-dimensional coordinate
with object's reflectivity
reflection intensity
distance from the point to the center of the scanner
horizontal angle
vertical angle
and deviation value. The point cloud is obtained by two methods
one is obtained by scanning the target object by the three-dimensional sensing device
such as LiDAR sensor and RGB-D camera
and the other is obtained by reconstruction from two-dimensional medical images. The point cloud can express the geometric position
shape
and scale of the target object. The point cloud has a wide range of applications in areas such as autonomous driving
robots
surveillance systems
surveying and mapping geography
virtual reality
and medicine
which has achieved remarkable results. Many researchers in the field of medical imaging have also devoted themselves to the research of medical image point cloud processing algorithms. Point cloud can intuitively simulate the three-dimensional structure of biological organs and tissues. With important application value in clinical medicine
the classification
segmentation
registration
and other tasks based on medical point cloud can help doctors make accurate diagnosis and treatment. The point cloud-based medical diagnosis has advantages and has the potential for future application in clinical screening diagnosis
personalized medical device-assisted design
and 3D printing. At this stage
deep learning algorithms have achieved remarkable results in tasks such as target detection
segmentation
and recognition. Deep learning algorithms gradually become efficient and popular in tasks such as target detection
segmentation
and recognition. Therefore
increasing point cloud processing algorithms are gradually extended from traditional algorithms to deep learning algorithms. This article reviews the research and progress of point cloud algorithms in the medical field. This review aims to summarize the current point cloud methods used in the medical field and focuses on 1) the characteristics
acquisition methods
and data conversion methods of medical point clouds; 2) traditional algorithms and deep learning algorithms in medical point cloud segmentation; and 3) the definition and significance of medical point cloud registration tasks. This review is based on feature or non-feature registration method. Finally
although the point cloud method has been applied in the medical field
the current application of the point cloud-based frontier method in the medical point cloud is still insufficient. Applying state-of-the-art algorithms to medical point clouds also requires continuous in-depth exploration and research. To date
medical point clouds have been used to assist doctors in completing some diagnostic tasks
but they are still in the process of continuous development and cannot replace the role of clinicians. The clinical application of medical point cloud has some limitations and challenges: 1) In the application research of point clouds in the medical field
the first task is to obtain point cloud data that can accurately characterize disease information. At present
point cloud data acquisition methods are relatively simple. In the future
high-quality point cloud imaging equipment can be combined to obtain accurate medical point cloud dataset. In applied research
the first task is to obtain point cloud data that can accurately characterize medical anatomical structure information. Considering that the morphological structure of human tissues and organs is relatively complex
most of the point cloud data of human organs are obtained by reconstructing medical images (such as (computed tomography(CT)) and (magnetic resonance imaging(MRI)). Therefore
such point clouds are sparsely distributed
with noise and errors. Obtaining accurate and dense medical point cloud datasets from medical images is an important subject to be studied. 2) In addition to facing the challenges of sparse reconstruction and data imbalance in point clouds
the difficulty of labeling medical point cloud data sets
the high cost of data integration
and the inevitable subjective labeling errors are the reasons why deep learning algorithms have not been widely used in the field of medical point clouds. The small amount of sample data and the imbalance of sample data may affect the accuracy of disease diagnosis. In the future
methods such as semi-supervised learning
active learning
and generating samples against the generated network can be used to improve learning accuracy. 3) A large number of medical point clouds are generated in the hospital but are not used to train and improve the diagnostic model. With the emergence of super-resolution algorithms and point cloud up-sampling networks
the prediction of sparse point clouds to dense point clouds based on medical image reconstruction will be an important means to construct high-quality medical point clouds. In the future
with the improvement of the quality and quantity of medical point cloud data sets
the research of medical point cloud processing algorithms will attract more researchers. Current research only focuses on model training and evaluation of specific data sets
which makes the universality of these algorithms challenging. The application and development of point cloud in medical images are currently a hot topic. Although point clouds have gradually penetrated into a considerable number of fields in medicine
the application of the current frontier methods of point cloud processing in medical point clouds is still insufficient. Research work using medical point cloud still needs to invest more research energy and attention.
Aiger D, Mitra N J and Cohen-Or D. 2008.4-points congruent sets for robust pairwise surface registration//ACM SIGGRAPH 2008 Papers. Los Angeles: ACM: 1-10[ DOI: 10.1145/1399504.1360684 http://dx.doi.org/10.1145/1399504.1360684 ]
Aoki Y, Goforth H, Srivatsan R A and Lucey S. 2019. PointNetLK: robust and efficient point cloud registration using PointNet//Proceedings of 2019 IEEE Conference on Computer Vision and Pattern Recognition. Long Beach: IEEE: 7163-7172[ DOI: 10.1109/cvpr.2019.00733 http://dx.doi.org/10.1109/cvpr.2019.00733 ]
Balsiger F, Soom Y, Scheidegger O and Reyes M. 2019. Learning shape representation on sparse point clouds for volumetric image segmentation//Proceedings of the 22nd International Conference on Medical Image Computing and Computer-Assisted Intervention. Shenzhen: Springer: 273-281[ DOI: 10.1007/978-3-030-32245-8_31 http://dx.doi.org/10.1007/978-3-030-32245-8_31 ]
Besl P J and McKay N D. 1992. A method for registration of 3-D shapes. IEEE Transactions on Pattern Analysis and Machine Intelligence, 14(2):239-256[DOI:10.1109/34.121791]
Biswas T, Fauzi M F A, Abas F S, Logeswaran R and Nair H K. 2019. Wound area segmentation using 4-D probability map and superpixel region growing//Proceedings of 2019 IEEE International Conference on Signal and Image Processing Applications. Kuala Lumpur: IEEE: 29-34[ DOI: 10.1109/icsipa45851.2019.8977739 http://dx.doi.org/10.1109/icsipa45851.2019.8977739 ]
Chen Q G, Jin X, Zhu H H and Salehi H S. 2020. Classification of pit and fissure for caries risk based on 3D surface morphology analysis of tooth//Proceedings of SPIE #11217, Lasers in Dentistry XXVI. San Francisco: SPIE: #112170E[ DOI: 10.1117/12.2544611 http://dx.doi.org/10.1117/12.2544611 ]
Chen X R, Song Z J and Wang M N. 2014. Automated global optimization surface-matching registration method for image-to-patient spatial registration in an image-guided neurosurgery system. Journal of Medical Imaging and Health Informatics, 4(6):942-947[DOI:10.1166/jmihi.2014.1346]
Chen X Z, Ma H M, Wan J, Li B and Xia T. 2017. Multi-view 3D object detection network for autonomous driving//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE: 6526-6534[ DOI: 10.1109/cvpr.2017.691 http://dx.doi.org/10.1109/cvpr.2017.691 ]
Cheng Q Q, Sun P Y, Yang C S, Yang Y B and Liu P X. 2020. A morphing-based 3D point cloud reconstruction framework for medical image processing. Computer Methods and Programs in Biomedicine, 193:#105495[DOI:10.1016/j.cmpb.2020.105495]
Cohen E, Deffieux T, DemenéC, Cohen L D and Tanter M. 2019.4D point cloud registration for tumor vascular networks monitoring from ultrasensitive Doppler images//Proceedings of the 16th International Symposium CMBBE and 4th Conference on Imaging and Visualization. New York: Springer: 437-456[ DOI: 10.1007/978-3-030-43195-2_35 http://dx.doi.org/10.1007/978-3-030-43195-2_35 ]
Deng J, Hou C H, Diao W and Liu Y M. 2017. Summarization of registration algorithm of 3D point cloud data. China Computer and Communication, (23):51-52, 56[DOI:10.3969/j.issn.1003-9767.2017.23.023]
邓嘉, 侯晨辉, 刁婉, 刘玉米. 2017.3维点云数据的配准算法综述.信息与电脑, (23):51-52, 56[DOI:10.3969/j.issn.1003-9767.2017.23.023]
Dong Y Y, Yang WK, Wang J W, Zhao Z J, Wang S H, Cui Q and Qiang Y. 2020. An improved supervoxel 3D region growing method based on PET/CT multimodal data for segmentation and reconstruction of GGNs. Multimedia Tools and Applications, 79(3):2309-2338[DOI:10.1007/s11042-019-08250-4]
Drokin I and Ericheva E. 2020. Deep learning on point clouds for false positive reduction at nodule detection in chest CT scans[EB/OL].[2020-05-31] . https://arxiv.org/pdf/2005.03654.pdf https://arxiv.org/pdf/2005.03654.pdf
Zanjani F G, Moin D A, Verheij B, Claessen F, Cherici T and Tan T. 2019. Deep learning approach to semantic segmentation in 3 d point cloud intra-oral scans of teeth//Proceedings of 2019 International Conference on Medical Imaging with Deep Learning. London: MIDL: 557-571
Ghazvinian Z F, Anssari M D, Verheij B, Claessen F, Cherici T and Tan T. 2019. Deep learning approach to semantic segmentation in 3D point cloud intra-oral scans of teeth//Proceedings of the 2nd International Conference on Medical Imaging with Deep Learning. London: MIDL: 557-571[ DOI: 10.5194/isprs-archives-XLII-2-W15-735-2019 http://dx.doi.org/10.5194/isprs-archives-XLII-2-W15-735-2019 ]
Graham B, Engelcke M and van der Maaten L. 2018.3D semantic segmentation with submanifold sparse convolutional networks//Proceedings of 2018 IEEE/CVF Conference on Computer vision and Pattern Recognition. Salt Lake City: IEEE: 9224-9232[ DOI: 10.1109/cvpr.2018.00961 http://dx.doi.org/10.1109/cvpr.2018.00961 ]
Guo J, Yao X X, Shen M Y, Wang JF and Liao W Y. 2018. A deep learning network for point cloud of medicine structure//Proceedings of the 9th International Conference on Information Technology in Medicine and Education. Hangzhou: IEEE: 683-687[ DOI: 10.1109/ITME.2018.00157 http://dx.doi.org/10.1109/ITME.2018.00157 ]
Guo Y L, Wang H Y, Hu Q Y, Liu H, Liu L and Bennamoun M. 2019. Deep learning for 3D point clouds: a survey[EB/OL].[2020-05-31] . https://arxiv.org/pdf/1912.12033.pdf https://arxiv.org/pdf/1912.12033.pdf
Gupta D and Anand R S. 2017. A hybrid edge-based segmentation approach for ultrasound medical images. Biomedical Signal Processing and Control, 31:116-126[DOI:10.1016/j.bspc.2016.06.012]
Gutiérrez-Becker B and Wachinger C. 2018. Deep multi-structural shape analysis: application to neuroanatomy//Proceedings of the 21st International Conference on Medical Image Computing and Computer-Assisted Intervention. Granada: Springer: 523-531[ DOI: 10.1007/978-3-030-00931-1_60 http://dx.doi.org/10.1007/978-3-030-00931-1_60 ]
Hanocka R, Hertz A, Fish N, Giryes R, Fleishman S and Cohen-Or D. 2019. MeshCNN:a network with an edge. ACM Transactions on Graphics, 38(4):#90[DOI:10.1145/3306346.3322959]
Hansen L, Dittmer D and Heinrich M P. 2019. Learning deformable point set registration with regularized dynamic graph CNNS for large lung motion in COPD patients//Proceedings of the 1st International Workshop on Graph Learning in Medical Imaging. Shenzhen: Springer: 53-61[ DOI: 10.1007/978-3-030-35817-4_7 http://dx.doi.org/10.1007/978-3-030-35817-4_7 ]
Hua B S, Tran M K and Yeung S K. 2018. Pointwise convolutional neural networks//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE: 984-993[ DOI: 10.1109/cvpr.2018.00109 http://dx.doi.org/10.1109/cvpr.2018.00109 ]
Huang J D, Kwok T H and Zhou C. 2017. V4PCS:volumetric 4PCS algorithm for global registration. Journal of Mechanical Design, 139(11):111403[DOI:10.1115/1.4037477]
Le T and Duan Y. 2018. Pointgrid: a deep network for 3d shape understanding//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE: 9204-9214[ DOI: 10.1109/cvpr.2018.00959 http://dx.doi.org/10.1109/cvpr.2018.00959 ]
Li J X, Chen B M and Hee Lee G. 2018a. So-net: Self-organizing network for point cloud analysis//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE: 9397-9406[ DOI: 10.1109/cvpr.2018.00979 http://dx.doi.org/10.1109/cvpr.2018.00979 ]
Li W, Zhang Y J, Hu Y, Chen Q, Tang W and Wang H. 2015. Combination of laser-point cloud and reverse engineering to rapidly establish a three-dimensional soft tissue model in cosmetic surgery. Chinese Journal of Tissue Engineering Research, 19(15):2346-2350 19(15):2346-2350)[DOI:10.3969/j.issn.2095-4344.2015.15.010]
李伟, 张玉洁, 胡筠, 陈淇, 汤炜, 王杭. 2015.激光点云结合逆向工程快速构建软组织立体模型在美容外科的应用.中国组织工程研究, 19(15):2346-2350[DOI:10.3969/j.issn.2095-4344.2015.15.010]
Li Y Y, Bu R, Sun M C, Wu W, Di X H and Chen B Q. 2018b. Pointcnn: Convolution on x-transformed points//Proceedings of Annual Conference on Neural Information Processing Systems 2018. Montréal: Curran Associates: 820-830
Lian C F, Wang L, Wu T H, Liu M X, Durán F, Ko C C and Shen D G. 2019. MeshSNet: deep multi-scale mesh feature learning for end-to-end tooth labeling on 3D dental surfaces//Proceedings of the 22nd International Conference on Medical Image Computing and Computer-Assisted Intervention. Shenzhen: Springer: 837-845[ DOI: 10.1007/978-3-030-32226-7_93 http://dx.doi.org/10.1007/978-3-030-32226-7_93 ]
Liang Z F, Guo Y L, Feng Y L, Chen W, Qiao L B, Zhou L, Zhang J F and Liu H Z. 2019. Stereo matching using multi-level cost volume and multi-scale feature constancy. IEEE Transactions on Pattern Analysis and Machine Intelligence: #8765737[ DOI: 10.1109/TPAMI.2019.2928550 http://dx.doi.org/10.1109/TPAMI.2019.2928550 ]
Liu W P, Sun J, Li W Y, Hu T and Wang P. 2019. Deep learning on point clouds and its application:a survey. Sensors, 19(19):#4188[DOI:10.3390/s19194188]
Liu Y L, Song Z J and Wang M N. 2017. A new robust markerless method for automatic image-to-patient registration in image-guided neurosurgery system. Computer Assisted Surgery, 22(S1):319-325[DOI:10.1080/24699322.2017.1389411]
Lu W X, Wan G W, Zhou Y, Fu X Y, Yuan P F and SongS Y. 2019. DeepVCP: an end-to-end deep neural network for point cloud registration//Proceedings of 2019 IEEE/CVF International Conference on Computer Vision. Seoul: IEEE: 12-21[ DOI: 10.1109/ICCV.2019.00010 http://dx.doi.org/10.1109/ICCV.2019.00010 ]
马丹, 张德强, 张文博, 李新. 2017.基于逆向工程技术的人体牙模3维模型重构.机械设计与制造, (2): 91-93[ DOI: 10.3969/j.issn.1001-3997.2017.02.025 http://dx.doi.org/10.3969/j.issn.1001-3997.2017.02.025 ]
Ma D, Zhang D Q, Zhang W B and Li X. 2017. Reconstruction of human model 3D model based on reverse engineering technology. Machinery Design and Manufacture, (2): 91-93 [ DOI: 10.3969/j.issn.1001-3997.2017.02.025 http://dx.doi.org/10.3969/j.issn.1001-3997.2017.02.025 ]
Mellado N, Aiger D and Mitra N J. 2014. Super 4PCS fast global pointcloud registration via smart indexing. Computer Graphics Forum, 33(5):205-215[DOI:10.1111/cgf.12446]
Nguyen L, Stoter S, Baum T, Kirschke J, Ruess M, Yosibash Z and Schillinger D. 2017. Phase-field boundary conditions for the voxel finite cell method:surface-free stress analysis of CT-based bone structures. International Journal for Numerical Methods in Biomedical Engineering, 33(12):e2880[DOI:10.1002/cnm.2880]
Porto L and Rohling R. 2020. Improving Interpretability of 2-D ultrasound of the lumbar spine//The 17th IEEE International Symposium on Biomedical Imaging. Iowa City: IEEE: 1-5[ DOI: 10.1109/ISBI45749.2020.9098446 http://dx.doi.org/10.1109/ISBI45749.2020.9098446 ]
Qi C R, Su H, Kaichun M and Guibas L J. 2017a. PointNet: deep learning on point sets for 3D classification and segmentation//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE: 652-660[ DOI: 10.1109/CVPR.2017.16 http://dx.doi.org/10.1109/CVPR.2017.16 ]
Qi C R, Yi L, Su H and Guibas L J. 2017b. Pointnet++: deep hierarchical feature learning on point sets in a metric space//Proceedings of Annual Conference on Neural Information Processing Systems 2017. Long Beach: NIPS: 5099-5108
Schaffert R, Wang J, Fischer P, Borsdorf A and Maier A. 2018. Metric-driven learning of correspondence weighting for 2-D/3-D image registration//Proceedings of the 40th German Conference on Pattern Recognition. Stuttgart:Springer:140-152[DOI:10.1007/978-3-030-12939-2_11]
Setio A A A, Ciompi F, Litjens G, Gerke P, Jacobs C, Van Riel S J, Wille M M W, Naqibullah M, Sánchez C I and van Ginneken B. 2016. Pulmonary nodule detection in CT images:false positive reduction using multi-view convolutional networks. IEEE Transactions on Medical Imaging, 35(5):1160-1169[DOI:10.1109/TMI.2016.2536809]
Sinko M, Kamencay P, Hudec R and Benco M. 2018.3D registration of the point cloud data using ICP algorithm in medical image analysis//Proceedings of 2018 ELEKTRO. Mikulov: IEEE: 1-6[ DOI: 10.1109/ELEKTRO.2018.8398245 http://dx.doi.org/10.1109/ELEKTRO.2018.8398245 ]
Su H, Maji S, Kalogerakis E and Learned-Miller E. 2015. Multi-view convolutional neural networks for 3D shape recognition//Proceedings of 2015 IEEE International Conference on Computer Vision. Santiago: IEEE: 945-953[ DOI: 10.1109/ICCV.2015.114 http://dx.doi.org/10.1109/ICCV.2015.114 ]
Tian S K, Dai N, Zhang B, Yuan F L, Yu Q and Cheng X S. 2019. Automatic classification and segmentation of teeth on 3D dental model using hierarchical deep learning networks. IEEE Access, 7:84817-84828[DOI:10.1109/ACCESS.2019.2924262]
Wang Y and Solomon J M. 2019. Deep closest point: Learning representations for point cloud registration//Proceedings of 2019 IEEE/CVF International Conference on Computer Vision. Seoul: IEEE: 3523-3532[ DOI: 10.1109/ICCV.2019.00362 http://dx.doi.org/10.1109/ICCV.2019.00362 ]
Wang Y, Sun Y B, Liu Z W, Sarma S E, Bronstein M M and Solomon J M. 2019. Dynamic graph CNN for learning on point clouds. ACM Transactions on Graphics, 38(5):#146[DOI:10.1145/3326362]
Wu B C, Wan A, Yue X Y and Keutzer K. 2018. SqueezeSeg: convolutional neural nets with recurrent CRF for real-time road-object segmentation from 3D LiDAR point cloud//Proceedings of 2018 IEEE International Conference on Robotics and Automation. Brisbane: IEEE: 1887-1893[ DOI: 10.1109/ICRA.2018.8462926 http://dx.doi.org/10.1109/ICRA.2018.8462926 ]
Wu W X, Qi Z A and Li F X. 2019. PointConv: deep convolutional networks on 3d point clouds//Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach: IEEE: 9621-9630[ DOI: 10.1109/cvpr.2019.00985 http://dx.doi.org/10.1109/cvpr.2019.00985 ]
Xie Y X, Tian J J and Zhu X X. 2019. Linking points with labels in 3D: a review of point cloud semantic segmentation[EB/OL].[2020-05-30] . https://arxiv.org/pdf/1908.08854.pdf https://arxiv.org/pdf/1908.08854.pdf
Xu Y F, Fan T Q, Xu M Y, Zeng L and Qiao Y. 2018. SpiderCNN: deep learning on point sets with parameterized convolutional filters//Proceedings of the 15th European Conference on Computer Vision. Munich: Springer: 87-102[ DOI: 10.1007/978-3-030-01237-3_6 http://dx.doi.org/10.1007/978-3-030-01237-3_6 ]
Yang L and Chakraborty R. 2019. A GMM based algorithm to generate point-cloud and its application to neuroimaging[EB/OL].[2020-05-30] . https://arxiv.org/pdf/1911.01705.pdf https://arxiv.org/pdf/1911.01705.pdf
Yang S M, Shi X Y, Zhang G F and Lv C S. 2018. A dual-platform laser scanner for 3D reconstruction of dental pieces. Engineering, 4(6):796-805[DOI:10.1016/j.eng.2018.10.005]
Yang X, Xia D, Kin T and Igarashi T. 2020. IntrA: 3D intracranial aneurysm dataset for deep learning[EB/OL].[2020-05-30] . https://arxiv.org/pdf/2003.02920.pdf https://arxiv.org/pdf/2003.02920.pdf
相关作者
相关机构
京公网安备11010802024621
