发布时间: 2020-10-16
摘要点击次数:
全文下载次数:
DOI: 10.11834/jig.200253
2020 | Volume 25 | Number 10

前沿进展

点云算法在医学领域的研究进展

李美佳¹, 于泽宽², 刘晓¹, 颜荣耀¹, 于媛媛¹, 王大明³, 陈涓⁴, 陆军³, 祁鹏³, 王俊杰³, 刘杰¹

1. 北京交通大学计算机与信息技术学院, 北京 100044;

2. 复旦大学工程与应用技术研究院, 上海 200433;

3. 北京医院(国家老年医学中心)神经外科, 北京 100730;

4. 北京医院(国家老年医学中心)放射科, 北京 100730

收稿日期: 2020-05-31; 修回日期: 2020-07-12; 预印本日期: 2020-07-19

基金项目: 国家自然科学基金项目（KKA309004533）；首都卫生科研发展专项（2020-4-4053）

第一作者简介: 李美佳, 1996年生, 女, 硕士研究生, 主要研究方向为医学图像处理。E-mail:mggadaily@bjtu.edu.cn;
于泽宽, 男, 博士后, 主要研究方向为医学影像智能诊断与分析。E-mail:yzk@fudan.edu.cn;
刘晓, 女, 博士研究生, 主要研究方向为医学图像处理。E-mail:19112022@bjtu.edu.cn;
颜荣耀, 男, 硕士研究生, 主要研究方向为医学图像处理。E-mail:kyonyan@foxmail.com;
于媛媛, 女, 硕士研究生, 主要研究方向为医学图像处理。E-mail:19125280@bjtu.edu.cn;
王大明, 男, 主任医师, 主要研究方向为脑和脊髓血管病的诊断和治疗。E-mail:daming2000@263.net;
陈涓, 女, 主任医师, 主要研究方向为神经影像诊断。E-mail:13521566485@163.com;
陆军, 男, 副主任医师, 主要研究方向为脑和脊髓血管病的诊断和治疗。E-mail:lujun@bjhmoh.cn;
祁鹏, 男, 副主任医师, 主要研究方向为脑和脊髓血管病的诊断和治疗。E-mail:qipeng2008@aliyun.com;
王俊杰, 男, 主治医师, 主要研究方向为脑和脊髓血管病的诊断和治疗。E-mail:dr.junjie@gmail.com.

中图法分类号: TP391.4

文献标识码: A

文章编号: 1006-8961(2020)10-2013-11

摘要

点云作为一种重要的3维数据，能够直观地模拟生物器官、组织等的3维结构，基于医学点云数据的分类、分割、配准、目标检测等任务可以辅助医生进行更为准确的诊断和治疗，在临床医学以及个性化医疗器械辅助设计与3D打印有着重要的应用价值。随着深度学习的发展，越来越多的点云算法逐步由传统算法扩展到深度学习算法中。本文对点云算法在医学领域的研究及其应用进行综述，旨在总结目前用于医学领域的点云方法，包括医学点云的特点、获取途径以及数据转换方法；医学点云分割中的传统算法和深度学习算法；以及医学点云的配准任务定义、意义，以及基于有/无特征的配准方法。总结了医学点云在临床应用中仍存在的限制和挑战：1）医学图像重建的人体器官点云分布稀疏且包含噪音、误差；2）医学点云数据集标注困难、制作成本高，可用于训练深度学习模型的公开数据集非常稀少；3）前沿的点云处理算法大都基于自然场景点云数据集训练，这些算法在医学点云处理中的鲁棒性和泛化能力还有待验证。随着医学点云数据集质量和数量的提升，医学点云处理算法的研究将会吸引更多的研究者。

关键词

点云; 医学应用; 深度学习; 分割; 配准

Progress of point cloud algorithm in medical field

Li Meijia¹, Yu Zekuan², Liu Xiao¹, Yan Rongyao¹, Yu Yuanyuan¹, Wang Daming³, Chen Juan⁴, Lu Jun³, Qi Peng³, Wang Junjie³, Liu Jie¹

1. School of Computer and Information Technology, Beijing Jiaotong University, Beijing 100044, China;

2. Academy for Engineering and Technology, Fudan University, Shanghai 200433, China;

3. Department of Neurosurgery, Beijing Hospital, National Center of Gerontology, Beijing 100730, China;

4. Department of Radiology, Beijing Hospital, National Center of Gerontology, Beijing 100730, China

Supported by: National Natural Science Foundation of China (KKA309004533)

Abstract

A point cloud refers to a set of data points in a three-dimensional space. Each point is composed of a three-dimensional coordinate, with object's reflectivity, reflection intensity, distance from the point to the center of the scanner, horizontal angle, vertical angle, and deviation value. The point cloud is obtained by two methods, one is obtained by scanning the target object by the three-dimensional sensing device, such as LiDAR sensor and RGB-D camera, and the other is obtained by reconstruction from two-dimensional medical images. The point cloud can express the geometric position, shape, and scale of the target object. The point cloud has a wide range of applications in areas such as autonomous driving, robots, surveillance systems, surveying and mapping geography, virtual reality, and medicine, which has achieved remarkable results. Many researchers in the field of medical imaging have also devoted themselves to the research of medical image point cloud processing algorithms. Point cloud can intuitively simulate the three-dimensional structure of biological organs and tissues. With important application value in clinical medicine, the classification, segmentation, registration, and other tasks based on medical point cloud can help doctors make accurate diagnosis and treatment. The point cloud-based medical diagnosis has advantages and has the potential for future application in clinical screening diagnosis, personalized medical device-assisted design, and 3D printing. At this stage, deep learning algorithms have achieved remarkable results in tasks such as target detection, segmentation, and recognition. Deep learning algorithms gradually become efficient and popular in tasks such as target detection, segmentation, and recognition. Therefore, increasing point cloud processing algorithms are gradually extended from traditional algorithms to deep learning algorithms. This article reviews the research and progress of point cloud algorithms in the medical field. This review aims to summarize the current point cloud methods used in the medical field and focuses on 1) the characteristics, acquisition methods, and data conversion methods of medical point clouds; 2) traditional algorithms and deep learning algorithms in medical point cloud segmentation; and 3) the definition and significance of medical point cloud registration tasks. This review is based on feature or non-feature registration method. Finally, although the point cloud method has been applied in the medical field, the current application of the point cloud-based frontier method in the medical point cloud is still insufficient. Applying state-of-the-art algorithms to medical point clouds also requires continuous in-depth exploration and research. To date, medical point clouds have been used to assist doctors in completing some diagnostic tasks, but they are still in the process of continuous development and cannot replace the role of clinicians. The clinical application of medical point cloud has some limitations and challenges: 1) In the application research of point clouds in the medical field, the first task is to obtain point cloud data that can accurately characterize disease information. At present, point cloud data acquisition methods are relatively simple. In the future, high-quality point cloud imaging equipment can be combined to obtain accurate medical point cloud dataset. In applied research, the first task is to obtain point cloud data that can accurately characterize medical anatomical structure information. Considering that the morphological structure of human tissues and organs is relatively complex, most of the point cloud data of human organs are obtained by reconstructing medical images (such as (computed tomography(CT)) and (magnetic resonance imaging(MRI)). Therefore, such point clouds are sparsely distributed, with noise and errors. Obtaining accurate and dense medical point cloud datasets from medical images is an important subject to be studied. 2) In addition to facing the challenges of sparse reconstruction and data imbalance in point clouds, the difficulty of labeling medical point cloud data sets, the high cost of data integration, and the inevitable subjective labeling errors are the reasons why deep learning algorithms have not been widely used in the field of medical point clouds. The small amount of sample data and the imbalance of sample data may affect the accuracy of disease diagnosis. In the future, methods such as semi-supervised learning, active learning, and generating samples against the generated network can be used to improve learning accuracy. 3) A large number of medical point clouds are generated in the hospital but are not used to train and improve the diagnostic model. With the emergence of super-resolution algorithms and point cloud up-sampling networks, the prediction of sparse point clouds to dense point clouds based on medical image reconstruction will be an important means to construct high-quality medical point clouds. In the future, with the improvement of the quality and quantity of medical point cloud data sets, the research of medical point cloud processing algorithms will attract more researchers. Current research only focuses on model training and evaluation of specific data sets, which makes the universality of these algorithms challenging. The application and development of point cloud in medical images are currently a hot topic. Although point clouds have gradually penetrated into a considerable number of fields in medicine, the application of the current frontier methods of point cloud processing in medical point clouds is still insufficient. Research work using medical point cloud still needs to invest more research energy and attention.

Key words

point clouds; medical applications; deep learning; segmentation; registration

0 引言

点云是3维空间中的一组数据点，每个点都是由其3维坐标和对象的反射率、反射强度、点到扫描仪中心的距离、水平角、垂直角以及偏差值等信息组成，这组数据点能够表达目标物体的几何位置、形状和尺度等信息。点云通常由两种方法获取：1)由3维传感设备(如LiDAR传感器和RGB-D相机)对目标物体进行扫描获得的点云数据(Liang等，2019)；2)由2维图像进行3维重建获得的点云数据。随着点云在自动驾驶、机器人、监视系统、测绘地理、虚拟现实等领域的广泛应用(Chen等，2017)，点云在识别、配准、分割、分类、采样和去噪等目标任务领域也取得了显著成果(Liang等，2019)。

点云能够直观地模拟出生物的器官、组织等3维结构，基于医学点云的病灶区域分割、分类和配准等任务能够为病情诊断、手术指导以及治疗规划等医疗过程提供重要信息。因此，吸引了大量医学领域研究人员开展点云算法在医学领域的应用研究。Yang和Chakraborty(2019)基于高斯混合模型的点云重建方法实现了脑部MRI(magnetic resonance imaging)图像中胼胝体区域的点云重建，对比分析了患有痴呆症患者和正常人的脑部胼胝体区域的点云差异，对于痴呆症的临床诊断有着重要的意义。此外，基于点云的人脸3维数据常应用于医疗美容手术(例如面部填充、隆鼻等美容项目)及其术后模拟中(李伟等，2015)。在牙齿正畸等治疗中，基于激光扫描和逆向工程构建的数字化点云牙齿模型(马丹等，2017)逐渐取代牙膜石膏模型，对正畸方案有着指导作用。

本文使用Web of Science[v.5.25.1]所有数据库进行数据检索，统计了1999—2019年以点云为主题的论文发表数量，结果如图 1。针对点云的研究逐渐增多，用于医学图像领域的点云方法的研究在2015—2019年稳步增长。本文围绕点云算法在医学领域的研究及其应用进行综述，内容主要包括：点云的获取及特点、点云数据转换方法和3维点云数据在医学影像分割及配准中的应用。重点是进行医学点云在分割、配准等任务的评估对比，基于点云的医学诊断方法具有优势，在临床筛查诊断以及个性化医疗器械辅助设计与3D打印方面具有潜在的应用价值。医学点云的处理应用流程如图 2所示。

图 1 基于Web of Science数据的1999—2019年点云主题论文数量统计图

Fig. 1 Number of point cloud papers in 1999—2019 of Web of Science

图 2 医学点云的处理应用流程(Su等，2015；Liu等，2017；Nguyen等，2017；Yang等，2018, 2020；Cheng等，2020)

Fig. 2 Flow chart of medical point cloud processing application (Su et al., 2015; Liu et al., 2017; Nguyen et al., 2017; Yang et al., 2018, 2020; Cheng et al., 2020)

1 点云特点及深度学习处理算法

3维数据包括多视角投影图像、网格、体素和点云等。这4种3维数据类型在一定程度上具有关联性，可通过运算进行数据类型转换，针对不同任务场景选择适宜的3维数据类型进行计算。

1.1 点云的特点

多视角投影图像数据是将3维数据模拟投影获取多视角2维图像，并对投影后的图像进行分割、目标检测等处理。如果3维数据规模过大无法直接计算或者针对处理任务尚无成熟算法，可以将3维数据转换为2维多视角投影图像进行计算。网格和体素是具有规则化结构的3维数据，规则化结构有利于将卷积思想应用到3维数据中。

点云是3维空间中的一组数据点的集合，能够保留原始的坐标信息。由于没有网格数据中点和边的关系以及体素数据获取中的离散化过程，因此点云具有稀疏、无序和非结构化的特点。无序性指点云集合中的每个元素点对顺序不敏感，非结构化指点云的不均匀分布使得点与点之间存在的空间关系无法用固定的空间结构描述。此外，点云经几何变换(如旋转、平移和缩放)后的计算结果应保持与几何变换前的计算结果近似不变(Qi等，2017a)。点云数据运算对内存及其计算性能的要求小于体素数据，且体素数据更容易受分辨率影响。相比于网格和体素数据，点云可以更有效地表示组织和器官的解剖形状，允许一次性处理整个3维模型并且更有效地利用体积信息(Balsiger等，2019)。在计算机图形学中常使用大型3D测量仪器进行离散采样来获得常规物体的点云模型。由于人体，特别是组织和器官的形态结构较为复杂，通常在获得人体的2维医学图像后，如MRI、CT(computed tomography)图像，根据2维医学图像重建对象的3D表面模型，将3D表面模型离散化为3D点云(Cheng等，2020)，在医学2维图像3维重构过程中按照网格、点云或体素的顺序获取。

1.2 点云数据类型转换

由于点云数据的无序性和非结构化的特性，当前点云处理方法大致分为两类：一是将点云转化成其他典型3维数据类型进行处理，如利用点云投影处理，转化为规则化结构(如网格、体素和图结构等规则结构)处理；二是不直接将点云转化为其他3维数据类型的处理。

1.2.1 类型转化的点云处理方法

由于计算机硬件性能的限制，百万级别大规模点云(如3D LiDAR数据)难以进行直接运算，大多数算法通过基于球面投影(Wu等，2018)的方法进行计算，先将3D LiDAR点云转换为2D LiDAR图像，然后使用2维卷积网络分割点云。Su等人(2015)将网格表示的3维模型投影多视角2维图像中完成目标物体的分类和检索，避免了对3D描述符直接进行识别并且成为当时准确性最高的一种识别方法。在医学领域中，Setio等人(2016)利用肺部CT数据提取出的立方体状病变候选区，将2维卷积网络应用于立方体候选区的9个对称面，并使用数据融合技术将输出单元合并，实现了肺结节检测。

将点云数据转化为网格、体素规则3维数据类型，或构建为其他规则结构(如图结构等)也是点云数据转化的常用方法。Tian等人(2019)将牙齿点云数据转化成体素数据并使用稀疏八叉树降低数据的复杂度，提升了3D CNN(convolutional neural networks)网络分类和分割的性能。在医学领域中，Drokin和Ericheva(2020)基于肺CT图像中疑似肺结节区域重建的点云数据，对比了PointNet(Qi等，2017a)、PointNet+ +(Qi等，2017b)和DGCNN(dynamic graph CNN)(Wang等，2019)的识别效果，基于图结构的DGCNN能保留点之间的相关几何信息，更好地表现了点云局部特征，在降低肺结节假阳性判别指标中取得最优结果。

1.2.2 非类型转化的点云处理方法

除了以上两种基于类型转化的点云处理方法外，PointNet网络开创性地提供了无需对点云进行数据转换而直接基于原始点云的计算思路，能够拟合任意的连续集合函数，对点云中的噪声和数据缺失具有鲁棒性，为对象分类、局部分割和语义分割提供了统一的方法。如图 3，网络中共享多层感知机(shared multilayer perceptron, SMP)和对称函数Max Pool，解决了点云的无序性给卷积带来的挑战。SO-Net(Li等，2018a)通过构建SOM(self-organization map)对点云的空间分布进行建模，在点云重建、目标分类和目标分割中具有良好的表现。Pointwise CNN(Hua等，2018)为点云输入定义了卷积运算符，卷积核以每个点作为中心，卷积核区域内的其他点云参与运算，通过逐点卷积实现了室内场景分割和对象识别。

图 3 PointNet网络结构(Qi等，2017a)

Fig. 3 PointNet network(Qi et al., 2017a)

MSPNet(multi-structure pointnet)(Gutiérrez-Becker和Wachinger，2018)是基于PointNet设计的对多个大脑结构点云进行形状分类的深度神经网络，应用于预测阿尔茨海默氏病、轻度认知障碍以及预测脑龄。MeshSNet(deep multi-scale Mesh feature learning net)(Lian等，2019)基于PointNet结构，结合多尺度图约束学习模块提取多尺度特征实现牙齿网格数据分割。OG-PointNet+ + (Guo等，2018)将不平衡八叉树与PointNet+ +相结合, 根据点云密度对点分组，在网络不同层中计算不同组，实现了点云的分类分割，该研究成功分割了手点云中的手掌、手指和指关节。以PointNet网络为基础的医学点云研究工作逐渐增多，尤其在医学点云分类、分割等任务上效果显著。

2 医学图像点云分割算法研究进展

2.1 传统的点云分割算法

传统的点云分割方法可分为基于边缘、基于区域生长法、基于模型以及无监督的分割方法。这些传统方法在超声图像、CT图像分割上起到了重要作用。基于混合边缘的超声医学图像分割方法(Gupta和Anand，2017)利用具有空间约束的核模糊聚类的特征和使用距离正则化水平集函数的主动轮廓方法完成超声图像的精确分割，但该分割精度取决于边缘检测能力，具有一定的不确定性。基于4-D概率图和超像素区域生长的伤口区域分割方法(Biswas等，2019)在准确性、稳定性和计算复杂度都有着良好的表现，但种子点的选择对于输出结果具有较大影响。基于改进的超体素3维区域生长方法(Dong等，2020)实现肺部CT图像中磨玻璃结节的准确分割。在牙齿3维表面形态分析的龋洞、裂隙分级前期预处理工作中通过区域生长法完成初步牙齿点云中的坑和裂缝分割(Chen等，2020)。虽然这些方法能够对病灶进行分割，但在进行自动分割之前需要一系列图像预处理步骤来获得质量更好的点云数据集，十分复杂。

2.2 基于深度学习的点云分割算法

随着点云技术和深度学习的发展，基于深度学习的点云分割算法在医学领域引起了广泛的关注和探讨(Guo等，2019；Liu等，2019；Xie等，2019)。PointNet能完成对象分类(object classification)、局部分割(part segmentation)和语义分割(semantic segmentation)等任务，功能强大。以PointNet为基础的PointNet+ +网络将点云的局部特征信息结合到CNN架构中，可以进行具有局部特征的分层特征学习，更好地捕捉局部特征，最终实现点云对象的分类与分割。

Yang等人(2020)在3D颅内动脉瘤数据集Intra(https://github.com/intra3d2019/IntrA)上提取点云数据，并针对这些点云数据，对11个流行的点云深度学习网络进行分割性能对比实验，在深度学习点云网络应用于医学影像分割方面进行了有益的探索。Ghazvinian等人(2019)基于PointCNN(Li等，2018b)模型设计了牙齿语义分割的方法，并取得较为满意的分割结果。本文对代表性的医学点云分割方法的结构和典型应用进行了总结，如表 1所示。

表 1 不同算法的结构特点以及在医学数据上的应用
Table 1 Structural characteristics of different algorithms and their application in medical data

下载CSV

文献	算法	采样层分组层映射层	输入类型	数据集和评价指标
				Intra(DSC), 输入2 048		牙齿网格(DSC)	牙齿网格(Sen)	胸CT (Sen)
				健康段	肿瘤段	牙齿网格(DSC)	牙齿网格(Sen)	胸CT (Sen)
Qi等人(2017a)	PointNet	none+none+MLP	point	0.841 7	0.495 9	0.781±0.134 p=1.6E-10	0.828±0.167 p=8.1E-7	0.775
Qi等人(2017b)	PN++g	FPS+neighborhood ball+MLP	point	0.966 2	0.851 8	-	-	-
Qi等人(2017b)	PN++	FPS+neighborhood ball+MLP	point	0.964	0.846 4	-	-	-
Qi等人(2017b)	PointNet++	FPS+neighborhood ball+MLP	point			-	-	0.738
Li等人(2018b)	PointCNN	random&FPS+kNN+ MLP	point	0.966 2	0.813 6	-	-	-
Xu等人(2018)	SpiderCNN	FPS+r ball+Taylor expansion	point	0.927 1	0.677 4	-	-	-
Li等人(2018a)	SO-Net	SOM-Nodes+r ball+MLP	point	0.970 9	0.887 6	-	-	-
Wu等人(2019)	PointConv	FPS+radius-nn +MLP	point	0.971 8	0.865 2	-	-	-
Wang等人(2019)	DGCNN	none+kNN+MLP	mesh	-	-	-	-	0.859
Hanocka等人(2019)	MeshCNN	edge collapse+ mesh edge	mesh	0.947 7	0.81.87	-	-	-
Lian等人(2019)	MeshSNet	none+none+MLP	mesh	-	-	0.946± 0.062 n/a	0.938± 0.060 n/a	-
Le和Duan(2018)	pointgrid	Point Quantization+encoder	voxel	0.885	0.535 2	-	-	-
Graham等人(2018)	SSCN-F	SC+VSC	voxel	0.946 2	0.705 4	-	-	-
Graham等人(2018)	SSCN-U	SC+VSC	voxel	0.937 8	0.673 9
注：加粗字体为每列最优值，“-”代表没有相关实验，表中缩写词和变量分别解释为PN+ +(PointNet+ + input with normal), PN+ +g(PointNet+ + input with normal and geodesic distance)，FPS(iterative farthest point sampling)，kNN(k-nearest neighbor)，r ball(ball with radius r)，SC(sparse convolution)，VSC(valid sparse convolution)，评价指标DSC(Sørensen-Dice coefficient)，Sen(sensitivity)。

3 医学点云配准算法研究进展

点云配准是对同一物体在不同方向上获取的点云数据寻找它们之间的对应关系，并将这些点云数据转换到全局坐标系下(邓嘉等，2017)。医学领域的配准研究中常基于不同模态、不同维度的数据进行。对于多模态图像配准，不同模态之间的差异给设计图像相似性度量带来挑战。对于同维度图像的配准，2D到2D配准受限于缺乏3维空间信息，3D到3D的配准通常基于体素数据，导致对计算机性能要求较高。点云的稀疏性可以降低对计算机的性能要求，同时点云在表现解剖结构上具有优势，医学点云配准具有实用价值。在当前点云配准方法中，可根据是否利用点云表面特征将配准划分为基于特征的点云配准方法和基于无特征的点云配准方法提取特征；然后选择相似性度量，并获取对应特征；最后通过对应特征求解变换参数(邓嘉等，2017)。

3.1 基于无特征的点云配准算法

基于无特征的配准算法包括：基于迭代最近点算法、正态分布变换算法、采样一致性初始配准法。基于无特征的配准算法不需要提取特征，且精度较高，因此其实用性、准确性及可靠性远远高于基于特征的配准算法。Besl和McKay(1992)最先将ICP(iterative closest point)算法用于3维数据配准，ICP算法也是当前应用最为广泛的配准方法。ICP算法计算两个点云中每个点的欧氏距离，最小化均方误差实现配准，这种逐点遍历的计算模式使得计算时间较长，且容易陷入局部最优解。针对这一缺点，大量学者基于ICP算法进行优化。DCP(deep closet point)是针对ICP现有缺点进行优化的网络模型(Wang和Solomon，2019)，通过合并DGCNN和一个注意模块提取的特征极大地提升了刚性配准算法的性能，但此网络目前只能处理小规模对象级object-level点云。

在医学点云配准中，基于改进四点快速鲁棒匹配算法(4-points congruent sets，4PCS)的粗配准到基于迭代最近点ICP的精配准的配准算法可用于颅骨CT图像的配准(Chen等，2014)。ICP与3D SIFT(scale-invariant feature transform)相结合的点云配准方法实现了CT/MRI图像与标准人类头骨的3D模型关键点配准(Sinko等，2018)。基于小鼠医学超敏多普勒数据提取出肿瘤和血管网络的点云数据，并应用改进的ICP算法对预先选择的非刚性变换分割血管段进行配准，完成了大脑中血管和肿瘤区域的不变结构检测，为进一步3D肿瘤分析提供基础结构信息(Cohen等，2019)。

3.2 基于特征的点云配准算法

四点快速鲁棒匹配算法(4PCS)是典型的基于点特征配准方法，根据四点一致集实现配准，对含有噪声和扰动的数据具有非常好的鲁棒性(Aiger等，2008)。该方法虽然在初始化良好的情况下能够获得较高的配准精度，但4PCS算法的时间复杂度为O(n²)，耗时较长。为解决4PCS配准耗时较长问题，S4PCS(super 4pcs)算法使用智能索引数据组织有效地将时间复杂度降至O(n)，并在百万级点云数据配准上取得较好的结果(Mellado等，2014)。在S4PCS算法的基础上提出了V4PCS (volumetric 4pcs)算法，该方法将4个共面点扩展为非共面点以进行全局配准(Huang等，2017)，证明了计算复杂度的显著降低，并在牙齿点云模型配准上取得了良好的结果。

由于深度学习在分割识别等任务中具有重要作用，研究学者也尝试将其引入到配准工作中。PointNetLK(Aoki等，2019)将修改后的LK(lucas & kanade)算法集成到PointNet框架中，这是PointNet网络结构首次引入到点云配准中，为深度学习在点云配准中的应用开辟了新的探索途径。DeepVCP(Lu等，2019)是首个基于端到端学习的点云配准框架，该方法能够避免动态对象的推断，充分利用静止对象的显著特征，实现了LiDAR模型配准工作的高鲁棒性。在医学领域中，Schaffert等人(2018)基于PointNet模型，依据点到平面对应关系(point to plane correspondence)配准标准从全局准则学习局部对应关系，实现了单椎骨3D CT体积模型到2D X射线图像配准，在微创医疗程序中的术前3D体积向术中2D X射线图像配准中有重要作用。Hansen等人(2019)设计了一个端到端的用于可变性点集(point set)配准方法，该方法可以使用DGCNN与正则化和完全可区分的高维相干点漂移(coherent point drift)模型从不规则点集中学习几何特征，完成了两个分别在大幅度呼入和呼出状态下肺部CT提取的关点集之间的配准，该方法在处理含有相对无关信息的点云时将高维相干点漂移模型进行优化，为表面点形状配准工作提供了解决思路。表 2列出了医学点云配准的部分代表性研究工作。

表 2 配准算法在医学点云中的应用实例
Table 2 Application example of registration algorithm in medical point cloud

下载CSV

文献	算法	类别	匹配数据	MTRE/mm	RMSE/mm
Chen等人(2014)	改进4PCS算法+ICP	Y+N	颅骨CT	-	-
Schaffert等人(2018)	PPC	Y	单椎骨3D CT/2D X-ray	0.75±0.21	-
	PPC-R			0.79±0.22	-
	PPC-RM			1.18±0.42	-
	PPC-L			0.74±0.26	-
Hansen等人(2019)	DGCNN+CPD	Y	呼入肺CT/呼出肺CT	4.7±4.1	-
Sinko等人(2018)	ICP(迭代次数为30)	N	头骨配准	-	8.057×10^-3
	ICP k-D tree (迭代次数为30)			-	8.159×10^-3
	ICP + 3D sift (迭代次数为30)			-	7.908×10^-3
Cohen等人(2019)	改进ICP	N	血管段配准	-	-
注：类别中字母Y代表基于特征的算法，字母N代表基于无特征的算法，Y+N代表两种方法相结合。评价指标MTRE(mean target registration error)和RMSE(root mean squared error)，“-”代表暂无数据。

4 结语

本文对点云在医学领域的研究方法进行综述，回顾了点云在医学上的应用：病灶、器官组织等分割，图像配准等方法。目前，医学点云已被用来辅助医生完成相关诊断任务，用于提取和分析医学影像中人工难以获取的信息，医学点云配准在实时手术指导中也有重要应用。然而，医学点云在临床应用中仍存在一些限制和挑战：

1) 在应用研究中，首要任务是获得能准确表征医学解剖结构信息的点云数据，由于人体内部组织的形态结构较为复杂，人体组织点云数据大多由医学图像(如CT、MRI)重建获取，因此这类点云分布稀疏，存在噪声和误差，如何通过医学图像获取精准稠密的医学点云数据集是待研究的重要问题。

2) 除了面对重建点云的稀疏和数据失衡等挑战，医学点云数据集标注困难、制作数据集成本高、难以避免的主观标注误差也是深度学习算法在医学点云领域尚未广泛应用的原因。

3) 当前的点云处理算法大都基于自然场景训练，这些算法在医学点云处理中的鲁棒性和泛化能力还有待验证。除了着眼于医学点云的计算，如何充分利用医学图像中所包含的3维信息也是前沿点云算法向医学领域拓展过程中的挑战。

点云在医学图像中的应用和发展是当前的热门话题。随着超分辨算法和点云上采样网络的出现，基于医学图像重建的稀疏点云到稠密点云的预测将会是构建高质量医学点云的重要手段。随着医学点云数据集质量和数量的提升，医学点云处理算法的研究将会吸引更多的研究者。

参考文献

Aiger D, Mitra N J and Cohen-Or D. 2008.4-points congruent sets for robust pairwise surface registration//ACM SIGGRAPH 2008 Papers. Los Angeles: ACM: 1-10[DOI: 10.1145/1399504.1360684]

Aoki Y, Goforth H, Srivatsan R A and Lucey S. 2019. PointNetLK: robust and efficient point cloud registration using PointNet//Proceedings of 2019 IEEE Conference on Computer Vision and Pattern Recognition. Long Beach: IEEE: 7163-7172[DOI: 10.1109/cvpr.2019.00733]

Balsiger F, Soom Y, Scheidegger O and Reyes M. 2019. Learning shape representation on sparse point clouds for volumetric image segmentation//Proceedings of the 22nd International Conference on Medical Image Computing and Computer-Assisted Intervention. Shenzhen: Springer: 273-281[DOI: 10.1007/978-3-030-32245-8_31]

Besl P J, McKay N D. 1992. A method for registration of 3-D shapes. IEEE Transactions on Pattern Analysis and Machine Intelligence, 14(2): 239-256 [DOI:10.1109/34.121791]

Biswas T, Fauzi M F A, Abas F S, Logeswaran R and Nair H K. 2019. Wound area segmentation using 4-D probability map and superpixel region growing//Proceedings of 2019 IEEE International Conference on Signal and Image Processing Applications. Kuala Lumpur: IEEE: 29-34[DOI: 10.1109/icsipa45851.2019.8977739]

Chen Q G, Jin X, Zhu H H and Salehi H S. 2020. Classification of pit and fissure for caries risk based on 3D surface morphology analysis of tooth//Proceedings of SPIE #11217, Lasers in Dentistry XXVI. San Francisco: SPIE: #112170E[DOI: 10.1117/12.2544611]

Chen X R, Song Z J, Wang M N. 2014. Automated global optimization surface-matching registration method for image-to-patient spatial registration in an image-guided neurosurgery system. Journal of Medical Imaging and Health Informatics, 4(6): 942-947 [DOI:10.1166/jmihi.2014.1346]

Chen X Z, Ma H M, Wan J, Li B and Xia T. 2017. Multi-view 3D object detection network for autonomous driving//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE: 6526-6534[DOI: 10.1109/cvpr.2017.691]

Cheng Q Q, Sun P Y, Yang C S, Yang Y B, Liu P X. 2020. A morphing-based 3D point cloud reconstruction framework for medical image processing. Computer Methods and Programs in Biomedicine, 193: #105495 [DOI:10.1016/j.cmpb.2020.105495]

Cohen E, Deffieux T, Demené C, Cohen L D and Tanter M. 2019.4D point cloud registration for tumor vascular networks monitoring from ultrasensitive Doppler images//Proceedings of the 16th International Symposium CMBBE and 4th Conference on Imaging and Visualization. New York: Springer: 437-456[DOI: 10.1007/978-3-030-43195-2_35]

Deng J, Hou C H, Diao W, Liu Y M. 2017. Summarization of registration algorithm of 3D point cloud data. China Computer and Communication, 56(23): 51-52, 56 (邓嘉, 侯晨辉, 刁婉, 刘玉米. 2017. 3维点云数据的配准算法综述. 信息与电脑, 56(23): 51-52, 56) [DOI:10.3969/j.issn.1003-9767.2017.23.023]

Dong Y Y, Yang W K, Wang J W, Zhao Z J, Wang S H, Cui Q, Qiang Y. 2020. An improved supervoxel 3D region growing method based on PET/CT multimodal data for segmentation and reconstruction of GGNs. Multimedia Tools and Applications, 79(3): 2309-2338 [DOI:10.1007/s11042-019-08250-4]

Drokin I and Ericheva E. 2020. Deep learning on point clouds for false positive reduction at nodule detection in chest CT scans[EB/OL].[2020-05-31]. https://arxiv.org/pdf/2005.03654.pdf

Zanjani F G, Moin D A, Verheij B, Claessen F, Cherici T and Tan T. 2019. Deep learning approach to semantic segmentation in 3 d point cloud intra-oral scans of teeth//Proceedings of 2019 International Conference on Medical Imaging with Deep Learning. London: MIDL: 557-571

Ghazvinian Z F, Anssari M D, Verheij B, Claessen F, Cherici T and Tan T. 2019. Deep learning approach to semantic segmentation in 3D point cloud intra-oral scans of teeth//Proceedings of the 2nd International Conference on Medical Imaging with Deep Learning. London: MIDL: 557-571[DOI: 10.5194/isprs-archives-XLII-2-W15-735-2019]

Graham B, Engelcke M and van der Maaten L. 2018.3D semantic segmentation with submanifold sparse convolutional networks//Proceedings of 2018 IEEE/CVF Conference on Computer vision and Pattern Recognition. Salt Lake City: IEEE: 9224-9232[DOI: 10.1109/cvpr.2018.00961]

Guo J, Yao X X, Shen M Y, Wang J F and Liao W Y. 2018. A deep learning network for point cloud of medicine structure//Proceedings of the 9th International Conference on Information Technology in Medicine and Education. Hangzhou: IEEE: 683-687[DOI: 10.1109/ITME.2018.00157]

Guo Y L, Wang H Y, Hu Q Y, Liu H, Liu L and Bennamoun M. 2019. Deep learning for 3D point clouds: a survey[EB/OL].[2020-05-31]. https://arxiv.org/pdf/1912.12033.pdf

Gupta D, Anand R S. 2017. A hybrid edge-based segmentation approach for ultrasound medical images. Biomedical Signal Processing and Control, 31: 116-126 [DOI:10.1016/j.bspc.2016.06.012]

Gutiérrez-Becker B and Wachinger C. 2018. Deep multi-structural shape analysis: application to neuroanatomy//Proceedings of the 21st International Conference on Medical Image Computing and Computer-Assisted Intervention. Granada: Springer: 523-531[DOI: 10.1007/978-3-030-00931-1_60]

Hanocka R, Hertz A, Fish N, Giryes R, Fleishman S, Cohen-Or D. 2019. MeshCNN:a network with an edge. ACM Transactions on Graphics, 38(4) [DOI:10.1145/3306346.3322959]

Hansen L, Dittmer D and Heinrich M P. 2019. Learning deformable point set registration with regularized dynamic graph CNNS for large lung motion in COPD patients//Proceedings of the 1st International Workshop on Graph Learning in Medical Imaging. Shenzhen: Springer: 53-61[DOI: 10.1007/978-3-030-35817-4_7]

Hua B S, Tran M K and Yeung S K. 2018. Pointwise convolutional neural networks//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE: 984-993[DOI: 10.1109/cvpr.2018.00109]

Huang J D, Kwok T H, Zhou C. 2017. V4PCS:volumetric 4PCS algorithm for global registration. Journal of Mechanical Design, 139(11): 111403 [DOI:10.1115/1.4037477]

Le T and Duan Y. 2018. Pointgrid: a deep network for 3d shape understanding//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE: 9204-9214[DOI: 10.1109/cvpr.2018.00959]

Li J X, Chen B M and Hee Lee G. 2018a. So-net: Self-organizing network for point cloud analysis//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE: 9397-9406[DOI: 10.1109/cvpr.2018.00979]

Li W, Zhang Y J, Hu Y, Chen Q, Tang W, Wang H. 2015. Combination of laser-point cloud and reverse engineering to rapidly establish a three-dimensional soft tissue model in cosmetic surgery. Chinese Journal of Tissue Engineering Research, 19(15): 2346-2350 (李伟, 张玉洁, 胡筠, 陈淇, 汤炜, 王杭. 2015. 激光点云结合逆向工程快速构建软组织立体模型在美容外科的应用. 中国组织工程研究, 19(15): 2346-2350) [DOI:10.3969/j.issn.2095-4344.2015.15.010]

Li Y Y, Bu R, Sun M C, Wu W, Di X H and Chen B Q. 2018b. Pointcnn: Convolution on x-transformed points//Proceedings of Annual Conference on Neural Information Processing Systems 2018. Montréal: Curran Associates: 820-830

Lian C F, Wang L, Wu T H, Liu M X, Durán F, Ko C C and Shen D G. 2019. MeshSNet: deep multi-scale mesh feature learning for end-to-end tooth labeling on 3D dental surfaces//Proceedings of the 22nd International Conference on Medical Image Computing and Computer-Assisted Intervention. Shenzhen: Springer: 837-845[DOI: 10.1007/978-3-030-32226-7_93]

Liang Z F, Guo Y L, Feng Y L, Chen W, Qiao L B, Zhou L, Zhang J F and Liu H Z. 2019. Stereo matching using multi-level cost volume and multi-scale feature constancy. IEEE Transactions on Pattern Analysis and Machine Intelligence: #8765737[DOI: 10.1109/TPAMI.2019.2928550]

Liu W P, Sun J, Li W Y, Hu T, Wang P. 2019. Deep learning on point clouds and its application:a survey. Sensors, 19(19) [DOI:10.3390/s19194188]

Liu Y L, Song Z J, Wang M N. 2017. A new robust markerless method for automatic image-to-patient registration in image-guided neurosurgery system. Computer Assisted Surgery, 22(S1): 319-325 [DOI:10.1080/24699322.2017.1389411]

Lu W X, Wan G W, Zhou Y, Fu X Y, Yuan P F and Song S Y. 2019. DeepVCP: an end-to-end deep neural network for point cloud registration//Proceedings of 2019 IEEE/CVF International Conference on Computer Vision. Seoul: IEEE: 12-21[DOI: 10.1109/ICCV.2019.00010]

Ma D, Zhang D Q, Zhang W B and Li X. 2017. Reconstruction of human model 3D model based on reverse engineering technology. Machinery Design and Manufacture, (2): 91-93 [DOI: 10.3969/j.issn.1001-3997.2017.02.025] (马丹, 张德强, 张文博, 李新. 2017.基于逆向工程技术的人体牙模3维模型重构.机械设计与制造, (2): 91-93[DOI: 10.3969/j.issn.1001-3997.2017.02.025])

Mellado N, Aiger D, Mitra N J. 2014. Super 4PCS fast global pointcloud registration via smart indexing. Computer Graphics Forum, 33(5): 205-215 [DOI:10.1111/cgf.12446]

Nguyen L, Stoter S, Baum T, Kirschke J, Ruess M, Yosibash Z, Schillinger D. 2017. Phase-field boundary conditions for the voxel finite cell method:surface-free stress analysis of CT-based bone structures. International Journal for Numerical Methods in Biomedical Engineering, 33(12) [DOI:10.1002/cnm.2880]

Porto L and Rohling R. 2020. Improving Interpretability of 2-D ultrasound of the lumbar spine//The 17th IEEE International Symposium on Biomedical Imaging. Iowa City: IEEE: 1-5[DOI: 10.1109/ISBI45749.2020.9098446]

Qi C R, Su H, Kaichun M and Guibas L J. 2017a. PointNet: deep learning on point sets for 3D classification and segmentation//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE: 652-660[DOI: 10.1109/CVPR.2017.16]

Qi C R, Yi L, Su H and Guibas L J. 2017b. Pointnet++: deep hierarchical feature learning on point sets in a metric space//Proceedings of Annual Conference on Neural Information Processing Systems 2017. Long Beach: NIPS: 5099-5108

Schaffert R, Wang J, Fischer P, Borsdorf A, Maier A. 2018. Metric-driven learning of correspondence weighting for 2-D/3-D image registration//Proceedings of the 40th German Conference on Pattern Recognition. Stuttgart:Springer [DOI:10.1007/978-3-030-12939-2_11]

Setio A A A, Ciompi F, Litjens G, Gerke P, Jacobs C, Van Riel S J, Wille M M W, Naqibullah M, Sánchez C I, van Ginneken B. 2016. Pulmonary nodule detection in CT images:false positive reduction using multi-view convolutional networks. IEEE Transactions on Medical Imaging, 35(5): 1160-1169 [DOI:10.1109/TMI.2016.2536809]

Sinko M, Kamencay P, Hudec R and Benco M. 2018.3D registration of the point cloud data using ICP algorithm in medical image analysis//Proceedings of 2018 ELEKTRO. Mikulov: IEEE: 1-6[DOI: 10.1109/ELEKTRO.2018.8398245]

Su H, Maji S, Kalogerakis E and Learned-Miller E. 2015. Multi-view convolutional neural networks for 3D shape recognition//Proceedings of 2015 IEEE International Conference on Computer Vision. Santiago: IEEE: 945-953[DOI: 10.1109/ICCV.2015.114]

Tian S K, Dai N, Zhang B, Yuan F L, Yu Q, Cheng X S. 2019. Automatic classification and segmentation of teeth on 3D dental model using hierarchical deep learning networks. IEEE Access, 7: 84817-84828 [DOI:10.1109/ACCESS.2019.2924262]

Wang Y and Solomon J M. 2019. Deep closest point: Learning representations for point cloud registration//Proceedings of 2019 IEEE/CVF International Conference on Computer Vision. Seoul: IEEE: 3523-3532[DOI: 10.1109/ICCV.2019.00362]

Wang Y, Sun Y B, Liu Z W, Sarma S E, Bronstein M M, Solomon J M. 2019. Dynamic graph CNN for learning on point clouds. ACM Transactions on Graphics, 38(5): #146 [DOI:10.1145/3326362]

Wu B C, Wan A, Yue X Y and Keutzer K. 2018. SqueezeSeg: convolutional neural nets with recurrent CRF for real-time road-object segmentation from 3D LiDAR point cloud//Proceedings of 2018 IEEE International Conference on Robotics and Automation. Brisbane: IEEE: 1887-1893[DOI: 10.1109/ICRA.2018.8462926]

Wu W X, Qi Z A and Li F X. 2019. PointConv: deep convolutional networks on 3d point clouds//Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach: IEEE: 9621-9630[DOI: 10.1109/cvpr.2019.00985]

Xie Y X, Tian J J and Zhu X X. 2019. Linking points with labels in 3D: a review of point cloud semantic segmentation[EB/OL].[2020-05-30]. https://arxiv.org/pdf/1908.08854.pdf

Xu Y F, Fan T Q, Xu M Y, Zeng L and Qiao Y. 2018. SpiderCNN: deep learning on point sets with parameterized convolutional filters//Proceedings of the 15th European Conference on Computer Vision. Munich: Springer: 87-102[DOI: 10.1007/978-3-030-01237-3_6]

Yang L and Chakraborty R. 2019. A GMM based algorithm to generate point-cloud and its application to neuroimaging[EB/OL].[2020-05-30]. https://arxiv.org/pdf/1911.01705.pdf

Yang S M, Shi X Y, Zhang G F, Lv C S. 2018. A dual-platform laser scanner for 3D reconstruction of dental pieces. Engineering, 4(6): 796-805 [DOI:10.1016/j.eng.2018.10.005]

Yang X, Xia D, Kin T and Igarashi T. 2020. IntrA: 3D intracranial aneurysm dataset for deep learning[EB/OL].[2020-05-30]. https://arxiv.org/pdf/2003.02920.pdf