Current Issue Cover
集成注意力增强和双重相似性引导的多模态脑部图像配准

田梨梨1,2, 程欣宇1,2, 唐堃1,2, 张健1,2, 王丽会1,2(1. 贵州省智能医学影像分析与精准诊断重点实验室, 贵阳 550025;2.
2. 贵州大学计算机科学与技术学院, 贵阳 550025)

摘 要
目的 医学图像配准是医学图像处理和分析的关键环节,由于多模态图像的灰度、纹理等信息具有较大差异,难以设计准确的指标来量化图像对的相似性,导致无监督多模态图像配准的精度较低。因此,本文提出一种集成注意力增强和双重相似性引导的无监督深度学习配准模型(ensemble attention-based and dual similarity guidance registration network,EADSG-RegNet),结合全局灰度相似性和局部特征相似性共同引导参数优化,以提高磁共振T2加权图像和T1加权模板图像配准的精度。方法 EADSG-RegNet模型包含特征提取、变形场估计和重采样器。设计级联编码器和解码器实现图像对的多尺度特征提取和变形场估计,在级联编码器中引入集成注意力增强模块(integrated attention augmentation module,IAAM),通过训练的方式学习提取特征的重要程度,筛选出对配准任务更有用的特征,使解码器更准确地估计变形场。为了能够准确估计全局和局部形变,使用全局的灰度相似性归一化互信息(normalized mutual information,NMI)和基于SSC (self-similarity context)描述符的局部特征相似性共同作为损失函数训练网络。在公开数据集和内部数据集上验证模型的有效性,采用Dice分数对配准结果在全局灰质和白质以及局部组织解剖结构上作定量分析。结果 实验结果表明,相比于传统配准方法和深度学习配准模型,本文方法在可视化结果和定量分析两方面均优于其他方法。对比传统方法ANTs (advanced normalization tools)、深度学习方法voxelMorph和ADMIR (affine and deformable medical image registration),在全局灰质区域,Dice分数分别提升了3.5%,1.9%和1.5%。在全局白质区域分别提升了3.4%,1.6%和1.3%。对于局部组织结构,Dice分数分别提升了5.2%,3.1%和1.9%。消融实验表明,IAAM模块和SSC损失分别使Dice分数提升1.2%和1.5%。结论 本文提出的集成注意力增强的无监督多模态医学图像配准网络,通过强化有用特征实现变形场的准确估计,进而实现图像中细小区域的准确配准,对比实验验证了本文模型的有效性和泛化能力。
关键词
Multimodal brain image registration with integrated attention augmentation and dual similarity guidance

Tian Lili1,2, Cheng Xinyu1,2, Tang Kun1,2, Zhang Jian1,2, Wang Lihui1,2(1. Key Laboratory of Intelligent Medical Image Analysis and Precise Diagnosis of Guizhou Province, Guiyang 550025, China;2.
2. School of Computer Science and Technology, Guizhou University, Guiyang 550025, China)

Abstract
Objective Medical image registration has been widely used on the aspect of clinical diagnosis, treatment, intraoperative navigation, disease prediction and radiotherapy planning. Non-learning registration algorithms have matured nowadays in common. Non-learning-based registration algorithms have optimized the deformation parameters iteratively to cause poor robustness because of the huge limitations in the computation speed. Various deep convolution neural networks (DCNNs) models have been running in medical image registration due to the powerful feature expression and learning. DCNNs-based image registration has been divided into supervised and unsupervised categories. The supervised-learning-based registration algorithms have intensive data requirements, which require locking the anatomical landmarks to identify the deformation areas, the performance of reliability of the landmarks has been greatly relied on even the supervised-learning based registration algorithm plays well. Real label information still cannot be acquired. Scholars have focused on unsupervised image registration to complete the defects of supervised image registration. To assess the deformation parameters of the image pair directly via appropriate optimization goals and deformation area constraints. It is difficult to design accurate metric to quantify the similarity of image pairs because the low multimodal images (MI)-based demonstration accuracy in the context of the quite differences amongst content, grayscale, texture and others. Unsupervised registration has been opted in appropriate image similarity to optimize targets involving mean square error, correlation coefficient and normalized mutual information. Most of these similarity assessments have been based on global gray scale. The local deformation still cannot be assessed accurately via good quality e registration structure. An integrated ensemble attention-based augmentation and dual similarity guidance registration network(EADSG-RegNet) has upgraded the registration accuracy of T2-weighted magnetic resonance image and T1-weighted magnetic resonance template image. Method EADSG-RegNet network has been designated to assess the deformation area between the moving and fixed image pairs. The feature extraction, deformation field estimation and resampler have been illustrated in the network mentioned above. A cascade encoder and encoder have been designed to realize the multi-scale feature extraction and deformation area assessment based on U-Net structure modification. An integrated attention augmentation module (IAAM) in the cascade encoder to improve feature extraction capabilities have been demonstrated to improve the accuracy of registration. In a word, the extracted features have been learned to decode the deformation area accurately. Integrated attention augmentation module has been applied to generate the weights of feature channels of the global average feature via global average pooling of the input feature map. The global feature channels (the number of channels is n) are shuffled firstly for twice obtain 3×n channels have been calculated in total. Each shuffled global channel feature block has been deducted in dimension via a 1×1×1 convolution. Next, the concatenated features have been mapped to 1×1×1×n weighting coefficients via weighting coefficient to multiply the original feature maps for bottleneck to generate the attention features. The global and local deformation can be accurately assessed in the network training stage. The applications of global gray-scale similarity normalized mutual information (NMI) and the local feature similarity based on the self-similarity context (SSC) descriptor as the loss function to guide the training of the network. The smoothness of the deformation area has been maintained and a regularization has been added to the loss function. Internal dataset and public dataset have been added to verify the performance and generalizability of the model. All T2 weighted magnetic resonance images have been preprocessed firstly and a given T1 template has been pre-aligned. The effectiveness of the network in terms of visualization results and quantitative analysis results have been analyzed. Dice score has been used to analyze the registration results quantitatively. The registration results have been assessed in the global gray matter, white matter and local organizational structures respectively. Result To assess the performance of the registration model, the symmetric image normalization method(SyN) implemented in advanced normalization tolls(ANTs) software package, the deep learning registration models voxelMorph framework and affine and deformable medical image registration(ADMIR), which are the state-of-the-art algorithms in traditional and deep learning-based registration methods. This research has analyzed the registration results quantitatively via the overall structure and several local anatomical structures. The gray matter and white matter have been automatically segmented using FMRIB Software Library(FSL). Nine small anatomical structures have been segmented manually using ITK-Snap. Compared with the ANTs, voxelMorph and ADMIR, the average Dice score on gray matter increased by 3.5%, 1.9%, 1.5%. The average Dice score on white matter increased by 3.4%, 1.6%, 1.3%. For the nine anatomical structures, the average Dice score of the proposed model has been increased by 5.2%, 3.1%, 1.9%. In addition, the registration speed has been improved by dozens of times compared with the traditional ANTs algorithm. The impact of the attention module and feature-based similarity loss on the registration results have been further illustrated. This research have done the ablation experiments of IAAM and SSC-based loss further. The results have demonstrated that the IAAM and the SSC-based loss can increase the Dice score in 1.2% and 1.5% respectively. The registration models have been illustrated to get consistent results with the clinical research via analyzing the volume difference in some brain regions between control groups and drug addicts. Conclusion The unsupervised multimodal medical image registration network with integrated attention augmentation module has been illustrated to achieve accurate estimation of the deformation area based on augmented features and accurate registration.
Keywords

订阅号|日报