Feature detection algorithm of musculoskeletal ultrasound image and its application of image stitching

Huanhuan Yan; Peizhen Zhang; Yinong Wang; Xuan Li; Wei Yang; Qing Wang

doi:10.11834/jig.190339

Medical Image Processing | Views : 0 下载量: 41 CSCD: 1

PDF
Export
Share
Collection
Album

Feature detection algorithm of musculoskeletal ultrasound image and its application of image stitching
Vol. 25, Issue 5, Pages: 1032-1042(2020)
Received：19 July 2019，

Revised：2019-9-23，

Accepted：30 September 2019，

Published：16 May 2020
DOI： 10.11834/jig.190339
稿件说明：

移动端阅览

DOI：

Huanhuan Yan, Peizhen Zhang, Yinong Wang, Xuan Li, Wei Yang, Qing Wang. Feature detection algorithm of musculoskeletal ultrasound image and its application of image stitching[J]. Journal of Image and Graphics, 2020, 25(5): 1032-1042. DOI： 10.11834/jig.190339.

摘要

目的

肌骨超声宽景图像易出现解剖结构错位、断裂等现象，其成像算法中的特征检测影响宽景图像的质量，也是超声图像配准、分析等算法的关键步骤，但目前仍未有相关研究明确指出适合提取肌骨超声图像特征点的算法。本文利用结合SIFT（scale invariant feature transform）描述子的FAST（features from accelerated segment test）算法以及SIFT、SURF（speeded-up robust features）、ORB（oriented FAST and rotated binary robust independent elementary features（BRIEF））算法对肌骨超声图像序列进行图像拼接，并对各算法的性能进行比较评估，为肌骨超声图像配准、宽景成像提供可参考的特征检测解决方案。

方法

采集5组正常股四头肌的超声图像序列，每组再采样10幅图像。利用经典的图像拼接算法进行肌骨图像的特征检测以及图像拼接。分别利用上述4种算法提取肌骨超声图像的特征点；对特征点进行特征匹配，估算出图像间的形变矩阵；对所有待拼接的图像进行坐标变换以及融合处理，得到拼接全景图，并在特征检测性能、特征匹配性能、图像配准性能以及拼接效果等方面对4种算法进行评估比较。

结果

实验结果表明，与SIFT、SURF、ORB算法相比，FAST-SIFT算法所提取的特征点分布更均匀，可以检测到大部分肌纤维的端点，且特征点检测时间最短，约4 ms，其平均匹配对数最多，是其他特征检测算法的25倍，其互信息和归一化互相关系数均值分别为1.016和0.748，均高于其他3种特征检测算法，表明其图像配准精度更高。且FAST-SIFT算法的图像拼接效果更好，没有明显的解剖结构错位、断裂、拼接不连贯等现象。

结论

与SIFT、SURF、ORB算法相比，FAST-SIFT算法是更适合提取肌骨超声图像特征点的特征检测算法，在图像配准精度等方面都具有一定的优势。

Abstract

Objective

Musculoskeletal ultrasound (MSKUS) is an imaging diagnosis method commonly applied in the diagnosis and treatment of musculoskeletal diseases. The feature detection of MSKUS image plays an important role in image registration

image analysis of MSKUS images

and extended field-of-view ultrasound imaging

requiring extraction of the effective feature points. However

the contrast of the ultrasound image is low

and speckle noise and image artifacts are presented in the MSKUS images. These limitations negatively affect the extraction of the feature points of MSKUS image. Consequently

the accuracy of image registration and the quality of image stitching are affected. This condition may lead to misalignment and fracture of anatomical structure on the MSKUS panoramic image. An algorithm suitable for detecting feature points of MSKUS images has not been clearly determined. The objectives of this study are to evaluate the performance of the four local feature detection algorithms on stitching MSKUS sequence images

including scale invariant feature transform (SIFT)

speeded-up robust features (SURF)

oriented FAST and rotated binary robust independent elementary features(ORB)

and features from accelerated segment test (FAST) combined with SIFT descriptor

and to provide a basis and reference solution of feature detection for MSKUS image registration and extended field-of-view ultrasound imaging in future research.

Method

Ultrasound image sequences of the quadriceps muscles in five normal human subjects are collected. From the image sequence of each subject

10 images are resampled every five frames for image feature detection and image stitching. The classical image stitching method proposed by Brown is adopted in this study

which includes the following three main steps. First

the feature points of the MSKUS image are extracted by SIFT

SURF

ORB

and FAST-SIFT. Then

based on the obtained feature points and their corresponding feature point descriptors

the nearest neighbor distance ratio method is applied to achieve rough feature matching

and the random sample consensus (RANSAC) algorithm is used to realize fine feature matching. The projection transformation matrix is taken as the basic model to estimate the optimal deformation matrix between the two images. Finally

the deformation matrix between the two images is used to obtain the internal parameters of the camera and the external parameters of the camera. These camera parameters can be used to transform the 10 images into the coordinate system of the reference image. After coordinate transformation

all the 10 images are stitched together. Then

the MSKUS panorama is post-processed using the maximum flow minimum cut algorithm to find the unwell-stitched overlapped area. The multiband fusion is used to reduce the artificial and rough seam zones. The MSKUS panorama is eventually obtained. To evaluate the performance of the four algorithms SIFT

SURF

ORB algorithms

and FAST-SIFT

detection of feature points

feature matching

image registration

and image stitching are assessed.

Result

The experimental results show that compared with SIFT

SURF

and ORB algorithms

FAST-SIFT is able to extract more uniform distribution feature points and detect most of the end points of the muscle fibers. Furthermore

the detection time of feature points by using FAST-SIFT is much shorter of approximately 4 ms.The average number of matching points in FAST-SIFT is the largest

which is 2~5 times as many as other feature detection algorithms. FAST-SIFT algorithm has the highest correct rate of feature matching. The mean value of mutual information and normalized cross-correlation coefficient of FAST-SIFT are 1.016 and 0.748

respectively

which are higher than that of the other three feature detection algorithms. Result indicates high accuracy of image registration. Moreover

the stitched panorama of MSKUS by using the FAST-SIFT algorithm shows good image stitching

that is

no obvious misalignment and fracture of anatomical structure are found.

Conclusion

Compared with SIFT

SURF

and ORB

FAST-SIFT algorithm is more suitable to extract the feature points of MSKUS image. It has advantages in the distribution of feature points

the detection time of feature points

the average number of matching points

the correct rate of feature matching

image registration accuracy

and image stitching result.

关键词

Keywords

references

Bay H, Ess A, Tuytelaars T and Van Gool L. 2008. Speeded-up robust features (SURF). Computer Vision and Image Understanding, 110(3):346-359[DOI:10.1016/j.cviu.2007.09.014]

Boykov Y and Kolmogorov V. 2004. An experimental comparison of min-cut/max-flow algorithms for energy minimization in vision. IEEE Transactions on Pattern Analysis and Machine Intelligence, 26(9):1124-1137[DOI:10.1109/TPAMI.2004.60]

Brown M and Lowe D G. 2007. Automatic panoramic image stitching using invariant features. International Journal of Computer Vision, 74(1):59-73[DOI:10.1007/s11263-006-0002-3]

Calonder M, Lepetit V, Strecha C and Fua P. 2010. BRIEF: binary robust independent elementary features//Proceedings of the 11th European Conference on Computer Vision. Heraklion, Crete, Greece: Springer: 778-792[ DOI:10.1007/978-3-642-15561-1_56 http://dx.doi.org/10.1007/978-3-642-15561-1_56 ]

Feng X Y. 2014. Study on the Technology and Algorithm Related to the Real-Time Construction of Wide Field View of Medical Ultrasonic Image. Qinhuangdao: Yanshan University

冯希玉. 2014.实时宽景医学超声图像获取的相关技术与算法研究.秦皇岛: 燕山大学

Fischler M A and Bolles R C. 1981. Random sample consensus:a paradigm for model fitting with applications to image analysis and automated cartography. Communications of the ACM, 24(6):381-395[DOI:10.1145/358669.358692]

Gauglitz S, Höllerer T and Turk M. 2011. Evaluation of interest point detectors and feature descriptors for visual tracking. International Journal of Computer Vision, 94(3):335-360[DOI:10.1007/s11263-011-0431-5]

Huang Q H, Deng Q F, Li L, Yang J L and Li X L. 2019. Scoliotic imaging with a novel double-sweep 2.5-dimensional extended field-of-view ultrasound. IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control, 66(8):1304-1315[DOI:10.1109/TUFFC.2019.2920422]

Huang Q H, Zeng Z Z and Li X L. 2018. 2.5-D extended field-of-view ultrasound. IEEE Transactions on Medical Imaging, 37(4):851-859[DOI:10.1109/TMI.2017.2776971]

Kong L Y, Huang Q H, Lu M H, Zheng S H, Jin L W and Chen S P. 2010. Accurate image registration using SIFT for extended-field-of-view sonography//Proceedings of the 4th International Conference on Bioinformatics and Biomedical Engineering. Chengdu, China: IEEE: 1-4[ DOI:10.1109/ICBBE.2010.5517503 http://dx.doi.org/10.1109/ICBBE.2010.5517503 ]

Lourakis M I A. 2005. A brief description of the Levenberg-Marquardt algorithm implemened by Levmar[EB/OL ] .[2019-05-22 ] . http://web.eecs.utk.edu/[XC范围号.tif,JZ]mjr/ECE505/Miscellaneous/levmar.pdf http://web.eecs.utk.edu/[XC范围号.tif,JZ]mjr/ECE505/Miscellaneous/levmar.pdf

Lowe D G. 2004. Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision, 60(2):91-110[DOI:10.1023/B:VISI.0000029664.99615.94]

Mani V R S and Arivazhagan S. 2013. Survey of medical image registration. Journal of Biomedical Engineering and Technology, 1(2):8-25[DOI:10.12691/jbet-1-2-1]

Meng X F. 2014. On the Trajectory Tracking of Right Ventricles Using Two-Dimensional Ultrasound Images. Shenyang: Northeastern University http://cdmd.cnki.com.cn/Article/CDMD-10145-1015707414.htm .

孟祥菲. 2014.基于二维超声图像的右心室运动轨迹跟踪.沈阳: 东北大学

Rosten E, Porter R and Drummond T. 2010. Faster and better:a machine learning approach to corner detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 32(1):105-119[DOI:10.1109/TPAMI.2008.275]

Rublee E, Rabaud V, Konolige K and Bradski G. 2011. ORB: an efficient alternative to SIFT or SURF//Proceedings of 2011 International Conference on Computer Vision. Barcelona, Spain: IEEE: 2564-2571[ DOI:10.1109/ICCV.2011.6126544 http://dx.doi.org/10.1109/ICCV.2011.6126544 ]

Shum H Y and Szeliski R. 2000. Systems and experiment paper:construction of panoramic image mosaics with global and local alignment. International Journal of Computer Vision, 36(2):101-130[DOI:10.1023/a:1008195814169]

Strakowski J A and Visco C J. 2019. Diagnostic and therapeutic musculoskeletal ultrasound applications of the shoulder. Muscle&Nerve, 60(1):1-6[DOI:10.1002/mus.26505]

Triggs B, McLauchlan P F, Hartley R I and Fitzgibbon A W. 1999. Bundle adjustment-a modern synthesis//International Workshop on Vision Algorithms: Theory and Practice. Berlin, Heidelberg: Springer: 298-372[ DOI:10.1007/3-540-44480-7_21 http://dx.doi.org/10.1007/3-540-44480-7_21 ]

Wang G, Wang Z C, Chen Y F and Zhao W D. 2015a. Robust point matching method for multimodal retinal image registration. Biomedical Signal Processing and Control, 19:68-76[DOI:10.1016/j.bspc.2015.03.004]

Wang Y, Ban X J, Chen J, Hu B and Yang X. 2015b. License plate recognition based on SIFT feature. Optik, 126(21):2895-2901[DOI:10.1016/j.ijleo.2015.07.040]

Wei Q, Dobigeon N and Tourneret J Y. 2015. Bayesian fusion of multi-Band Images. IEEE Journal of Selected Topics in Signal Processing, 9(6):1117-1127[DOI:10.1109/JSTSP.2015.2407855]

Weng L, Tirumalai A P, Lowery C M, Nock L F, Gustafson D E, Von Behren P L and Kim J H. 1997. US extended-field-of-view imaging technology. Radiology, 203(3):877-880[DOI:10.1148/radiology.203.3.9169720]

Zheng S H, Huang Q H, Jin L W and Wei G. 2012. Real-time extended-field-of-view ultrasound based on a standard PC. Applied Acoustics, 73(4):423-432[DOI:10.1016/j.apacoust.2011.09.013]

Alert me when the article has been cited

提交

High-precision parallax image stitching method using feature points clustering

Dominant sub-plane registration algorithm for large parallax image stitching

Vision Character based Multi-Sensor Image Registration

Review of remote sensing image matching methods for lunar exploration

Indoor large-scale panoramic visual localization dataset