浏览全部资源
扫码关注微信
收稿：2019-08-16，
修回：2019-10-11，
录用：2019-10-18，
纸质出版：2020-05-16
移动端阅览
目的
2
为解决小重叠度、大视角影像自然拼接问题，提出一种统筹图像变换和缝合线生成过程的影像拼接新方法。
方法
2
主要包括两部分：1）将单应性映射引入到薄板样条变换（thin plate spline，TPS）并建立二次图像变换模型，在同一变换框架下进行全局单应性透视变换及基于径向基函数的局部映射调整，实现拼接影像透视特性良好保持与减少局部变形，且有效满足缝合线控制点无误差配准计算需要；2）根据控制点像素配准精度优于非控制点像素配准精度，对基准影像控制点集进行三角构网，并从中搜索初始缝合线，结合二次配准模型参数和动态规划匹配过程，获得控制点更密集且配准误差总体最小的最终缝合线，有效抑制图像融合时鬼影现象的产生，且计算实现上更简单。
结果
2
对两组标准影像进行拼接测试，并与3种方法（双单应性变换（dual-homography warping，DHW）、平滑变化的仿射拼接（smoothly varying affine stitching，SVA）、尽可能投影（as-projective-as-possible，APAP）的移动直接线性变换拼接（image stitching with moving direct linenr transform（DLT））对比，本文方法拼接影像局部变形失真最小、匹配控制点配准精度最高：在railtracks影像训练集（trarning set，TR）、测试集（test set）上分别为0和2.27；在temple影像训练集TR、测试集TE上分别为0和1.46；不同场景多路采集视频影像拼接透视特性保持良好，视觉效果自然。
结论
2
本文方法影像拼接时无先验知识要求、参数可线性求解，整体视觉效果流畅、重叠区域与非重叠区域过渡平滑，具有较好的应用价值。
Objective
2
Image stitching technology aims to resample a group of interrelated (overlapping) images into a single image with a large angle of view and high resolution. This technology effectively overcomes the density limitation of a sensor array on the focal plane of common image acquisition equipment. Currently
numerous applications
such as realize virtual reality
security monitoring
and remote sensing detection
use the image stitching technology to obtain visual information of macro scenes and achievements have been made in the development of image stitching technology. However
due to the complexity of scene content
illumination conditions
and the changeability of camera pose
the acquisition of natural-looking visual effects of stitching image still depends mainly on post-processing interactively. Especially for applications with low measurement requirements and emphasis on visual experience
the parameter-free image stitching technology ignoring prior knowledge (scene 3D structure
camera parameters) is more valuable. To this end
this study presents a novel approach of stitching parametric-free images by comprehensively considering the process of image registration and fusion.
Method
2
The ghost phenomenon originates from an inaccurate image transformation model. Potential parameter estimation error has the greatest impact on the visual experience of stitching images. At present
the effective way to solve the problem is to find an optimal seam-line in overlapping regions of images to be stitched and take only one image pixel on each side of the seam-line for image fusion operation. Usually
the parameters of image transformation model are estimated using a set of control points located in overlapping regions of images. Thus
the accuracy of transformation at control points is better than that at non-control points. Accordingly
this study proposes to automatically generate an optimal seam-line from matched feature points for image stitching purpose. This technology mainly involves two aspects. The first one is the twice registration framework based on modified TPS (thin plate spline) transformation. It is established to meet the needs of error-free control point registration and automatic generation of reliable seam-line. In this step
the global homograph mapping and local mapping adjustment based on radial basis function are ingeniously incorporated into the uniform spatial transformation framework. Thus
it maintains perspective geometry and reduces local distortion in stitching image as much as possible. The second one is the reliable seam-line generated based on the fact that the registration accuracy of control point pixels is better than that of non-control point pixels in the process of image transformation. In this step
the initial seam-line is primarily searched out from triangulated control points in the reference image. Then
it is refined by adding more control points through dynamic programming matching process and remapping non-control point pixels with the transformation parameters estimated in advance. With the small registration error of pixels on seam-line
troublesome ghost phenomenon in stitching images can be effectively suppressed with computationally simpler image fusion operation.
Result
2
The experiments on two sets of standard images from the Internet are performed and compared with state-of-the-art methods in extant literature. The transformation accuracy of training set and test set of matching control points is found to be the highest in this study
whereas the local distortion of mosaic images is the smallest. Moreover
the stitching test is also performed on multiple videos with two different scenes. Its visual mosaic effect is quite natural.
Conclusion
2
The proposed method has a natural transition between images' overlapping and non-overlapping areas and achieves better visual effect in stitching images. Furthermore
it requires no prior knowledge on images
and its parameters can be solved linearly. Thus
it exhibits huge potential for application.
Bookstein F L. 1989. Principal warps:thin-plate splines and the decomposition of deformations. IEEE Transactions on Pattern Analysis and Machine Intelligence, 11(6):567-585[DOI:10.1109/34.24792]
Chang C H, Sato Y and Chuang Y Y. 2014. Shape-preserving half-projective warps for image stitching//Proceedings of 2014 IEEE Conference on Computer Vision and Pattern Recognition. Columbus: IEEE: 3254-3261[ DOI:10.1109/CVPR.2014.422 http://dx.doi.org/10.1109/CVPR.2014.422 ]
Chen H, Long A Q and Peng Y H. 2009. Building panoramas from photographs taken with an uncalibrated hand-held camera. Chinese Journal of Computers, 32(2):328-335
陈辉, 龙爱群, 彭玉华. 2009.由未标定手持相机拍摄的图片构造全景图.计算机学报, 32(2):328-335)[DOI:10.3724/SP.J.1016.2009.00328]
Cheng Z G and Zhang L. 2016. An aerial image mosaic method based on UAV position and attitude information. Acta Geodaetica et Cartographica Sinica, 45(6):698-705
程争刚, 张利. 2016.一种基于无人机位姿信息的航拍图像拼接方法.测绘学报, 45(6):698-705)[DOI:10.11947/j.AGCS.2016.20150567]
Chon J, Kim H and Lin C S. 2010. Seam-line determination for image mosaicking:a technique minimizing the maximum local mismatch and the global cost. ISPRS Journal of Photogrammetry and Remote Sensing, 65(1):86-92[DOI:10.1016/j.isprsjprs.2009.09.001]
Dornaika F and Chung R. 2004. Mosaicking images with parallax. Signal Processing:Image Communication, 19(8):771-786[DOI:10.1016/j.image.2004.06.008]
Du W Z, Song Z H, Yan W B and Wu L G. 2016. Fast image stitching under single camera rotate monitoring. Journal of Image and Graphics, 21(2):245-254
杜往泽, 宋执环, 闫文博, 吴乐刚. 2016.单摄像头旋转监控下的快速图像拼接.中国图象图形学报, 21(2):245-254)[DOI:10.11834/jig.20160214]
Duplaquet M L. 1998. Building large image mosaics with invisible seam lines//Proceedings of the SPIE 338, Visual Information Processing Ⅶ. Orlando: SPIE: 369-377[ DOI:10.1117/12.316427 http://dx.doi.org/10.1117/12.316427 ]
Fang X Y, Zhang M M, Pan Z G, Luo B and Wang P. 2007. Research of image mosaic based on graph cut. Journal of Image and Graphics, 12(12):2050-2056
方贤勇, 张明敏, 潘志庚, 罗斌, 王鹏. 2007.基于图切割的图像拼接技术研究.中国图象图形学报, 12(12):2050-2056)[DOI:10.11834/jig.20071203]
Gao J H, Kim S J and Brown M S. 2011. Constructing image panoramas using dual-homography warping//Proceedings of CVPR 2011. Colorado Springs: IEEE: 49-56[ DOI:10.1109/CVPR.2011.5995433 http://dx.doi.org/10.1109/CVPR.2011.5995433 ]
Guo D B, Lu Z Y, Jiao W D and Cui L L. 2009. Dynamic programming in segments for occlusion detection/stereo matching. Acta Electronica Sinica, 37(7):1516-1521
郭大波, 卢朝阳, 焦卫东, 崔玲玲. 2009.遮挡检测/立体匹配中的分段动态规划法.电子学报, 37(7):1516-1521)[DOI:10.3321/j.issn:0372-2112.2009.07.022]
Kerschner M. 2001. Seamline detection in colour orthoimage mosaicking by use of twin snakes. ISPRS Journal of Photogrammetry and Remote Sensing, 56(1):53-64[DOI:10.1016/S0924-2716(01)00033-8]
Kwatra V, Schödl A, Essa I, Turk G and Bobick A. 2003. Graphcut textures:image and video synthesis using graph cuts. ACM Transactions on Graphics (TOG), 22(3):277-286[DOI:10.1145/882262.882264]
Li J, Sheng Y H, Zhang K, Duan P and Wu H. 2013. Panoramic stitching method based on uncalibrated ordinary camera. Journal of System Simulation, 25(9):2070-2074
李佳, 盛业华, 张卡, 段平, 吴辉. 2013.基于未标定普通相机的全景图像拼接方法.系统仿真学报, 25(9):2070-2074)[DOI:10.16182/j.cnki.joss.2013.09.032]
Lin C C, Pankanti S U, Ramamurthy K N and Aravkin A Y. 2015. Adaptive as-natural-as-possible image stitching//Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition. Boston: IEEE: 1155-1163[ DOI:10.1109/CVPR.2015.7298719 http://dx.doi.org/10.1109/CVPR.2015.7298719 ]
Lin W Y, Liu S Y, Matsushita Y, Ng T T and Cheong L F. 2011. Smoothly varying affine stitching//Proceedings of CVPR 2011. Provi dence: IEEE: 345-352[ DOI:10.1109/CVPR.2011.5995314 http://dx.doi.org/10.1109/CVPR.2011.5995314 ]
Lowe D G. 2004. Distinctive image features from scale-invariant Keypoints. International Journal of Computer Vision, 60(2):91-110[DOI:10.1023/B:VISI.0000029664.99615.94]
Marzotto R, Fusiello A and Murino V. 2004. High resolution video mosaicing with global alignment//Proceedings of 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Washington: IEEE: #8161456[ DOI:10.1109/CVPR.2004.1315099 http://dx.doi.org/10.1109/CVPR.2004.1315099 ]
Qin X J, Wang Q, Wang H L, Zheng H B and Chen S N. 2015. Image fusion method based on best seam-line for serial remote sensing images mosaic. Computer Science, 42(10):306-310
秦绪佳, 王琪, 王慧玲, 郑红波, 陈胜男. 2015.基于最佳缝合线的序列遥感图像拼接融合方法.计算机科学, 42(10):306-310)[DOI:10.11896/j.issn.1002-137X.2015.10.062]
Rui Y K and Wang J C. 2007. A new study of compound algorithm based on sweepline and divide-and-conquer algorithms for constructing delaunay triangulation. Acta Geodaetica et Cartographica Sinica, 36(3):358-362
芮一康, 王结臣. 2007. Delaunay三角形构网的分治扫描线算法.测绘学报, 36(3):358-362)[DOI:10.3321/j.issn:1001-1595.2007.03.020]
Su M S, Hwang W L and Cheng K Y. 2004. Analysis on multiresolution mosaic images. IEEE Transactions on Image Processing, 13(7):952-959[DOI:10.1109/TIP.2004.828416]
Szeliski R. 1996. Video mosaics for virtual environments. IEEE Computer Graphics and Applications, 16(2):22-30[DOI:10.1109/38.486677]
Xiao F, Wu H Z, Xiao L and Tang Y. 2007. An ambient light independent image mosaic algorithm. Journal of Image and Graphics, 12(9):1671-1675
肖甫, 吴慧中, 肖亮, 汤杨. 2007.一种光照鲁棒的图像拼接融合算法.中国图象图形学报, 12(9):1671-1675)[DOI:10.11834/jig.20070929]
Zaragoza J, Chin T J, Brown M S and Suter D. 2013. As-projective-as-possible image stitching with moving DLT//Proceedings of 2013 IEEE Conference on Computer Vision and Pattern Recognition. Portland: IEEE: 2339-2346[ DOI:10.1109/CVPR.2013.303 http://dx.doi.org/10.1109/CVPR.2013.303 ]
Zaragoza J, Chin T J, Tran Q H, Brown M S and Suter D. 2014. As-projective-as-possible image stitching with moving DLT. IEEE Transactions on Pattern Analysis and Machine Intelligence, 36(7):1285-1298[DOI:10.1109/TPAMI.2013.247]
Zhang Z X and Zhang J Q. 2012. Digital Photogrammetry. 2nd ed. Wuhan:Wuhan University Press
张祖勋, 张剑清.数字摄影测量学(第二版).武汉:武汉大学出版社, 2012
相关作者
相关机构
京公网安备11010802024621
