Current Issue Cover
结合最佳缝合线和多分辨率融合的图像拼接

谷雨1, 周阳2, 任刚1, 冯秋晨1, 鲁国智1(1.杭州电子科技大学通信息传输与融合技术国防重点学科实验室, 杭州 310018;2.北京宇航系统工程研究所, 北京 100076)

摘 要
目的 针对图像拼接过程中,缝合线通过运动物体或配准不准确区域等情况导致融合图像出现鬼影、重影的问题,提出了一种基于差异图像加权的改进最佳缝合线算法,采用基于多分辨率和加权平均的分区图像融合算法解决了拼接线问题。方法 首先将两幅图像的重叠区域划分为缝合线区域和过渡区域;在缝合线区域内,使用差异图像加权的最佳缝合线搜索准则构建准则值图像,基于动态规划思想来搜索得到最佳缝合线;基于缝合线生成掩码图像,并对重叠区域图像进行扩展,采用多分辨率融合算法实现了非严格重叠区域的融合;在过渡区域采用加权平均算法来消除拼接线。结果 采用含有大量运动物体的图像序列对算法进行测试,实验结果表明,基于差分图像加权的最佳缝合线有效避开了大部分运动物体,当缝合线难以绕开运动物体时,能够尽量少地穿过运动物体;通过多分辨率和加权平均融合算法消除了拼缝等问题。结论 提出的最佳缝合线算法能够有效地避免缝合线通过运动物体、配准不准确的区域,将多分辨率图像融合算法应用于非严格重叠图像融合,能够合成高质量的全景图像。
关键词
Image stitching by combining optimal seam and multi-resolution fusion

Gu Yu1, Zhou Yang2, Ren Gang1, Feng Qiuchen1, Lu Guozhi1(1.Fundamental Science on Communication Information Transmission and Fusion Technology Laboratory, Hangzhou Dianzi University, Hangzhou 310018, China;2.Bejing Institue of astronautical system engineering, Bejing 100076, China)

Abstract
Objective Image stitching can synthesize a panoramic image from multiple successive images and can be applied in many military and civil applications. The ghost problem exists in the overlap areas between two images being stitched when moving objects exist or a registration error occurs during image stitching. The issues of stitching line, color inconsistency, and so on occur when the camera exposure time and illumination change during imaging. These factors may affect the panoramic image if the images are simply fused. The improved image fusion technique, which is one of the key technologies in image stitching, can be used to solve these problems. Considering that the optimal seam algorithm is an effective method, an improved optimal seam algorithm based on differential image weighting is proposed to solve the ghost problem (seams pass through moving objects or inaccurate registration areas) in the classical optimal seam algorithm. A partition fusion algorithm based on multi-resolution fusion and weighted average fusion is presented to solve the stitching line problem caused by the change in exposure time and illumination.Method First, the images being stitched are mapped to a cylindrical surface after registration. The Harris corner is used to find correspondences between images. Second, the overlap areas between the image being stitched and the fused image are calculated and partitioned into three areas, namely, an optimal seam search area and two transition regions. The optimal seam search area is set to occupy three-fifth of the space, and the two transition regions occupy one-fifth each. Third, a differential image-weighted optimal seam algorithm is proposed to search for the optimal seam in the seam search regions. Aside from considering the difference in color and structure, the metric of computing the optimal seam for each pixel is also weighted by image difference. The weighting coefficient is set to infinity if this difference is above a certain threshold. Therefore, the moving-object region can be bypassed when searching seam line if the image difference is large. After the metric image is computed, a dynamic programming algorithm is used to search for the optimal seam in this metric image. Finally, a mask image is generated based on the obtained optimal seam. To address the many invalid areas caused by the mapping to a cylindrical surface, the method of image extension is adopted to fill these areas by using nearby pixels before combining two images. The multi-resolution image fusion algorithm is applied to the entire overlapping region after image extension. Then, the weighted average fusion algorithm is adopted to eliminate the stitching line in the transition regions.Result Several image sequences at the crossing captured by a mobile phone are used to test the proposed algorithm. Afterward, the proposed algorithm is compared with the algorithm of dynamic image mosaic via SIFT and dynamic programming and the stitching algorithm implemented in OpenCV. Many moving objects (e.g.cars and pedestrians) exist in the images. Experimental results demonstrate that the probability of the optimal seam passing through moving objects and inaccurate registration areas is reduced due to the improved optimal seam search algorithm. The probability of passing through moving objects when the optimal seam cannot circumvent them is as low as possible. The stitching line is effectively eliminated by combining the algorithms of multi-resolution fusion and weighted average fusion, and the quality of the panoramic image using our algorithm is better than that using the multi-resolution fusion algorithm only, especially in the scenario where varying illumination exists. The stitching result is the same or better than that using OpenCV, although some distortion exists in the panoramic image due to misregistration. A comparison of the computational costs of the proposed algorithm and the dynamic image mosaic algorithm is also presented.Conclusion The proposed optimal seam algorithm can avoid the problem in which the seam line passes through the moving-object region or a registration error occurs. The multi-resolution image fusion algorithm is applied to non-strict overlapping image fusion through image extension after region partitioning, and a high-quality panoramic image is synthesized. However, the proposed algorithm has the following shortcomings. The code must be optimized to meet real-time requirements, and global registration optimization should be used to reduce the distortion problem in the panoramic image during the process of image stitching.
Keywords

订阅号|日报