Image stitching can synthesize a panoramic image from multiple successive images and can be applied in many military and civil applications. The ghost problem exists in the overlap areas between two images being stitched when moving objects exist or a registration error occurs during image stitching. The issues of stitching line

color inconsistency

and so on occur when the camera exposure time and illumination change during imaging. These factors may affect the panoramic image if the images are simply fused. The improved image fusion technique

which is one of the key technologies in image stitching

can be used to solve these problems. Considering that the optimal seam algorithm is an effective method

an improved optimal seam algorithm based on differential image weighting is proposed to solve the ghost problem (seams pass through moving objects or inaccurate registration areas) in the classical optimal seam algorithm. A partition fusion algorithm based on multi-resolution fusion and weighted average fusion is presented to solve the stitching line problem caused by the change in exposure time and illumination. First

the images being stitched are mapped to a cylindrical surface after registration. The Harris corner is used to find correspondences between images. Second

the overlap areas between the image being stitched and the fused image are calculated and partitioned into three areas

namely

an optimal seam search area and two transition regions. The optimal seam search area is set to occupy three-fifth of the space

and the two transition regions occupy one-fifth each. Third

a differential image-weighted optimal seam algorithm is proposed to search for the optimal seam in the seam search regions. Aside from considering the difference in color and structure

the metric of computing the optimal seam for each pixel is also weighted by image difference. The weighting coefficient is set to infinity if this difference is above a certain threshold. Therefore

the moving-object region can be bypassed when searching seam line if the image difference is large. After the metric image is computed

a dynamic programming algorithm is used to search for the optimal seam in this metric image. Finally

a mask image is generated based on the obtained optimal seam. To address the many invalid areas caused by the mapping to a cylindrical surface

the method of image extension is adopted to fill these areas by using nearby pixels before combining two images. The multi-resolution image fusion algorithm is applied to the entire overlapping region after image extension. Then

the weighted average fusion algorithm is adopted to eliminate the stitching line in the transition regions. Several image sequences at the crossing captured by a mobile phone are used to test the proposed algorithm. Afterward

the proposed algorithm is compared with the algorithm of dynamic image mosaic via SIFT and dynamic programming and the stitching algorithm implemented in OpenCV. Many moving objects (e.g.cars and pedestrians) exist in the images. Experimental results demonstrate that the probability of the optimal seam passing through moving objects and inaccurate registration areas is reduced due to the improved optimal seam search algorithm. The probability of passing through moving objects when the optimal seam cannot circumvent them is as low as possible. The stitching line is effectively eliminated by combining the algorithms of multi-resolution fusion and weighted average fusion

and the quality of the panoramic image using our algorithm is better than that using the multi-resolution fusion algorithm only

especially in the scenario where varying illumination exists. The stitching result is the same or better than that using OpenCV

although some distortion exists in the panoramic image due to misregistration. A comparison of the computational costs of the proposed algorithm and the dynamic image mosaic algorithm is also presented. The proposed optimal seam algorithm can avoid the problem in which the seam line passes through the moving-object region or a registration error occurs. The multi-resolution image fusion algorithm is applied to non-strict overlapping image fusion through image extension after region partitioning

and a high-quality panoramic image is synthesized. However

the proposed algorithm has the following shortcomings. The code must be optimized to meet real-time requirements

and global registration optimization should be used to reduce the distortion problem in the panoramic image during the process of image stitching.