Objective

2

Image stitching is a technology for solving the field-of-view (FOV) limitation of images by stitching multiple overlapping images to generate a wide-FOV image. Parallax image stitching remains a challenging problem. Although numerous methods and abundant commercial tools are beneficial in helping users organize and appreciate photo collections

many of these tools fail to provide convincing results when parallax images are given. Parallax image stitching methods with local homography transforms for partitioning cells are the most popular. However

many of these methods have a high rate of wrongly matching feature points and low accuracy in aligning feature points in different viewpoint images. We propose a novel stitching method using the hierarchical agglomerative clustering of feature points with their plane similarity information to improve the precision rate of matching feature points.

Method

2

First

we develop a feature point shift algorithm using the clustering results of the feature points with planar information. The scale-invariant transform feature points of all the images with different viewpoints are extracted. The

k

-nearest neighbors for each feature point are found using the k-d (lk-dimension) tree algorithm.

$$ K$$

minimum sample sets are constructed

and each set includes four noncollinear feature points to compute the homography and residual matrices. Second

the planar information similarities of all the feature points are computed in accordance with the residual matrix

and all the feature points are divided into different clusters using the hierarchical agglomerative clustering algorithm. The feature points in each cluster have a common plane and the same homography transformation. If the mean of the residual matrix in one cluster is larger than a threshold

then the feature points in this cluster are labeled as wrongly matching feature points. Third

we propose an image stitching algorithm that partitions an image into cells with blend weights for multiplane images. All images are partitioned into equal-sized cells. The local homography transformation of each cell is computed via linear transformation with blend weights. The weight of each feature point is computed using the plane information of feature points. If one feature point and its cluster cent point have the same planar label

then the weight is equal to 1; otherwise

the weight is equal to their Gaussian kernel radial distance. Lastly

the aligned images are rendered as a panorama using a multiband blending method.

Result

2

We compare our feature point detection algorithm with the random sample consensus (RANSAC) algorithm on traditional building and pavilion images. The RANSAC algorithm found 427 and 541 matched feature points; our algorithm found 435 and 589 matched feature points. For the traditional building images

the RANSAC algorithm has six pairs of wrongly matched points; our algorithm has only one pair of wrongly matched points. For the pavilion images

the RANSAC algorithm has up to 20 pairs of wrongly matched points; our algorithm has only one pair of wrongly matched points. On 20 other different scene images

the average number of error feature points detected by the RANSAC algorithm is 30

and that of our method is only 3. The average number of wrongly matched point pairs of the RANSAC algorithm is eight

and that of our method is only two. We compare our image stitching method with three state-of-the-art methods (automatic stitching(AutoStitch)

as projective as possible(APAP)

and adaptive as-natural-as-possible(AANAP)) on traditional and modern building images. AutoStitch present an obvious seam line and ghosting in the results because of the global homograph. APAP and AANAP have better results with some ghosting. On the 20 different scene images

the peak signal-to-noise ratio and structural similarity index measure of our method are increased by 8.7% and 9.6%

respectively

making it the second best approach.

Conclusion

2

This study presents a novel method for high-precision parallax image stitching using feature point clustering. The experiment results show that our method

which constructs local homography transformation to shift feature points with the planar information clustering results

can increase the number of matched feature points

reduce the number of wrongly shifting and wrongly matching feature points

and increase the precision of feature point alignment compared with the state-of-the-art image stitching approaches of AutoStitch

APAP

and AANAP. The results also show that the proposed image stitching algorithm

which partitions images into cells and with blend weights for multiplane images

can achieve better image stitching performance with regard to pixel and image structure indexes than the state-of-the-art image stitching approaches of AutoStitch

APAP

and AANAP.