浏览全部资源
扫码关注微信
纸质出版日期： 2018-07-16 ，
移动端阅览
何川, 周军. 具有直线结构保护的网格化图像拼接[J]. 中国图象图形学报, 2018,23(7):973-983.
Chuan He, Jun Zhou. Mesh-based image stitching algorithm with linear structure protection[J]. Journal of Image and Graphics, 2018,23(7):973-983.
目的
2
基于网格变形的图像配准方式
针对待拼接图片重叠区域的视差具有一定的容忍性
并且能够适应更复杂的图像拼接场景。在NISwGSP (natural image stitching with the global similarity prior)算法基础上提出了一种具有直线结构保护的图像拼接算法(MISwLP)
该算法通过提取图片中的直线结构并施加约束
可以得到视觉效果自然、畸变较小的图像拼接结果。
方法
2
首先对图片进行网格划分
建立网格优化模型
针对网格顶点坐标集定义能量函数
在保证图片重叠区域高度对齐的同时
对网格进行相似性连续约束
并辅以直线结构约束
最后使用共轭梯度最小二乘法求解得到最优网格顶点集
指导网格变形。
结果
2
针对不同场景下的图片进行拼接实验
同时和几种比较流行的图像拼接软件和算法进行比较。结果表明
同经典拼接算法
比如Autostitch相比
基于网格优化的图像拼接算法能够适应更加复杂的多平面场景
在减小投影失真和对齐误差方面表现更好; 同现在比较好的几种网格拼接算法
比如SPHP (shape-preserving half-projective warps for image stitching)、APAP (as-projective-as-possible image stitching with moving DLT)、NISwGSP等的比较
MISwLP算法不仅能够很好地对齐图像和避免投影失真
并且能够保持图像重叠区域到非重叠区域的一致性
即保护原图中的直线结构。
结论
2
提出了一种基于网格优化的直线约束方法
对于具有显著几何结构的图像拼接场景
能够较好地保护拼接后图像中原有的直线结构
具有较好的应用价值。
Objective
2
The image registration method based on mesh deformation can handle some parallax in the overlapping area of input images and can adapt to more complex scenarios where the scenery is not in the same plane.A new mesh-based image stitching with linear structure protection (MISwLP) is proposed.This algorithm applies constraints to the lines extracted from images to protect them from being distorted by the mesh deformation process
thus obtaining natural panoramas with reduced distortion.
Method
2
MISwLP is based on mesh deformation.Images are meshed with a set of vertices
and image deformation is guided by the indexed vertices.The algorithm can be implemented with four steps.The first step is called APAP (As-Projective-As-Possible Image Stitching with Moving DLT) pre-registration.The APAP algorithm is applied to align the images
and the feature matching pairs obtained by the APAP algorithm can be used to obtain all the vertex matching pairs in the overlapped area of the image matching pairs
which are called matching points.These matching points are distributed evenly and can be used to obtain good alignment capability for the mesh optimization model.The second step is called global similarity estimation.The relative 2D rotation angle and the relative scale between two images are estimated in this step.Then
a similarity transform between the two images can be constructed.In the third step
a mesh optimization model is established for the input vertices of the images.The mesh optimization process is implemented in two stages.In the first stage
the energy function includes three terms
namely
the alignment
local similarity
and global similarity terms
and the original vertices are used as the input for this function.It is solved by the least-squares conjugate gradient method.The first stage aims to align the images.Then
the outputs of the first stage are used as the input vertices of the second stage.In the second stage
a new term called line protection is added for further optimization.The lines are extracted by the LSD algorithm with a threshold or user-guided interface and then sampled across the grid.The line protection term constrains the sample points in a straight line.The optimization solution is computed with a sparse matrix effectively.At this time
the distorted lines in the first stage of this step are straightened.In the fourth step
a texture mapping method is applied by affine transforming the input grids into the output grids.All images are blended with a linear blending method.
Result
2
The performance of MISwLP is verified using images captured from different sceneries by handheld devices
such as mobile phones and digital cameras
and several open datasets.The scenes include urban and nature sceneries.MISwLP can handle more complicated image stitching tasks
in which the scenery consists of two planes
than image-stitching algorithms
which use only one global holography
such as AutoStitch.Furthermore
MISwLP produces a natural stitching result and reduced projective distortion.In addition
MISwLP outperforms several state-of-the-art methods
such as SPHP (Shape-Preserving Half-Projective Warps for Image Stitching)
APAP
and NISwGSP (Natural Image Stitching with the Global Similarity Prior).These algorithms use similarity transform to protect the non-overlapping area from projective distortion.Consequently
inconsistency is introduced between the overlapped and non-overlapped areas.The human eyes can perceive the destruction of certain geometry structures of the transitional area.MISwLP handles this problem with a line protection term and provides a good result with only a few geometry distortions.The proposed method works especially well for urban sceneries that contain many linear structures.For sceneries with no evident geometry
a user-guided auxiliary method is provided for selecting lines to protect.MISwLP is based on the NISwGSP algorithm
but the experiments show that their time complexities are nearly the same.
Conclusion
2
The performance of the proposed method is superior to those of state-of-the-art image-stitching methods.MISwLP protects the linear structure in the image stitching process
thereby providing a good stitching result with no geometry and projective distortions.Therefore MISwLP has good application value.
图像拼接网格变形直线保护能量函数最优化共轭梯度最小二乘法投影失真
image stitchingmesh deformationlines protectionenergy functionoptimizationleast-squared conjugate gradient methodprojective distortion
Brown M, Lowe D G.Automatic panoramic image stitching using invariant features[J].International Journal of Computer Vision, 2007, 74(1):59-73.[DOI:10.1007/s11263-006-0002-3]
Joshi H, Sinha K L.A survey on image mosaicing techniques[J].International Journal of Advanced Research in Computer Engineering&Technology, 2013, 2(2):1-11.
Igarashi T, Igarashi Y.Implementing as-rigid-as-possible shape manipulation and surface flattening[J].Journal of Graphics, Gpu, and Game Tools, 2009, 14(1):17-30.[DOI:10.1080/2151237x.2009.10129273]
Gao J H, Kim S J, Brown M S. Constructing image panoramas using dual-homography warping[C]//Proceedings of 2011 IEEE Conference on Computer Vision and Pattern Recognition. Springs, Colorado, USA: IEEE, 2011: 49-56. [DOI:10.1109/cvpr.2011.5995433http://dx.doi.org/10.1109/cvpr.2011.5995433]
Lin W Y, Liu S Y, Matsushita Y, et al. Smoothly varying affine stitching[C]//Proceedings of 2011 IEEE Conference on Computer Vision and Pattern Recognition. Colorado Springs, USA: IEEE, 2011: 345-352. [DOI:10.1109/cvpr.2011.5995314http://dx.doi.org/10.1109/cvpr.2011.5995314]
Zaragoza J, Chin T J, Brown M S, et al. As-projective-as-possible image stitching with moving DLT[C]//Proceedings of 2013 IEEE Conference on Computer Vision and Pattern Recognition. Portland, OR, USA: IEEE, 2013: 2339-2346. [DOI:10.1109/CVPR.2013.303http://dx.doi.org/10.1109/CVPR.2013.303]
Li S W, Yuan L, Sun J, et al. Dual-feature warping-based motion model estimation[C]//Proceedings of 2015 IEEE International Conference on Computer Vision. Santiago, Chile: IEEE, 2015: 4283-4291. [DOI:10.1109/iccv.2015.487http://dx.doi.org/10.1109/iccv.2015.487]
Joo K, Kim N, Oh T H, et al. Line meets as-projective-as-possible image stitching with moving DLT[C]//Proceedings of 2015 IEEE International Conference on Image Processing. Québec City, Canada: IEEE, 2015: 1175-1179. [DOI:10.1109/icip.2015.7350985http://dx.doi.org/10.1109/icip.2015.7350985]
Liu F, Gleicher M, Jin H L, et al. Content-preserving warps for 3D video stabilization[C]//Proceedings of 2009 ACM SIGGRAPH. New Orleans, Louisiana, USA: ACM, 2009: #44. [DOI:10.1145/1576246.1531350http://dx.doi.org/10.1145/1576246.1531350]
Gao J H, Li Y, Chin T J, et al. Seam-driven image stitching[C]//Proceedings of 2013 EUROGRAPHICS. Girona, Spain: EUROGRAPHICS, 2013. [DOI:10.2312/conf/eg2013/short/045-048http://dx.doi.org/10.2312/conf/eg2013/short/045-048]
Zhang F, Liu F. Parallax-tolerant image stitching[C]//Proceedings of 2014 IEEE Conference on Computer Vision and Pattern Recognition. Columbus, OH, USA: IEEE, 2014: 3262-3269. [DOI:10.1109/cvpr.2014.423http://dx.doi.org/10.1109/cvpr.2014.423]
Lin K M, Jiang N J, Cheong L F, et al. SEAGULL: seam-guided local alignment for parallax-tolerant image stitching[C]//Proceedings of the 14th European Conference on Computer Vision-ECCV 2016. Amsterdam, The Netherlands: Springer, 2016: 370-385. [DOI:10.1007/978-3-319-46487-9_23http://dx.doi.org/10.1007/978-3-319-46487-9_23]
Achanta R, Shaji A, Smith K, et al.SLIC superpixels compared to state-of-the-art superpixel methods[J].IEEE Transactions on Pattern Analysis and Machine Intelligence, 2012, 34(11):2274-2282.[DOI:10.1109/tpami.2012.120]
Chang C H, Sato Y, Chuang Y Y. Shape-preserving half-projective warps for image stitching[C]//Proceedings of 2014 IEEE Conference on Computer Vision and Pattern Recognition. Columbus, Ohio, USA: IEEE, 2014: 3254-3261. [DOI:10.1109/cvpr.2014.422http://dx.doi.org/10.1109/cvpr.2014.422]
Lin C C, Pankanti S U, Ramamurthy K N, et al. Adaptive as-natural-as-possible image stitching[C]//Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition. Boston, Massachusetts, USA: IEEE, 2015: 1155-1163. [DOI:10.1109/CVPR.2015.7298719http://dx.doi.org/10.1109/CVPR.2015.7298719]
Chen Y S, Chuang Y Y. Natural image stitching with the global similarity prior[C]//Proceedings of the 14th European Conference on European Conference on Computer Vision. Amsterdam, Netherlands: Springer, 2016: 186-201. [DOI:10.1007/978-3-319-46454-1_12http://dx.doi.org/10.1007/978-3-319-46454-1_12]
Li N, Xu Y F, Wang C.Quasi-homography warps in image stitching[J].IEEE Transactions on Multimedia, 2017.[DOI:10.1109/tmm.2017.2771566](in Press)
Xiang T Z, Xia G S, Bai X, et al. Image stitching by line-guided local warping with global similarity constraint[J]. arXiv: 1702. 07935, 2017.
Yao D, Zhou J, Xue Z. Homography matrix genetic consensus estimation algorithm[C]//Proceedings of 2010 International Conference on Audio Language and Image Processing. Shanghai, China: IEEE, 2010: 1139-1143. [DOI:10.1109/icalip.2010.5685084http://dx.doi.org/10.1109/icalip.2010.5685084]
相关作者
相关机构
京公网安备11010802024621