Image scaling based on the spring deformation model

Xiangli Gu; Jing Chi; Caiming Zhang

doi:10.11834/jig.170374

Column of GDC 2017 | Views : 0 下载量: 60 CSCD: 2

PDF
Export
Share
Collection
Album

Image scaling based on the spring deformation model
Vol. 23, Issue 5, Pages: 756-765(2018)
Received：08 July 2017，

Revised：2018-1-19，

Published：16 May 2018
DOI： 10.11834/jig.170374
稿件说明：

移动端阅览

DOI：

Xiangli Gu, Jing Chi, Caiming Zhang. Image scaling based on the spring deformation model[J]. Journal of Image and Graphics, 2018, 23(5): 756-765. DOI： 10.11834/jig.170374.

摘要

目的

随着显示设备分辨率和纵横比的多样化，内容感知图像缩放技术逐渐成为图像处理领域新的研究热点之一，为了既能实现图像有效缩放，又能较好保持图像主体区域的几何结构完整，提出一种基于弹簧近似的内容感知图像缩放方法。

方法

首先，为控制图像缩放，对输入图像进行显著性检测和特征直线检测；其次，在图像上构建一个平面三角网格，视三角网格的每一条边为一根弹簧，则整个三角网格构成一个弹簧系统，可利用该弹簧系统的变形实现图像缩放，其中，根据每根弹簧所在图像区域的显著度设置弹簧的弹性系数，该系数可在缩放时有效保持图像的主体区域，避免主体内容缩放不均匀或变形；然后，以图像直线特征保持为约束，构造用于约束弹簧系统变形的目标函数；最后，利用纹理贴图技术把弹簧系统每个三角形对应的图像贴回弹簧系统，得到缩放后的目标图像。

结果

为证明本文方法的有效性，对大量图像进行多比例缩放测试，并与现有方法进行比较。在同比例缩放下，本文方法可以更好地保持图像中的主体区域，以及特征直线，使放缩后的图像整体具有更好的视觉效果。本文方法的计算时间小于0.19 s，时间成本与现有方法大致相同。

结论

基于弹簧系统的内容感知图像缩放方法，通过弹簧系统变形带动网格变形进而实现图像缩放，与现有的内容感知图像缩放方法相比，本文方法所处理的图像视觉效果更好，实验结果表明，本文方法能够更有效、快速地处理各种图像的内容感知缩放。

Abstract

Objective

With the emergence of a large number of diverse and versatile display devices

the size of images must be adjusted appropriately with different resolutions and aspect ratios before displaying them on the devices. Content-aware image scaling technology has gradually become the focus of new research in the field of image processing

which resizes an image such that the prominent feature of the image is intact and the homogenous content is distorted as little as possible. This study proposes a novel content-aware image scaling method based on spring analogy to achieve image scaling effectively and have better capability to prevent the distortion of significant objects and feature lines.

Method

We resize an image by representing an image as a triangular mesh in 2D space and viewing each triangle edge as a spring. By deforming the spring system

we can implicitly retarget the image to its new size. First

for the input image

we use the saliency detection algorithm to generate the saliency of each pixel and employ the Hough transformation to detect the feature lines. Second

Delaunay triangulation of the sampling nodes of the image is employed to build a 2D triangular mesh on the input image to build a spring system for the image. A spring system on the mesh is constructed. Each edge of the mesh is considered a segment of an elastic spring

and the length of the edge is used as the initial equilibrium length of the spring. Once the boundary locations are changed by an external force

the interior will expand or contract to reach a new balance of the spring system

leading to a deformed mesh. Then

the image zoom is implemented through optimized deformation of the spring system. Furthermore

we assign a stiffness value to each spring segment to constrain its deformation in the balance of the spring system. Obviously

a high saliency of the image area is corresponds to great stiffness of the segments in the area and small distortion of the segments. Thus

we propose a new measure for spring stiffness

which uses image saliency to compute the elastic coefficient. The coefficient can effectively avoid the main content of inhomogeneous deformation. Finally

the objective function for constraining the deformation of the spring system is constructed by keeping the image linear features as constraints. We map the input image onto the deformed mesh to obtain the final resized image by using the trilinear interpolation method.

Result

Extensive experiments and comparisons with other state-of-the-art methods (i.e.

seam carving

quadratic programming

object size adjustment

shape-preserving approach

and methods based on spring analogy) show that our algorithm can better maintain the main region of the image and the feature line

and the scaled image has a better visual effect as a whole. The calculation time of the proposed method is less than 0.19 s

which is approximately the same as that of the existing method.

Conclusion

An image scaling method based on spring analogy is proposed to achieve image zooming. Extensive experiments show that

compared with the existing content-aware image scaling approaches

our method has better visual effects and better ability to prevent the distortion of significant objects and feature lines

关键词

Keywords

references

Avidan S, Shamir A. Seam carving for content-aware image resizing[J]. ACM Transactions on Graphics, 2007, 26(3):#10.[DOI:10.1145/1276377.1276390]

Guo D Y, Ding J D, Tang J H, et al. NIF-based seam carving for image resizing[J]. Multimedia Systems, 2015, 21(6):603-613.[DOI:10.1007/s00530-014-0425-6]

Oliveira S A F, Rocha Neto A R, Bezerra F N. A Novel Genetic Algorithms and SURF-Based Approach for Image Retargeting[M]. Tarrytown, NY, USA:Pergamon Press, Inc., 2016.[DOI:10.1016/j.eswa.2015.09.015]

Nie D D, Ma L Z. Fast image resizing method under energy balance principle[J]. Journal of Computer-Aided Design&Computer Graphics, 2014, 26(8):1314-1319.

聂栋栋, 马利庄.能量均衡原则下的快速图像尺寸调整算法[J].计算机辅助设计与图形学学报, 2014, 26(8):1314-1319.

Rubinstein M, Shamir A, Avidan S. Multi-operator media retargeting[J]. ACM Transactions on Graphics, 2009, 28(3):#23.[DOI:10.1145/1531326.1531329]

Zhang L X, Li K S, Ou Z M, et al. Seam warping:a new approach for image retargeting for small displays[J]. Soft Computing, 2017, 21(2):447-457.[DOI:10.1007/s00500-015-1795-1]

Shafieyan F, Karimi N, Mirmahboub B, et al. Image retargeting using depth assisted saliency map[J]. Signal Processing:Image Communication, 2017, 50:34-43.[DOI:10.1016/j.image.2016.10.006]

Zhu L L, Chen Z B. Fast genetic multi-operator image retargeting[C]//Proceedings of 2016 Visual Communications and Image Processing. Chengdu,China: IEEE, 2016: 1-4. [ DOI:10.1109/VCIP.2016.7805467 http://dx.doi.org/10.1109/VCIP.2016.7805467 ]

Wang Y S, Tai C L, Sorkine O, et al. Optimized scale-and-stretch for image resizing[J]. ACM Transactions on Graphics, 2008, 27(5):#118.[DOI:10.1145/1409060.1409071]

Shi M L, Yang L, Peng G Q, et al. A content-aware image resizing method with prominent object size adjusted[C]//Proceedings of the 17th ACM Symposium on Virtual Reality Software and Technology. Hong Kong, China: ACM, 2010: 175-176. [ DOI:10.1145/1889863.1889901 http://dx.doi.org/10.1145/1889863.1889901 ]

Shi M L, Xu D. A prominent object size adjustable method for content-aware image resizing[J]. Journal of Computer-Aided Design&Computer Graphics, 2011, 23(5):915-922.

施美玲, 徐丹.主体大小能控的内容感知图像缩放[J].计算机辅助设计与图形学学报, 2011, 23(5):915-922.

Chen R J, Freedman D, Karni Z, et al. Content-aware image resizing by quadratic programming[C]//Proceedings of 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops. San Francisco, CA, USA: IEEE, 2010: 1-8. [ DOI:10.1109/CVPRW.2010.5543281 http://dx.doi.org/10.1109/CVPRW.2010.5543281 ]

Zhang G X, Cheng M M, Hu S M, et al. A shape-preserving approach to image resizing[J]. Computer Graphics Forum, 2009, 28(7):1897-1906.[DOI:10.1111/j.1467-8659.2009.01568.x]

Wang D, Li G Q, Jia W J, et al. Saliency-driven scaling optimization for image retargeting[J]. The Visual Computer, 2011, 27(9):853-860.[DOI:10.1007/s00371-011-0559-x]

Lin S S, Yeh I C, Lin C H, et al. Patch-based image warping for content-aware retargeting[J]. IEEE Transactions on Multimedia, 2013, 15(2):359-368.[DOI:10.1109/TMM.2012.2228475]

Guo Y W, Liu F, Shi J, et al. Image retargeting using mesh parametrization[J]. IEEE Transactions on Multimedia, 2009, 11(5):856-867.[DOI:10.1109/TMM.2009.2021781]

Jin Y, Liu L G, Wu Q B. Nonhomogeneous scaling optimization for realtime image resizing[J]. The Visual Computer, 2010, 26(6-8):769-778.[DOI:10.1007/s00371-010-0472-8]

Xu J L, Kang H M, Chen F L. Content-aware image resizing using quasi-conformal mapping[J]. The Visual Computer, 2017, doi:10.1007/s00371-017-1350-4. (in Press)

Ran L Q, Lv N, Meng X X. Image retargeting based on spring analogy[C] //Proceedings of 2014 International Conference on Progress in Informatics and Computing. Shanghai, China: IEEE, 2014: 250-254. [ DOI:10.1109/PIC.2014.6972335 http://dx.doi.org/10.1109/PIC.2014.6972335 ]

Blom F J. Considerations on the spring analogy[J]. International Journal for Numerical Methods in Fluids, 2000, 32(6):647-668.[DOI:10.1002/(SICI)1097-0363(20000330)32:6<647::AID-FLD979>3.0.CO;2-K]

Itti L, Koch C, Niebur E. A model of saliency-based visual attention for rapid scene analysis[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1998, 20(11):1254-1259.[DOI:10.1109/34.730558]

Harel J, Koch C, Perona P. Graph-based visual saliency[C]//Proceedings of the 19th International Conference on Neural Information Processing Systems. Cambridge: MIT Press, 2006: 545-552.

Goferman S, Zelnik-Manor L, Tal A. Context-aware saliency detection[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2012, 34(10):1915-1926.[DOI:10.1109/TPAMI.2011.272]

Von Gioi R G, Jakubowicz J, Morel J M, et al. LSD:a line segment detector[J]. Image Processing on Line, 2012, 2:35-55.[DOI:10.5201/ipol.2012.gjmr-lsd]

Fernandes L A F, Oliveira M M. Real-time line detection through an improved Hough transform voting scheme[J]. Pattern Recognition, 2008, 41(1):299-314.[DOI:10.1016/j.patcog.2007.04.003]

Xu Y, Liu L G, Gotsman C, et al. Capacity-constrained Delaunay triangulation for point distributions[J]. Computers&Graphics, 2011, 35(3):510-516.[DOI:10.1016/j.cag.2011.03.031]

Zhu C Y, Byrd R H, Lu P H, et al. L-BFGS-B-Fortran subroutines for large-scale bound constrained optimization[J]. ACM Transactions on Mathematical Software, 1999, 23:550-560.

Alert me when the article has been cited

提交

Feature preservation with combined filters for mesh denoising

Edge-preserving and contrast-enhancement image scaling

Image retargeting quality assessment by preserving important regions