浏览全部资源
扫码关注微信
Received：11 September 2017，
Revised：2017-10-31，
Published：16 April 2018
移动端阅览
目的
2
显示设备的多样化使得图像重定向的作用日益凸显。不同的重定向方法产生不同视觉感受的重定向图像，而如何评价重定向图像的质量，优化重定向算法是当前研究的热点与难点，为此，提出一种结合双向相似性变换的重定向图像质量评价方法。
方法
2
首先对原始图像和重定向图像进行像素点双向匹配，利用网格顶点坐标对计算前向变换矩阵和后向变换矩阵。然后由相似性变换矩阵与标准变换矩阵间的距离得到重定向图像的几何失真。由网格面积缺失得到重定向图像的信息损失。最后结合网格的显著性，融合前向匹配与后向匹配的几何失真和信息损失得到重定向图像的质量。
结果
2
该方法在RetargetMe和CUHK数据库上的
$${\rm KRCC}$$
（Kendall rank correlation coefficient）和SROCC（Spearman rank-order correlation coefficient）性能分别达到了0.46和0.71，较现有方法有较大提升。在前向匹配与后向匹配测试中，双向匹配的测试结果优于单向匹配。
结论
2
本文方法将图像的重定向处理看做相似性变换过程。实验结果表明，从相似性变换矩阵中提取的相关特征能够较精确度量重定向图像的几何失真，而由此引发的网格面积缺失也能准确反映出重定向图像的信息损失。另外，采用双向匹配机制一定程度上减少了像素匹配误差对实验结果的影响，有效提升了重定向图像质量预测的准确性。该方法对重定向图像的质量评价效果好，适用于重定向图像的质量预测及算法优化。
Objective
2
Image retargeting
which has become an increasingly in-demand tool with the proliferation of mobile devices
aims to adjust images into different sizes or aspect ratios for various display screens. Many retargeting methods have been proposed during the past few years
but a single method that works efficiently on any image still does not exist. Different images favor different retargeting algorithms
and a key problem is to estimate the performance of each retargeting operator. Image retargeting quality assessment (IRQA) is an effective way to improve the performance of image retargeting techniques and be utilized to select favorable retargeting approaches for real applications. Nevertheless
objective IRQA is always a challenging research problem. First
the resolution of a retargeted image is different from that of its original image; thus
the problems of IRQA become different from those of the traditional image quality assessment (IQA). For example
traditional full-reference IQA methods
such as structural similarity and feature similarity indices
measure pixel-to-pixel similarity to capture image quality scores
which cannot be directly applied to IRQA. Second
traditional IQA metrics mainly focus on estimating the perceptual similarities between a source image and its corresponding non-geometrically distorted version. With respect to the IRQA problem
the perceptual quality of a retargeted image is intensely related to human cognition of this image. The structure and semantic information of an object should be consistent with the prior knowledge of humans. We propose a new method for IRQA via bidirectional similarity transformation to accurately evaluate the quality of retargeted images.
Method
2
Geometric distortion and information loss are two important issues in image retargeting. We propose a novel metric to quantify the geometric distortion and information loss of a retargeted image. Instead of only establishing a pseudo mapping relationship between the retargeted image and its original image
we regenerate the retargeted image from the original image and regenerate the original image from the retargeted image inversely. The issue of pixel matching is then converted into a field of similarity transformation. We use the scale-invariant feature transform (SIFT)-flow algorithm to extract a dense SIFT descriptor for each pixel in the original and retargeted images to build a reliable matching relation between the images. As a result
pixel-wise correspondences are established
then forward and backward similarity transformation matrices can be calculated from their corresponding mesh vertex coordinates. Similarity transformation matrix contains important information about the image retargeting process
which controls mesh deformation
and is a decisive factor for geometric distortion and information loss. Geometric distortion is calculated from the distance between a similarity transformation matrix and the benchmark transformation matrix in this study. Rotation and scaling parameters can reflect geometric distortion; hence
the distance defined in this work is composed of two components:the absolute distance difference and aspect change
which are never applied in previous methods. A large distance between the estimated and benchmark transformation matrices usually means large information loss. However
if a salient object is discarded or cropped
then the above geometric distortion measurement cannot correctly reflect such a type of information loss. Therefore
the pixels in the original or retargeted image are mapped to its opposite image with the forward or backward transformation
and the information loss is calculated from the missing areas. The quality of the retargeted image is obtained by the geometric distortion and information loss from the forward and backward transformations.
Result
2
Experimental results on the publicly available RetargetMe and CUHK datasets demonstrate the superiority of the proposed method. In the RetargetMe dataset
the Kendall rank correlation coefficient of the method reaches 0.46
and it has a good evaluation output on each subclass
especially on foreground objects
texture
and geometric structure. In the CUHK database
the Spearman's rank-order correlation coefficient is above 0.71. The results of one-way matching
including forward and backward matching
are worse than those of bidirectional matching affected by matching error. In this method
grid size affects similarity transformation and hence also influences geometric distortion and information loss measurements. We test different grid sizes
and the results indicate that the choice of grid size has a certain effect on quality prediction. The grid size of 16×16 has a relatively high performance on the CUHK and RetargetMe databases.
Conclusion
2
We present a novel IRQA method based on bidirectional similarity transformation. Unlike in traditional IRQA metrics that only estimate the matching similarity between original and retargeted images
we regenerate the retargeted image from the original image and the original image from the retargeted image to extract effective features from the similarity transformation matrix and the loss of mesh areas. The major contribution of this study is that it considers the retargeting operator as a process of image similarity transformation. The similarity transformation matrix connects original and retargeted images
and it exerts a great influence on the retargeted image quality. Features extracted from the similarity transformation matrix can measure geometric distortion accurately. Geometric change can lead to information loss if partial information is preserved or discarded. Thus
features extracted from the reduced mesh
$${\rm area}$$
can reflect the information loss of each retargeted image inerrably. The bidirectional matching mechanism that we employ can effectively reduce the influence of pixel matching error. Our quality assessment method therefore has a better correlation with subjective scores
outperforms existing methods
and is suitable for image retargeting quality prediction and optimizing retargeting algorithms.
Zhang GX, Cheng M M, Hu S M, et al. A shape-preserving approach to image resizing[J]. Computer Graphics Forum, 2010, 28(7):1897-1906.[DOI:10.1111/j.1467-8659.2009.01568.x]
Zhang Y B, Fang Y M, Lin W S, et al. Backward registration-based aspect ratio similarity for image retargeting quality assessment[J]. IEEE Transactions on Image Processing, 2016, 25(9):4286-4297.[DOI:10.1109/TIP.2016.2585884]
Shao F, Lin W C, Lin W S, et al. QoE-guided warping for stereoscopic image retargeting[J]. IEEE Transactions on Image Processing, 2017, 26(10):4790-4805.[DOI:10.1109/TIP.2017.2721546]
Rubinstein M, Shamir A, Avidan S. Improved seam carving for video retargeting[J]. ACM Transactions on Graphics (TOG), 2008, 27(3):#16.[DOI:10.1145/1360612.1360615]
Wolf L, Guttmann M, Cohen-Or D. Non-homogeneous content-driven video-retargeting[C]//Proceedings of the 2007 IEEE 11th International Conference on Computer Vision. Rio de Janeiro, Brazil: IEEE, 2007: 1-6. [ DOI:10.1109/ICCV.2007.4409010 http://dx.doi.org/10.1109/ICCV.2007.4409010 ]
Krähenbühl P, Lang M, Hornung A, et al. A system for retargeting of streaming video[J]. ACM Transactions on Graphics, 2009, 28(5):#126.[DOI:10.1145/1618452.1618472]
Jiang Q P, Shao F, Lin W S, et al. Learning sparse representation for objective image retargeting quality assessment[J]. IEEE Transactions on Cybernetics, 2017, PP(qq):1-14.[DOI:10.1109/TCYB.2017.2690452]
Manjunath B S, Ohm J R, Vasudevan V V, et al. Color and texture descriptors[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2001, 11(6):703-715.[DOI:10.1109/76.927424]
Kasutani E, Yamada A. The MPEG-7 color layout descriptor: a compact image feature description for high-speed image/video segment retrieval[C]//Proceedings of 2001 International Conference on Image Processing. Thessaloniki, Greece: IEEE, 2001, 1: 674-677. [ DOI:10.1109/ICIP.2001.959135 http://dx.doi.org/10.1109/ICIP.2001.959135 ]
Simakov D, Caspi Y, Shechtman E, et al. Summarizing visual data using bidirectional similarity[C]//Proceedings of 2008 IEEE Conference on Computer Vision and Pattern Recognition. Anchorage, AK, USA: IEEE, 2008: 1-8. [ DOI:10.1109/CVPR.2008.4587842 http://dx.doi.org/10.1109/CVPR.2008.4587842 ]
Fang Y M, Zeng K, Wang Z, et al. Objective quality assessment for image retargeting based on structural similarity[J]. IEEE Journal on Emerging and Selected Topics in Circuits and Systems, 2014, 4(1):95-105.[DOI:10.1109/JETCAS.2014.2298919]
Hsu C C, Lin C W, Fang Y M, et al. Objective quality assessment for image retargeting based on perceptual geometric distortion and information loss[J]. IEEE Journal of Selected Topics in Signal Processing, 2014, 8(3):377-389.[DOI:10.1109/JSTSP.2014.2311884]
Liang Y, Liu Y J, Gutierrez D. Objective quality prediction of image retargeting algorithms[J]. IEEE Transactions on Visualization and Computer Graphics, 2017, 23(2):1099-1110.[DOI:10.1109/TVCG.2016.2517641]
Karimi M, Samavi S, Karimi N, et al. Quality assessment of retargeted images by salient region deformity analysis[J]. Journal of Visual Communication and Image Representation, 2017, 43:108-118.[DIO:10.1016/j.jvcir.2016.12.011]
Liu C, Yuen J, Torralba A. SIFT flow:dense correspondence across scenes and its applications[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2011, 33(5):978-994.[DOI:10.1109/TPAMI.2010.147]
Yan Q, Xu L, Shi J P, et al. Hierarchical saliency detection[C]//Proceedings of 2013 IEEE Conference on Computer Vision and Pattern Recognition. Portland, OR, USA: IEEE, 2013: 1155-1162. [ DOI:10.1109/CVPR.2013.153 http://dx.doi.org/10.1109/CVPR.2013.153 ]
Torr P H S, Zisserman A. MLESAC:a new robust estimator with application to estimating image geometry[J]. Computer Vision and Image Understanding, 2000, 78(1):138-156.[DOI:10.1006/cviu.1999.0832]
Pritch Y, Kav-venaki E, Peleg S. Shift-map image editing[C]//Proceedings of the 2009 IEEE 12th International Conference on Computer Vision. Kyoto, Japan: IEEE, 2009: 151-158. [ DOI:10.1109/ICCV.2009.5459159 http://dx.doi.org/10.1109/ICCV.2009.5459159 ]
Wang Y S, Tai C L, Sorkine O, et al. Optimized scale-and-stretch for image resizing[J]. ACM Transactions on Graphics, 2008, 27(5):#118.[DOI:10.1145/1409060.1409071]
Rubinstein M, Shamir A, Avidan S. Multi-operator media retargeting[J]. ACM Transactions on Graphics, 2009, 28(3):#23.[DOI:10.1145/1531326.1531329]
Dong W M, Zhou N, Paul J C, et al. Optimized image resizing using seam carving and scaling[J]. ACM Transactions on Graphics, 2009, 28(5):#125.[DOI:10.1145/1618452.1618471]
Karni Z, Freedman D, Gotsman C. Energy-based image deformation[J]. Computer Graphics Forum, 2009, 28(5):1257-1268.[DOI:10.1111/j.1467-8659.2009.01503.x]
相关文章
相关作者
相关机构
京公网安备11010802024621
