Current Issue Cover
结合双向相似性变换的重定向图像质量评价

富振奇, 邵枫, 蒋刚毅, 郁梅(宁波大学信息科学与工程学院, 宁波 315211)

摘 要
目的 显示设备的多样化使得图像重定向的作用日益凸显。不同的重定向方法产生不同视觉感受的重定向图像,而如何评价重定向图像的质量,优化重定向算法是当前研究的热点与难点,为此,提出一种结合双向相似性变换的重定向图像质量评价方法。方法 首先对原始图像和重定向图像进行像素点双向匹配,利用网格顶点坐标对计算前向变换矩阵和后向变换矩阵。然后由相似性变换矩阵与标准变换矩阵间的距离得到重定向图像的几何失真。由网格面积缺失得到重定向图像的信息损失。最后结合网格的显著性,融合前向匹配与后向匹配的几何失真和信息损失得到重定向图像的质量。结果 该方法在RetargetMe和CUHK数据库上的KRCC(Kendall rank correlation coefficient)和SROCC(Spearman rank-order correlation coefficient)性能分别达到了0.46和0.71,较现有方法有较大提升。在前向匹配与后向匹配测试中,双向匹配的测试结果优于单向匹配。结论 本文方法将图像的重定向处理看做相似性变换过程。实验结果表明,从相似性变换矩阵中提取的相关特征能够较精确度量重定向图像的几何失真,而由此引发的网格面积缺失也能准确反映出重定向图像的信息损失。另外,采用双向匹配机制一定程度上减少了像素匹配误差对实验结果的影响,有效提升了重定向图像质量预测的准确性。该方法对重定向图像的质量评价效果好,适用于重定向图像的质量预测及算法优化。
关键词
Image retargeting quality assessment via bidirectional similarity transformation

Fu Zhenqi, Shao Feng, Jiang Gangyi, Yu Mei(Faculty of Information Science and Engineering Ningbo University, Ningbo 315211, China)

Abstract
Objective Image retargeting, which has become an increasingly in-demand tool with the proliferation of mobile devices, aims to adjust images into different sizes or aspect ratios for various display screens. Many retargeting methods have been proposed during the past few years, but a single method that works efficiently on any image still does not exist. Different images favor different retargeting algorithms, and a key problem is to estimate the performance of each retargeting operator. Image retargeting quality assessment (IRQA) is an effective way to improve the performance of image retargeting techniques and be utilized to select favorable retargeting approaches for real applications. Nevertheless, objective IRQA is always a challenging research problem. First, the resolution of a retargeted image is different from that of its original image; thus, the problems of IRQA become different from those of the traditional image quality assessment (IQA). For example, traditional full-reference IQA methods, such as structural similarity and feature similarity indices, measure pixel-to-pixel similarity to capture image quality scores, which cannot be directly applied to IRQA. Second, traditional IQA metrics mainly focus on estimating the perceptual similarities between a source image and its corresponding non-geometrically distorted version. With respect to the IRQA problem, the perceptual quality of a retargeted image is intensely related to human cognition of this image. The structure and semantic information of an object should be consistent with the prior knowledge of humans. We propose a new method for IRQA via bidirectional similarity transformation to accurately evaluate the quality of retargeted images. Method Geometric distortion and information loss are two important issues in image retargeting. We propose a novel metric to quantify the geometric distortion and information loss of a retargeted image. Instead of only establishing a pseudo mapping relationship between the retargeted image and its original image, we regenerate the retargeted image from the original image and regenerate the original image from the retargeted image inversely. The issue of pixel matching is then converted into a field of similarity transformation. We use the scale-invariant feature transform (SIFT)-flow algorithm to extract a dense SIFT descriptor for each pixel in the original and retargeted images to build a reliable matching relation between the images. As a result, pixel-wise correspondences are established, then forward and backward similarity transformation matrices can be calculated from their corresponding mesh vertex coordinates. Similarity transformation matrix contains important information about the image retargeting process, which controls mesh deformation, and is a decisive factor for geometric distortion and information loss. Geometric distortion is calculated from the distance between a similarity transformation matrix and the benchmark transformation matrix in this study. Rotation and scaling parameters can reflect geometric distortion; hence, the distance defined in this work is composed of two components:the absolute distance difference and aspect change, which are never applied in previous methods. A large distance between the estimated and benchmark transformation matrices usually means large information loss. However, if a salient object is discarded or cropped, then the above geometric distortion measurement cannot correctly reflect such a type of information loss. Therefore, the pixels in the original or retargeted image are mapped to its opposite image with the forward or backward transformation, and the information loss is calculated from the missing areas. The quality of the retargeted image is obtained by the geometric distortion and information loss from the forward and backward transformations. Result Experimental results on the publicly available RetargetMe and CUHK datasets demonstrate the superiority of the proposed method. In the RetargetMe dataset, the Kendall rank correlation coefficient of the method reaches 0.46, and it has a good evaluation output on each subclass, especially on foreground objects, texture, and geometric structure. In the CUHK database, the Spearman's rank-order correlation coefficient is above 0.71. The results of one-way matching, including forward and backward matching, are worse than those of bidirectional matching affected by matching error. In this method, grid size affects similarity transformation and hence also influences geometric distortion and information loss measurements. We test different grid sizes, and the results indicate that the choice of grid size has a certain effect on quality prediction. The grid size of 16×16 has a relatively high performance on the CUHK and RetargetMe databases. Conclusion We present a novel IRQA method based on bidirectional similarity transformation. Unlike in traditional IRQA metrics that only estimate the matching similarity between original and retargeted images, we regenerate the retargeted image from the original image and the original image from the retargeted image to extract effective features from the similarity transformation matrix and the loss of mesh areas. The major contribution of this study is that it considers the retargeting operator as a process of image similarity transformation. The similarity transformation matrix connects original and retargeted images, and it exerts a great influence on the retargeted image quality. Features extracted from the similarity transformation matrix can measure geometric distortion accurately. Geometric change can lead to information loss if partial information is preserved or discarded. Thus, features extracted from the reduced mesh area can reflect the information loss of each retargeted image inerrably. The bidirectional matching mechanism that we employ can effectively reduce the influence of pixel matching error. Our quality assessment method therefore has a better correlation with subjective scores, outperforms existing methods, and is suitable for image retargeting quality prediction and optimizing retargeting algorithms.
Keywords

订阅号|日报