Objective

2

Image retargeting

which has become an increasingly in-demand tool with the proliferation of mobile devices

aims to adjust images into different sizes or aspect ratios for various display screens. Many retargeting methods have been proposed during the past few years

but a single method that works efficiently on any image still does not exist. Different images favor different retargeting algorithms

and a key problem is to estimate the performance of each retargeting operator. Image retargeting quality assessment (IRQA) is an effective way to improve the performance of image retargeting techniques and be utilized to select favorable retargeting approaches for real applications. Nevertheless

objective IRQA is always a challenging research problem. First

the resolution of a retargeted image is different from that of its original image; thus

the problems of IRQA become different from those of the traditional image quality assessment (IQA). For example

traditional full-reference IQA methods

such as structural similarity and feature similarity indices

measure pixel-to-pixel similarity to capture image quality scores

which cannot be directly applied to IRQA. Second

traditional IQA metrics mainly focus on estimating the perceptual similarities between a source image and its corresponding non-geometrically distorted version. With respect to the IRQA problem

the perceptual quality of a retargeted image is intensely related to human cognition of this image. The structure and semantic information of an object should be consistent with the prior knowledge of humans. We propose a new method for IRQA via bidirectional similarity transformation to accurately evaluate the quality of retargeted images.

Method

2

Geometric distortion and information loss are two important issues in image retargeting. We propose a novel metric to quantify the geometric distortion and information loss of a retargeted image. Instead of only establishing a pseudo mapping relationship between the retargeted image and its original image

we regenerate the retargeted image from the original image and regenerate the original image from the retargeted image inversely. The issue of pixel matching is then converted into a field of similarity transformation. We use the scale-invariant feature transform (SIFT)-flow algorithm to extract a dense SIFT descriptor for each pixel in the original and retargeted images to build a reliable matching relation between the images. As a result

pixel-wise correspondences are established

then forward and backward similarity transformation matrices can be calculated from their corresponding mesh vertex coordinates. Similarity transformation matrix contains important information about the image retargeting process

which controls mesh deformation

and is a decisive factor for geometric distortion and information loss. Geometric distortion is calculated from the distance between a similarity transformation matrix and the benchmark transformation matrix in this study. Rotation and scaling parameters can reflect geometric distortion; hence

the distance defined in this work is composed of two components:the absolute distance difference and aspect change

which are never applied in previous methods. A large distance between the estimated and benchmark transformation matrices usually means large information loss. However

if a salient object is discarded or cropped

then the above geometric distortion measurement cannot correctly reflect such a type of information loss. Therefore

the pixels in the original or retargeted image are mapped to its opposite image with the forward or backward transformation

and the information loss is calculated from the missing areas. The quality of the retargeted image is obtained by the geometric distortion and information loss from the forward and backward transformations.

Result

2

Experimental results on the publicly available RetargetMe and CUHK datasets demonstrate the superiority of the proposed method. In the RetargetMe dataset

the Kendall rank correlation coefficient of the method reaches 0.46

and it has a good evaluation output on each subclass

especially on foreground objects

texture

and geometric structure. In the CUHK database

the Spearman's rank-order correlation coefficient is above 0.71. The results of one-way matching

including forward and backward matching

are worse than those of bidirectional matching affected by matching error. In this method

grid size affects similarity transformation and hence also influences geometric distortion and information loss measurements. We test different grid sizes

and the results indicate that the choice of grid size has a certain effect on quality prediction. The grid size of 16×16 has a relatively high performance on the CUHK and RetargetMe databases.

Conclusion

2

We present a novel IRQA method based on bidirectional similarity transformation. Unlike in traditional IRQA metrics that only estimate the matching similarity between original and retargeted images

we regenerate the retargeted image from the original image and the original image from the retargeted image to extract effective features from the similarity transformation matrix and the loss of mesh areas. The major contribution of this study is that it considers the retargeting operator as a process of image similarity transformation. The similarity transformation matrix connects original and retargeted images

and it exerts a great influence on the retargeted image quality. Features extracted from the similarity transformation matrix can measure geometric distortion accurately. Geometric change can lead to information loss if partial information is preserved or discarded. Thus

features extracted from the reduced mesh

$${\rm area}$$

can reflect the information loss of each retargeted image inerrably. The bidirectional matching mechanism that we employ can effectively reduce the influence of pixel matching error. Our quality assessment method therefore has a better correlation with subjective scores

outperforms existing methods

and is suitable for image retargeting quality prediction and optimizing retargeting algorithms.