Current Issue Cover
图像质量评价研究综述——从失真的角度

鄢杰斌, 方玉明, 刘学林(江西财经大学信息管理学院, 南昌 330032)

摘 要
随着多媒体技术的快速发展及广泛应用,图像质量评价因其在多媒体处理中的重要作用得到越来越多的关注,其作用包括图像数据筛选、算法参数选择与优化等。根据图像质量评价应用时是否需要参考信息,它可分为全参考图像质量评价、半参考图像质量评价和无参考图像质量评价,前两类分别需要全部参考信息和部分参考信息,而第3类不需要参考信息。无论是全参考、半参考还是无参考图像质量评价,图像失真对图像质量评价的影响均较大,主要体现在图像质量评价数据库构建和图像质量评价模型设计两方面。本文从图像失真的角度,主要概述2011—2021年国内外公开发表的图像质量评价模型,涵盖全参考、半参考和无参考模型。根据图像的失真类型,将图像质量评价模型分为针对合成失真的图像质量评价模型、针对真实失真的图像质量评价模型和针对算法相关失真的图像质量评价模型。其中,合成失真是指人工添加噪声,如高斯噪声和模糊失真,通常呈现均匀分布;真实失真是指在图像的获取中,由于环境、拍摄设备或拍摄操作不当等因素所引入的失真类型。相对合成失真,真实失真更为复杂,可能包括一种或多种失真,数据收集难度更大;算法相关失真是指图像处理算法或计算机视觉算法在处理图像时,由于算法本身的缺陷或性能不足等原因而出现在结果图像中的降质,相对合成失真和真实失真,算法相关失真的显著特点是该类型失真呈现非均匀分布。本文介绍现有的图像质量评价数据库,包括图像数据来源和数据库构建细节等;然后重点介绍图像质量评价模型的设计思想。最后总结了介绍的图像质量评价模型,并指出未来可能的发展方向。
关键词
The review of distortion-related image quality assessment

Yan Jiebin, Fang Yuming, Liu Xuelin(School of Information Technology, Jiangxi University of Finance and Economics, Nanchang 330032, China)

Abstract
Image quality assessment (IQA) has been widely applied to multimedia processing, such as low-quality image screening, parameter tuning, and algorithm optimization. IQA models can be classified into three categories like full-reference (FR), reduced-reference (RR) and no-reference/blind (NR/B) models in terms of the accessibility to reference information. The FR and RR IQA models are based on full and partial reference information each while NR/B models can assess image visual quality without reference information. The early IQA models mainly relies on feature engineering, and their performance is limited by the representation ability of hand-crafted features. Benefiting from the excellent representation ability of deep neural networks, the deep learning based IQA models have shown superior performance and surpassed the feature engineering based IQA models, especially in representing large number of images. Due to the strong representation ability, the deep learning based IQA models have attracted most of our attention in the past decade. Additionally, IQA models have been introduced to image restoration, super-resolution and other tasks to improve algorithm performance and facilitate better benchmarks. For instance, some differentiable FR-IQA models like structural similarity(SSIM) and multi-scale SSIM(MSSIM) can be used straightforward via optimization, facilitating the relevant models training. In the development of IQA research, image distortion plays an important role. Both image quality database construction and IQA models are developed by considering distortion. Previous IQA studies are constrained of dozens of pristine images, and then generate their distorted images via disrupting the pristine images artificially. Benefiting from the uniformity of this type of images, the global quality score of each image in the early image quality databases can be regarded as the quality label of the image patches (e.g., 32×32 pixels) extracted from the corresponding image. Consequently, huge amount of data like local patches and the associated global quality scores can be carried out to learn the mapping function from local patch to visual quality. However, the patch-based strategy may be problematic, since such a small patch is unable to capture image context information. Besides, this data augmentation operation could not be simply adapted to quality assessment of authentic distorted images and other types of images such as 3D synthesized images, since authentic distortions are much more complex than simulated distortions and the distortions in 3D synthesized images exhibit non-uniform distribution. Customized solutions regarding to image distortion are required to be implemented in IQA models, e.g., we can use local variation measurement and global change modeling to capture the distortion in 3D synthesized images. Our research mainly introduces the distortion-related IQA models from 2011 to 2021, aiming to facilitate a review on the evolution of image quality databases and IQA models both. In the light of image distortion categories, IQA models are classified into three categories:the models for simulated distortions, the models for authentic distortions, and the models for the distortions related to algorithms. Specifically, simulated distortions are also called artificial distortions, which show uniform distribution, including Gaussian noise and motion blur, etc. Authentic distortions are mainly introduced in the context of image photography scenario, shooting equipment, inadequate operation in shooting process. In comparison of simulated distortions, authentic distortions are more complicated, leading to the challenge of data collection. Authentic distortions can be also classified into several major types, like out-of-focus, blur and over-exposure. A large number of authentical distorted images collection need to be followed some distortion rules, and it is a labor-intensive task. The distortions related to algorithms refer to the image degradations occurred in the resultant images due to the intrinsic defects or the limited performance of image processing or computer vision algorithms, and a distinctive characteristic of this type of distortions is that their distributions are inhomogeneous, which differs from other two types of distortions. Overall, current public image quality databases are initial to be promoted, including the details of image data sources and database construction. The designation of IQA models is then involved in. Finally, we summarize the IQA models introduced in this paper, and point out the potential development directions in the future.
Keywords

订阅号|日报