Current Issue Cover
无参考图像质量评价研究进展

方玉明, 眭相杰, 鄢杰斌, 刘学林, 黄丽萍(江西财经大学信息管理学院, 南昌 330032)

摘 要
图像质量评价一直是图像处理和计算机视觉领域的一个基础问题,图像质量评价模型也广泛应用于图像/视频编码、超分辨率重建和图像/视频视觉质量增强等相关领域。图像质量评价主要包括全参考图像质量评价、半参考图像质量评价和无参考图像质量评价。全参考图像质量评价和半参考图像质量评价分别指预测图像质量时参考信息完全可用和部分可用,而无参考图像质量评价是指预测图像质量时参考信息不可用。虽然全参考和半参考图像质量评价模型较为可靠,但在计算过程中必须依赖参考信息,使得应用场景极为受限。无参考图像质量评价模型因不需要依赖参考信息而有较强的适用性,一直都是图像质量评价领域研究的热点。本文主要概述2012—2020年国内外公开发表的无参考图像质量评价模型,根据模型训练过程中是否需要用到主观分数,将无参考图像质量评价模型分为有监督学习和无监督学习的无参考图像质量评价模型。同时,每类模型分成基于传统机器学习算法的模型和基于深度学习算法的模型。对基于传统机器学习算法的模型,重点介绍相应的特征提取策略及思想;对基于深度学习算法的模型,重点介绍设计思路。此外,本文介绍了图像质量评价在新媒体数据中的研究工作及图像质量评价的应用。最后对介绍的无参考图像质量评价模型进行总结,并指出未来可能的发展方向。
关键词
Progress in no-reference image quality assessment

Fang Yuming, Sui Xiangjie, Yan Jiebin, Liu Xuelin, Huang Liping(School of Information Technology, Jiangxi University of Finance and Economics, Nanchang 330032, China)

Abstract
Image quality assessment (IQA) has been a fundamental issue in the fields of image processing and computer vision. It has also been extensively applied to other relevant research areas, such as image/video coding, super-resolution and visual enhancement. In general, IQA consists of subjective and objective evaluations. Subjective evaluation always refers to estimating the visual quality of images by subject, with the goal of building test benchmarks. Objective evaluation typically resorts to computational algorithms (i.e., IQA models) to make visual quality predictions, and its ultimate objective is to provide consistent judgment with subjects. The effectiveness of objective IQA models must be verified on test benchmarks built via subjective evaluation. Undoubtedly, subjective evaluation cannot be fully embedded into multimedia processing applications because such process is time-consuming and labor-intensive. By contrast, an objective IQA model can work efficiently as an important module in multimedia processing applications, playing roles in visual image quality monitoring, image filtering, and visual quality enhancement. Given their availability, research on objective IQA models has elicited considerable attention from industries and academia. Objective IQA models can be classified into three categories: full-reference (FR), reduced-reference (RR), and no-reference/blind (NR) models. FR and RR models denote that reference information for estimating the visual quality of images is completely and partially available, respectively. Meanwhile, an NR model indicates that reference information is unavailable for visual quality prediction. Although reference-based IQA models (i.e., FR and RR models) are relatively reliable, their applications are limited to specific scenarios due to their dependence on reference information. By contrast, NR-IQA models are more flexible than reference-based models because they are free from the constraint of reference information. Consequently, NR-IQA models have consistently been a popular research topic over the past decades. In this study, we introduce NR-IQA models published from 2012 to 2020 to provide a comprehensive survey on feature engineering and end-to-end learning techniques in NR-IQA. In accordance with whether subjective quality scores are involved in training procedures, NR-IQA models are classified into two categories: opinion-aware/supervised and opinion-unaware/unsupervised NR-IQA models. To present a clear and integrated description, each category is further divided into two subclasses: traditional machine learning-based models (MLMs) and deep learning-based models (DLMs). For the former subclass, we mostly investigate their individual feature extraction schemes and the principle behind these schemes. In particular, a widely adopted feature extraction approach in MLMs, namely, natural scene statistics (NSS), is introduced in this study. The principle of NSS is as follows: some visual features of quality perfect images follow certain associated distributions; meanwhile, different types of distortions will break this rule in corresponding methods. On the basis of this observation/fact, researchers have proposed many NSS-based NR-IQA methods, in which the estimated parameters of the established distributions are used as quality-aware features. Thereafter, a machine learning algorithm is selected to train the IQA models. Another well-known feature extraction approach described in this study relies on dictionary learning, which is frequently accompanied by sparse coding. The core component of this type of feature extraction approach is to learn a dictionary by searching for a group of over-complete bases. Then, these over-complete bases are used to build a reference system for image representation. A test image can be concretely represented directly or indirectly by the constructed dictionary by using sparse indexes or cluster centroids. Image representations are further used as quality-aware features to capture variations in image quality. For the latter subclass (i.e., DLMs), the design principles described in detail in this paper mostly correspond to different architectures of deep neural networks. In particular, we introduce three different schemes for designing opinion-aware DLMs and commonly used strategies in opinion-unaware DLMs. To guarantee length balance among various contents and clearly exhibit the differences between NR-IQA models designed for natural images and other types of images, we introduce them separately in subsections. In addition, we provide a brief introduction into IQA research on new media, including virtual reality, light field, and underwater sonar images, along with the applications of IQA models. Finally, an in-depth conclusion about NR-IQA models is drawn in the last section. We summarize the current achievements and limitations of MLMs and DLMs. Furthermore, we highlight the potential development trends and directions of NR-IQA models for further improvements from the perspectives of image contents and NR-IQA models.
Keywords

订阅号|日报