Current Issue Cover
相位一致性指导的全参考全景图像质量评价

夏雨蒙1, 王永芳1,2, 王闯1(1.上海大学通信与信息工程学院, 上海 200444;2.上海大学上海先进通信与数据科学研究所, 上海 200444)

摘 要
目的 全景图像的质量评价和传输、处理过程并不是在同一个空间进行的,传统的评价算法无法准确地反映用户在观察球面场景时产生的真实感受,针对观察空间与处理空间不一致的问题,本文提出一种基于相位一致性的全参考全景图像质量评价模型。方法 将平面图像进行全景加权,使得平面上的特征能准确反映球面空间质量畸变。采用相位一致性互信息的相似度获取参考图像和失真图像的结构相似度。接着,利用相位一致性局部熵的相似度反映参考图像和失真图像的纹理相似度。将两部分相似度融合可得全景图像的客观质量分数。结果 实验在全景质量评价数据集OIQA(omnidirectional image quality assessment)上进行,在原始图像中引入4种不同类型的失真,将提出的算法与6种主流算法进行性能对比,比较了基于相位信息的一致性互信息和一致性局部熵,以及评价标准依据4项指标。实验结果表明,相比于现有的6种全景图像质量评估算法,该算法在PLCC(Pearson linear correlation coefficient)和SRCC(Spearman rank order correlation coefficient)指标上比WS-SSIM(weighted-to-spherically-uniform structural similarity)算法高出0.4左右,并且在RMSE(root of mean square error)上低0.9左右,4项指标最优,能够获得更好的拟合效果。结论 本文算法解决了观察空间和映射空间不一致的问题,并且融合了基于人眼感知的多尺度互信息相似度和局部熵相似度,获得与人眼感知更为一致的客观分数,评价效果更为准确,更加符合人眼视觉特征。
关键词
Phase consistency guided full-reference panoramic image quality assessment algorithm

Xia Yumeng1, Wang Yongfang1,2, Wang Chuang1(1.School of Communication and Information Engineering, Shanghai University, Shanghai 200444, China;2.Shanghai Institute for Advanced Communication and Data Science, Shanghai University, Shanghai 200444, China)

Abstract
Objective Panoramic images introduce distortion in the process of acquisition, compression, and transmission. To provide viewers with a real experience, the resolution of a panoramic image is higher than that of the traditional image. The higher the resolution is, the more bandwidth is needed for transmission, and the more space is needed for storage. Therefore, image compression technology is conducive to improving transmission efficiency. At the same time, the compression distortion is introduced. With the increasing demand of viewers for panoramic image/video visual experience, the research on virtual reality visual system becomes increasingly important, and the quality evaluation of panoramic image/video is an indispensable part. The traditional subjective observation process of image is realized through the screen, and the design of objective quality assessment algorithm is based on 2D planes. When assessing the quality of panoramic images, viewers need to freely switch the perspective to observe the whole spherical scene with the help of head-mounted equipment. However, the transmission, storage, and processing are all in the projection format of the panoramic image, which causes the problem of inconsistency between the observation and processing spaces. As a result, the traditional assessment algorithm cannot accurately reflect the viewers’ real feelings when observing the sphere, and cannot directly reflect the distortion degree of the spherical scene. To solve the problem of inconsistency between the observation and processing spaces, this study proposes a phase-consistency guided panoramic image quality assessment (PC-PIQA) algorithm.Method The structure and texture information are rich in high-resolution panoramic images, and they are the important features of the human visual system to understand the scene content. The proposed PC-PIQA model can solve the inconsistency between the observation space and processing plane by utilizing the features. Its panoramic statistical similarity is only related to the description parameters rather than the video content. First, the equirectangular projection format is mapped to the cube map projection (CMP) format, and the panoramic weight under the CMP format is used to solve the problem of inconsistent observation space and processing space.Then, the high-order phase-consistent mutual information of a single plane in the CMP format is calculated to describe the similarity of structural information between the reference image and distorted image at different orders.Next, the texture similarity is calculated by using the similarity of the first-order phase congruence local entropy. Finally, the visual quality of a single plane can be obtained by fusing the two parts of quality. According to the human eye’s attention to the panoramic content, the different perceptual weights are assigned to six planes to obtain the overall quality score.Result Experiments are conducted on the panoramic evaluation data set called omnidirectional image quality assessment (OIQA). The original images are added by four different types of distortion, including JPEG compression, JPEG2000 compression, Gaussian blur, and Gaussian noise. The proposed algorithm is compared with six kinds of mainstream algorithm performance, including peak signal-to-noise ratio (PSNR), structural similarity (SSIM), craster parabolic projection PSNR (CPP-PSNR), weighted-to-spherically-uniform PSNR (WS-PSNR), spherical PSNR (S-PSNR) and weighted-to-spherically-uniform SSIM (WS-SSIM). The assessment criteria contains four indicators, including Pearson linear correlation coefficient (PLCC), Spearman rank-order correlation coefficient (SRCC), Kendall rank-order correlation coefficient (KRCC), and root of mean square error (RMSE). In addition, we also list the performance obtained separately for structural similarity based on the panoramic weighted-mutual information (PW-MI) and texture similarity based on the panoramic weighted-local entropy (PW-LE), which can prove that each factor plays a significant role in improving the performance. The experimental results show that the PLCC and SRCC indexes of this proposed algorithm are approximately 0.4 higher than that of the other existing models, and the RMSE index is approximately 0.9 lower. All the indexes are the best compared with the other existing six panoramic image-quality assessment algorithms. Meanwhile, the individual performance of PV-MI and PV-LE is also better than that of the reference panoramic algorithms. The algorithm not only solves the problem of inconsistency between the observation and processing spaces, but also has robustness to different distortion types and achieves the best fitting effect. The human visual system has different sensitivities to different scales of images, and experiment results show that the sampling scales with parameters of 2 and 4 perform better. Therefore, the mutual information of each order of phase consistency on the two scales and the local entropy of the first order of phase consistency are finally fused. The high-order phase consistency has a negative effect on the calculation of similarity. The proposed model performs best when using the local entropy with the first-order phase consistency.Conclusion The proposed algorithm solves the problem of inconsistency between the observation and processing space, and combines the multi-scale mutual information similarity and local entropy similarity based on human eye perception to obtain an objective score that is more consistent with the human eye perception. The assessment result is more accurate and consistent with the human visual system.The panoramic quality evaluation model proposed in this paper is classified as a traditional algorithm. With the development of deep learning, the framework implemented by neural networks can also obtain high accuracy. Further experiments are needed to determine if our model can be further integrated into neural network-based panoramic quality assessment.
Keywords

订阅号|日报