浏览全部资源
扫码关注微信
收稿：2020-09-04，
修回：2021-1-25，
录用：2021-2-1，
纸质出版：2021-07-16
移动端阅览
目的
2
全景图像的质量评价和传输、处理过程并不是在同一个空间进行的，传统的评价算法无法准确地反映用户在观察球面场景时产生的真实感受，针对观察空间与处理空间不一致的问题，本文提出一种基于相位一致性的全参考全景图像质量评价模型。
方法
2
将平面图像进行全景加权，使得平面上的特征能准确反映球面空间质量畸变。采用相位一致性互信息的相似度获取参考图像和失真图像的结构相似度。接着，利用相位一致性局部熵的相似度反映参考图像和失真图像的纹理相似度。将两部分相似度融合可得全景图像的客观质量分数。
结果
2
实验在全景质量评价数据集OIQA（omnidirectional image quality assessment）上进行，在原始图像中引入4种不同类型的失真，将提出的算法与6种主流算法进行性能对比，比较了基于相位信息的一致性互信息和一致性局部熵，以及评价标准依据4项指标。实验结果表明，相比于现有的6种全景图像质量评估算法，该算法在PLCC（Pearson linear correlation coefficient）和SRCC（Spearman rank order correlation coefficient）指标上比WS-SSIM（weighted-to-spherically-uniform structural similarity）算法高出0.4左右，并且在RMSE（root of mean square error）上低0.9左右，4项指标最优，能够获得更好的拟合效果。
结论
2
本文算法解决了观察空间和映射空间不一致的问题，并且融合了基于人眼感知的多尺度互信息相似度和局部熵相似度，获得与人眼感知更为一致的客观分数，评价效果更为准确，更加符合人眼视觉特征。
Objective
2
Panoramic images introduce distortion in the process of acquisition
compression
and transmission. To provide viewers with a real experience
the resolution of a panoramic image is higher than that of the traditional image. The higher the resolution is
the more bandwidth is needed for transmission
and the more space is needed for storage. Therefore
image compression technology is conducive to improving transmission efficiency. At the same time
the compression distortion is introduced. With the increasing demand of viewers for panoramic image/video visual experience
the research on virtual reality visual system becomes increasingly important
and the quality evaluation of panoramic image/video is an indispensable part. The traditional subjective observation process of image is realized through the screen
and the design of objective quality assessment algorithm is based on 2D planes. When assessing the quality of panoramic images
viewers need to freely switch the perspective to observe the whole spherical scene with the help of head-mounted equipment. However
the transmission
storage
and processing are all in the projection format of the panoramic image
which causes the problem of inconsistency between the observation and processing spaces. As a result
the traditional assessment algorithm cannot accurately reflect the viewers' real feelings when observing the sphere
and cannot directly reflect the distortion degree of the spherical scene. To solve the problem of inconsistency between the observation and processing spaces
this study proposes a phase-consistency guided panoramic image quality assessment (PC-PIQA) algorithm.
Method
2
The structure and texture information are rich in high-resolution panoramic images
and they are the important features of the human visual system to understand the scene content. The proposed PC-PIQA model can solve the inconsistency between the observation space and processing plane by utilizing the features. Its panoramic statistical similarity is only related to the description parameters rather than the video content. First
the equirectangular projection format is mapped to the cube map projection (CMP) format
and the panoramic weight under the CMP format is used to solve the problem of inconsistent observation space and processing space.Then
the high-order phase-consistent mutual information of a single plane in the CMP format is calculated to describe the similarity of structural information between the reference image and distorted image at different orders.Next
the texture similarity is calculated by using the similarity of the first-order phase congruence local entropy. Finally
the visual quality of a single plane can be obtained by fusing the two parts of quality. According to the human eye's attention to the panoramic content
the different perceptual weights are assigned to six planes to obtain the overall quality score.
Result
2
Experiments are conducted on the panoramic evaluation data set called omnidirectional image quality assessment (OIQA). The original images are added by four different types of distortion
including JPEG compression
JPEG2000 compression
Gaussian blur
and Gaussian noise. The proposed algorithm is compared with six kinds of mainstream algorithm performance
including peak signal-to-noise ratio (PSNR)
structural similarity (SSIM)
craster parabolic projection PSNR (CPP-PSNR)
weighted-to-spherically-uniform PSNR (WS-PSNR)
spherical PSNR (S-PSNR) and weighted-to-spherically-uniform SSIM (WS-SSIM). The assessment criteria contains four indicators
including Pearson linear correlation coefficient (PLCC)
Spearman rank-order correlation coefficient (SRCC)
Kendall rank-order correlation coefficient (KRCC)
and root of mean square error (RMSE). In addition
we also list the performance obtained separately for structural similarity based on the panoramic weighted-mutual information (PW-MI) and texture similarity based on the panoramic weighted-local entropy (PW-LE)
which can prove that each factor plays a significant role in improving the performance. The experimental results show that the PLCC and SRCC indexes of this proposed algorithm are approximately 0.4 higher than that of the other existing models
and the RMSE index is approximately 0.9 lower. All the indexes are the best compared with the other existing six panoramic image-quality assessment algorithms. Meanwhile
the individual performance of PV-MI and PV-LE is also better than that of the reference panoramic algorithms. The algorithm not only solves the problem of inconsistency between the observation and processing spaces
but also has robustness to different distortion types and achieves the best fitting effect. The human visual system has different sensitivities to different scales of images
and experiment results show that the sampling scales with parameters of 2 and 4 perform better. Therefore
the mutual information of each order of phase consistency on the two scales and the local entropy of the first order of phase consistency are finally fused. The high-order phase consistency has a negative effect on the calculation of similarity. The proposed model performs best when using the local entropy with the first-order phase consistency.
Conclusion
2
The proposed algorithm solves the problem of inconsistency between the observation and processing space
and combines the multi-scale mutual information similarity and local entropy similarity based on human eye perception to obtain an objective score that is more consistent with the human eye perception. The assessment result is more accurate and consistent with the human visual system.The panoramic quality evaluation model proposed in this paper is classified as a traditional algorithm. With the development of deep learning
the framework implemented by neural networks can also obtain high accuracy. Further experiments are needed to determine if our model can be further integrated into neural network-based panoramic quality assessment.
Ai D, Dong J J, Lin N and Liu Y. 2018. Advance of 360-degree video coding for virtual reality: a survey. Application Research of Computers, 35(6): 1606-1612
艾达, 董久军, 林楠, 刘颖. 2018. 用于虚拟现实的360度视频编码技术新. 计算机应用研究, 35(6): 1606-1612[DOI:10.3969/j.issn.1001-3695.2018.06.002]
Cheung G, Yang L Y, Tan Z G and Huang Z. 2017. A content-aware metric for stitched panoramic image quality assessment//Proceedings of 2017 IEEE International Conference on Computer Vision Workshops (ICCVW). Venice: IEEE: 2487-2494[ DOI: 10.1109/ICCVW.2017.293 http://dx.doi.org/10.1109/ICCVW.2017.293 ]
Dedhia B, Chiang J C and Char Y F. 2019. Saliency prediction for omnidirectional images considering optimization on sphere domain//Proceedings of 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). Brighton, United Kingdom: IEEE: 2142-2146[ DOI: 10.1109/ICASSP.2019.8683125 http://dx.doi.org/10.1109/ICASSP.2019.8683125 ]
Greene N. 1986. Environment mapping and other applications of world projections. IEEE Computer Graphics and Applications, 6(11): 21-29[DOI:10.1109/MCG.1986.276658]
HoréA and Ziou D. 2010. Image quality metrics: PSNR vs. SSIM//Proceedings of the 20th IEEE International Conference on Pattern Recognition (ICPR). Istanbul: IEEE: 2366-2369[ DOI: 10.1109/ICPR.2010.579 http://dx.doi.org/10.1109/ICPR.2010.579 ]
Kovesi P. 1999. Image features from phase congruency. Videre: Journal of Computer Vision Research, 1(3): 1-26
Lebreton P and Raake A. 2018. GBVS360, BMS360, ProSal: extending existing saliency prediction models from 2D to omnidirectional images. Signal Processing: Image Communication, 69: 69-78[DOI:10.1016/j.image.2018.03.006]
Li Q H, Lin W S and Fang Y M. 2016. No-reference quality assessment for multiply-distorted images in gradient domain. IEEE Signal Processing Letters, 23(4): 541-545[DOI:10.1109/LSP.2016.2537321]
Ren Y F, Sun L, Wu G W and Huang W Z. 2017. DIBR-synthesized image quality assessment based on local entropy analysis//Proceedings of 2017 IEEE International Conference on the Frontiers and Advances in Data Science (FADS). Xi'an, China: IEEE: 2017: 86-90[ DOI: 10.1109/FADS.2017.8253200 http://dx.doi.org/10.1109/FADS.2017.8253200 ]
Sun Y L, Lu A and Yu L. 2017. Weighted-to-spherically-uniform quality evaluation for omnidirectional video. IEEE Signal Processing Letters, 24(9): 1408-1412[DOI:10.1109/LSP.2017.2720693]
Upenik E and Ebrahimi T. 2019. Saliency driven perceptual quality metric for omnidirectional visual content//Proceedings of 2019 IEEE International Conference on Image Processing (ICIP). Taipei, China: IEEE: 4335-4339[ DOI: 10.1109/ICIP.2019.8803637 http://dx.doi.org/10.1109/ICIP.2019.8803637 ]
Upenik E,Řeřábek M and Ebrahimi T. 2016. Testbed for subjective evaluation of omnidirectional visual content//Proceedings of 2016 IEEE Picture Coding Symposium (PCS). Nuremberg, Germany: IEEE: 1-5[ DOI: 10.1109/PCS.2016.7906378 http://dx.doi.org/10.1109/PCS.2016.7906378 ]
Wang Z, Bovik A C, Sheikh H R and Simoncelli E P. 2004. Image quality assessment: from error visibility to structural similarity. IEEE Transactions on Image Processing, 13(4): 600-612[DOI:10.1109/TIP.2003.819861]
Xu J H, Luo Z Y, Zhou W, Zhang W Y and Chen Z B. 2019a. Quality assessment of stereoscopic 360-degree images from multi-viewports//Proceedings of 2019 IEEE Picture Coding Symposium (PCS). Ningbo, China: IEEE: 1-5[ DOI: 10.1109/PCS48520.2019.8954555 http://dx.doi.org/10.1109/PCS48520.2019.8954555 ]
Xu M, Li C, Chen Z Z, Wang Z L and Guan Z Y. 2019b. Assessing visual quality of omnidirectional videos. IEEE Transactions on Circuits and Systems for Video Technology, 29(12): 3516-3530[DOI:10.1109/TCSVT.2018.2886277]
Xu X, Zhang H Q and Xia Z F. 2018. Quality assessment of 360-degree spherical images based on feature extraction in the wavelet domain. Video Engineering, 2018, 42(4): 36-40
许欣, 张会清, 夏志方. 2018. 基于小波域特征提取的360度全景图像质量评价. 电视技术, 2018, 42(4): 36-40[DOI:10.16280/j.videoe.2018.04.007]
Yang S, Zhao J Z, Jiang T T, Wang J, Rahim T, Zhang B, Xu Z J and Fei Z S. 2017. An objective assessment method based on multi-level factors for panoramic videos//Proceedings of 2017 IEEE Visual Communications and Image Processing (VCIP). St. Petersburg, USA: IEEE: 1-4[ DOI: 10.1109/VCIP.2017.8305133 http://dx.doi.org/10.1109/VCIP.2017.8305133 ]
Yu M, Lakshman H and Girod B. 2015. A framework to evaluate omnidirectional video coding schemes//Proceedings of 2017 IEEE International Symposium on Mixed and Augmented Reality (ISMAR). Fukuoka, Japan: IEEE: 31-36[ DOI: 10.1109/ISMAR.2015.12 http://dx.doi.org/10.1109/ISMAR.2015.12 ]
Zakharchenko V, Choi K P, Alshina E and Park J H. 2017. Omnidirectional video quality metrics and evaluation process//Proceedings of 2017 IEEE Data Compression Conference (DCC). Snowbird, USA: IEEE: #472[ DOI: 10.1109/DCC.2017.90 http://dx.doi.org/10.1109/DCC.2017.90 ]
Zhou Y F, Yu M, Ma H L, Shao H and Jiang G Y. 2018. Weighted-to-spherically-uniform SSIM objective quality evaluation for panoramic video//Proceedings of 2018 IEEE International Conference on Signal Processing (ICSP). Beijing, China: IEEE: 54-57[ DOI: 10.1109/ICSP.2018.8652269 http://dx.doi.org/10.1109/ICSP.2018.8652269 ]
Zhu Y C, Zhai G T and Min X K. 2018. The prediction of head and eye movement for 360 degree images. Signal Processing: Image Communication, 69: 15-25[DOI:10.1016/j.image.2018.05.010]
相关作者
相关机构
京公网安备11010802024621
