浏览全部资源
扫码关注微信
Published： 16 October 2023 ，
移动端阅览
唐祎玲， 江顺亮， 徐少平， 肖建， 陈晓军. 2023. 考虑双目竞争视觉现象的非对称失真立体图像质量评价方法. 中国图象图形学报， 28(10):3049-3063
Tang Yiling， Jiang Shunliang， Xu Shaoping， Xiao Jian， Chen Xiaojun. 2023. Binocular rivalry-based stereoscopic images quality assessment relevant to its asymmetric and distorted contexts. Journal of Image and Graphics， 28(10):3049-3063
目的
2
现有方法存在特征提取时间过长、非对称失真图像预测准确性不高的问题，同时少有工作对非对称失真与对称失真立体图像的分类进行研究，为此提出了基于双目竞争的非对称失真立体图像质量评价方法。
方法
2
依据双目竞争的视觉现象，利用非对称失真立体图像两个视点的图像质量衰减程度的不同，生成单目图像特征的融合系数，融合从左右视点图像中提取的灰度空间特征与HSV（hue-saturation-value）彩色空间特征。同时，量化两个视点图像在结构、信息量和质量衰减程度等多方面的差异，获得双目差异特征。并且将双目融合特征与双目差异特征级联为一个描述能力更强的立体图像质量感知特征向量，训练基于支持向量回归的特征—质量映射模型。此外，还利用双目差异特征训练基于支持向量分类模型的对称失真与非对称失真立体图像分类模型。
结果
2
本文提出的质量预测模型在4个数据库上的SROCC（Spearman rank order correlation coefficient）和PLCC（Pearson linear correlation coefficient）均达到0.95以上，在3个非对称失真数据库上的均方根误差（root of mean square error，RMSE）取值均优于对比算法。在LIVE-II（LIVE 3D image quality database phase II）、IVC-I（Waterloo-IVC 3D image quality assessment database phase I）和IVC-II（Waterloo-IVC 3D image quality assessment database phase II）这3个非对称失真立体图像测试数据库上的失真类型分类测试中，对称失真立体图像的分类准确率分别为89.91%、94.76%和98.97%，非对称失真立体图像的分类准确率分别为95.46%，92.64%和96.22%。
结论
2
本文方法依据双目竞争的视觉现象融合左右视点图像的质量感知特征用于立体图像质量预测，能够提升非对称失真立体图像的评价准确性和鲁棒性。所提取双目差异性特征还能够用于将对称失真与非对称失真立体图像进行有效分类，分类准确性高。
Objective
2
Computer vision-related stereoscopic image quality assessment （SIQA） is focused on recently. It is essential for parameter setting and system optimizing for such domains of multiple stereoscopic image applications like image storage， compression， transmission， and display. Stereoscopic images can be segmented into two sorts of distorted images： symmetrically and asymmetrically distorted， in terms of the degree of degradation between the left and right views. For symmetric-based distorted stereoscopic images， the distortion type and degree occurred in the left and right views are basically in consistency. Early SIQA methods were effective in evaluating symmetrically distorted images by averaging scores or features derived from the two views. However， in practice， the stereoscopic images are often asymmetrically distorted， where the distortion type and level of the two views are different. Simply averaging the quality values of the two views cannot accurately simulate the binocular fusion process and the binocular rivalry phenomena in relevance to the human visual system. Consequently， the evaluation accuracy of these methods will be down to severe lower when the quality of asymmetrically distorted stereoscopic images is estimated. Previous studies have shown that when the left and right views of a stereoscopic image exhibit varying levels or types of distortion， binocular rivalry is primarily driven by one of the views. Specially， in the process of evaluating the quality of a stereoscopic image， the visual quality of one view has a greater impact on the stereopair quality evaluation than the other view. To address this issue， some methods have simulated the binocular rivalry phenomenon in human visual system， and used a weighted average method to fuse the visual information in the two views of stereo-pairs as well. However， existing methods are still challenged for its lower prediction accuracy of asymmetrically distorted images， and its feature extraction process is also time-consuming. To optimize the evaluation accuracy of asymmetrically distorted images， we develop a binocular rivalry-based no-reference SIQA method.
Method
2
Multiple information-contained is used to generate image quality degradation coefficients in the two views， which can describe the degradation level of the distorted images accurately. According to the binocular rivalry phenomena in human visual system， the image quality degradation coefficients are used to generate fusion coefficients， which can be used to fuse the views-derived monocular features， including gray-scale features and HSV color space-extracted statistics. Since the human visual system is sensitive to structural information， the binocular structural similarity map （BSSIM） is constructed to measure the structural difference between the left and right views. As one part of the binocular difference features， structural difference features are extracted from the BSSIM. To quantify the differences between the left and right views， other related binocular difference features like entropy difference and degradation difference are obtained further. Finally， the binocular fusion features and the binocular difference features are concatenated into a more descriptive quality-aware feature vector， and a support vector regression model is trained to map the feature vector to the perception quality. In addition， to classify the symmetrically distorted stereoscopic images and the asymmetrically distorted stereoscopic images， a support vector classification model is also trained using the binocular difference features.
Result
2
To verify the performance of the proposed SIQA method， 4 sorts of publicly benchmark stereoscopic image databases are employed in relevance to the symmetrically and asymmetrically distorted stereoscopic images-involved LIVE 3D IQA Database Phase II （LIVE-II）， Waterloo-IVC 3D IQA Database Phase I （IVC-I）， and Waterloo-IVC 3D IQA Database Phase II （IVC-II）. Symmetrically distorted stereoscopic images are only involved in the LIVE 3D IQA Database Phase I （LIVE-I）. Comparative analysis is carried out in related to 10 state-of-the-art SIQA metrics. To measure the performance， three kinds of commonly-used performance indicators are involved in， including Spearman rank ordered correlation coefficient （SROCC）， Pearson linear correlation coefficient （PLCC）， and the root-mean-squared error （RMSE）. The experimental results demonstrate that the SROCCs and the PLCCs （higher is better） of the proposed method are higher than 0.95. Furthermore， the RMSEs （lower is better） of the proposed method can be reached to a potential lower degree. Additionally， the proposed classifier is tested on LIVE-II， IVC-I， and IVC-II databases. For LIVE-II database， 95.46% of asymmetrically distorted stereoscopic images can be classified accurately. For IVC-I and IVC-II databases， each of classification accuracy of symmetrically distorted images can be reached to 94.76% and 98.97%， and each of the classification accuracy of asymmetrically distorted images can be reached to 92.64% and 96.22% as well.
Conclusion
2
The degradation level can be quantified for the two views of asymmetrically distorted stereoscopic images. The image quality degradation coefficients are employed to fuse the monocular features， and it is beneficial to develop a more descriptive binocular perception feature vector and an improved prediction accuracy and robustness of asymmetrically distorted stereoscopic images. The proposed classifier can be used to clarify the symmetrically distorted stereoscopic images and the asymmetrically distorted stereoscopic images as well.
立体图像质量评价（SIQA）非对称失真双目竞争图像质量衰减系数双目差异特征
stereoscopic image quality assessment （SIQA）asymmetric distortionsbinocular rivalryimage quality degradation coefficientsbinocular difference features
Benoit A， Le Callet P， Campisi P and Cousseau R. 2009. Quality assessment of stereoscopic images. EURASIP Journal on Image and Video Processing， 2008（1）： #659024 ［DOI： 10.1155/2008/659024http://dx.doi.org/10.1155/2008/659024］
Chang C C and Lin C J. 2011. LIBSVM： a library for support vector machines. ACM Transactions on Intelligent Systems and Technology， 2（3）： #27 ［DOI： 10.1145/1961189.1961199http://dx.doi.org/10.1145/1961189.1961199］
Chen M J， Cormack L K and Bovik A C. 2013a. No-reference quality assessment of natural stereopairs. IEEE Transactions on Image Processing， 22（9）： 3379-3391 ［DOI： 10.1109/TIP.2013.2267393http://dx.doi.org/10.1109/TIP.2013.2267393］
Chen M J， Su C C， Kwon D K， Cormack L K and Bovik A C. 2013b. Full-reference quality assessment of stereopairs accounting for rivalry. Signal Processing： Image Communication， 28（9）： 1143-1155 ［DOI： 10.1016/j.image.2013.05.006http://dx.doi.org/10.1016/j.image.2013.05.006］
Chen Y， Jin M L， Zhu K X， Liu H L and Chen D. 2021. Blind stereo image quality evaluation based on spatial domain and transform domain feature extraction. Journal of Electronics and Information Technology， 43（10）： 2958-2966
陈勇， 金曼莉， 朱凯欣， 刘焕淋， 陈东. 2021. 结合空间域与变换域特征提取的盲立体图像质量评价. 电子与信息学报， 43（10）： 2958-2966 ［DOI： 10.11999/JEIT200694http://dx.doi.org/10.11999/JEIT200694］
Fang Y M， Sui X J， Yan J B， Liu X L and Huang L P. 2021. Progress in no-reference image quality assessment. Journal of Image and Graphics， 26（2）： 265-286
方玉明， 眭相杰， 鄢杰斌， 刘学林， 黄丽萍. 2021. 无参考图像质量评价研究进展. 中国图象图形学报， 26（2）： 265-286 ［DOI： 10.11834/jig.200274http://dx.doi.org/10.11834/jig.200274］
Fang Y M， Sui X J， Yan J B， Zuo Y F， Wang J H and Li Z Q. 2021. Asymmetrically distorted 3D video quality assessment： from the motion variation to perceived quality. Signal Processing， 183： #108031 ［DOI： 10.1016/j.sigpro.2021.108031http://dx.doi.org/10.1016/j.sigpro.2021.108031］
Han Y Y， Kebschull J M， Campbell R AA， Cowan D， Imhof F， Zador A M and Mrsic-Flogel T D. 2017. A single-cell anatomical blueprint for intracortical information transfer from primary visual cortex ［EB/OL］. ［2022-04-07］. https://www.biorxiv.org/content/10.1101/148031v2.full.pdf+htmlhttps://www.biorxiv.org/content/10.1101/148031v2.full.pdf+html
Li S M， Han X and Chang Y L. 2019. Adaptive cyclopean image-based stereoscopic image-quality assessment using ensemble learning. IEEE Transactions on Multimedia， 21（10）： 2616-2624 ［DOI： 10.1109/TMM.2019.2907470http://dx.doi.org/10.1109/TMM.2019.2907470］
Lin Y C， Yang J C， Lu W， Meng Q G， Lv Z H and Song H B. 2017. Quality index for stereoscopic images by jointly evaluating cyclopean amplitude and cyclopean phase. IEEE Journal of Selected Topics in Signal Processing， 11（1）： 89-101 ［DOI： 10.1109/JSTSP.2016.2632422http://dx.doi.org/10.1109/JSTSP.2016.2632422］
Liu L X， Liu B， Huang H and Bovik A C. 2014. No-reference image quality assessment based on spatial and spectral entropies. Signal Processing： Image Communication， 29（8）： 856-863 ［DOI： 10.1016/j.image.2014.06.006http://dx.doi.org/10.1016/j.image.2014.06.006］
Liu L X， Liu B， Su C C， Huang H and Bovik A C. 2017. Binocular spatial activity and reverse saliency driven no-reference stereopair quality assessment. Signal Processing： Image Communication， 58： 287-299 ［DOI： 10.1016/j.image.2017.08.011http://dx.doi.org/10.1016/j.image.2017.08.011］
Liu Y， Yan W Q， Zheng Z， Huang B Q and Yu H W. 2020. Blind stereoscopic image quality assessment accounting for human monocular visual properties and binocular interactions. IEEE Access， 8： 33666-33678 ［DOI： 10.1109/ACCESS.2020.2974006http://dx.doi.org/10.1109/ACCESS.2020.2974006］
Ma J， An P， Shen L Q and Li K. 2018. Joint binocular energy-contrast perception for quality assessment of stereoscopic images. Signal Processing： Image Communication， 65： 33-45 ［DOI： 10.1016/j.image.2018.03.009http://dx.doi.org/10.1016/j.image.2018.03.009］
Mittal A， Moorthy A K and Bovik A C. 2012. No-reference image quality assessment in the spatial domain. IEEE Transactions on Image Processing， 21（12）： 4695-4708 ［DOI： 10.1109/TIP.2012.2214050http://dx.doi.org/10.1109/TIP.2012.2214050］
Moorthy A K and Bovik A C. 2011. Blind image quality assessment： from natural scene statistics to perceptual quality. IEEE Transactions on Image Processing， 20（12）： 3350-3364 ［DOI： 10.1109/TIP.2011.2147325http://dx.doi.org/10.1109/TIP.2011.2147325］
Moorthy A K， Su C C， Mittal A and Bovik A C. 2013. Subjective evaluation of stereoscopic image quality. Signal Processing： Image Communication， 28（8）： 870-883 ［DOI： 10.1016/j.image.2012.08.004http://dx.doi.org/10.1016/j.image.2012.08.004］
Redi J A， Gastaldo P， Heynderickx I and Zunino R. 2010. Color distribution information for the reduced-reference assessment of perceived image quality. IEEE Transactions on Circuits and Systems for Video Technology， 20（12）： 1757-1769 ［DOI： 10.1109/TCSVT.2010.2087456http://dx.doi.org/10.1109/TCSVT.2010.2087456］
Shao F， Li K M， Lin W S， Jiang G Y and Yu M. 2015. Using binocular feature combination for blind quality assessment of stereoscopic images. IEEE Signal Processing Letters， 22（10）： 1548-1551 ［DOI： 10.1109/LSP.2015.2413946http://dx.doi.org/10.1109/LSP.2015.2413946］
Shao F， Zhang Z Q， Jiang Q P， Lin W S and Jiang G Y. 2018. Toward domain transfer for no-reference quality prediction of asymmetrically distorted stereoscopic images. IEEE Transactions on Circuits and Systems for Video Technology， 28（3）： 573-585 ［DOI： 10.1109/TCSVT.2016.2628082http://dx.doi.org/10.1109/TCSVT.2016.2628082］
Sheikh H R， Bovik A C and de Veciana G. 2005. An information fidelity criterion for image quality assessment using natural scene statistics. IEEE Transactions on Image Processing， 14（12）： 2117-2128 ［DOI： 10.1109/TIP.2005.859389http://dx.doi.org/10.1109/TIP.2005.859389］
Shen L L， Chen X F， Pan Z Q， Fan K F， Li F and Lei J J. 2021. No-reference stereoscopic image quality assessment based on global and local content characteristics. Neurocomputing， 424： 132-142 ［DOI： 10.1016/j.neucom.2020.10.024http://dx.doi.org/10.1016/j.neucom.2020.10.024］
Shen L L and Wang Y. 2020. No-reference stereoscopic image quality assessment based on singular value decomposition. Journal of Tianjin University （Science and Technology）， 53（6）： 641-646
沈丽丽， 王莹. 2020. 基于奇异值分解的无参考立体图像质量评价. 天津大学学报：自然科学与工程技术版）， 53（6）： 641-646 ［DOI： 10.11784/tdxbz201903054http://dx.doi.org/10.11784/tdxbz201903054］
Su C C， Cormack L K and Bovik A C. 2015a. Oriented correlation models of distorted natural images with application to natural stereopair quality evaluation. IEEE Transactions on Image Processing， 24（5）： 1685-1699 ［DOI： 10.1109/TIP.2015.2409558http://dx.doi.org/10.1109/TIP.2015.2409558］
Su C C， Moorthy A K and Bovik A C. 2015b. Visual quality assessment of stereoscopic image and video： challenges， advances， and future trends//Deng C W， Ma L， Lin W S and Ngan K N， eds. Visual Signal Quality Assessment： Quality of Experience （QoE）. Cham， Germany： Springer： 185-212 ［DOI： 10.1007/978-3-319-10368-6_7http://dx.doi.org/10.1007/978-3-319-10368-6_7］
Temel D， Prabhushankar M and Alregib G. 2016. UNIQUE： unsupervised image quality estimation. IEEE Signal Processing Letters， 23（10）： 1414-1418 ［DOI： 10.1109/LSP.2016.2601119http://dx.doi.org/10.1109/LSP.2016.2601119］
Wang J H， Rehman A， Zeng K， Wang S Q and Wang Z. 2015. Quality prediction of asymme trically distorted stereoscopic 3D images. IEEE Transactions on Image Processing， 24（11）： 3400-3414 ［DOI： 10.1109/TIP.2015.2446942http://dx.doi.org/10.1109/TIP.2015.2446942］
Wang J H， Wang S Q and Wang Z. 2017. Asymmetrically compressed stereoscopic 3D videos： quality assessment and rate-distortion performance evaluation. IEEE Transactions on Image Processing， 26（3）： 1330-1343 ［DOI： 10.1109/TIP.2017.2651387http://dx.doi.org/10.1109/TIP.2017.2651387］
Wang J H， Zeng K and Wang Z. 2014. Quality prediction of asymmetrically distorted stereoscopic images from single views//Proceedings of 2014 IEEE International Conference on Multimedia and Expo. Chengdu， China： IEEE： 14-18 ［DOI： 10.1109/icme.2014.6890303http://dx.doi.org/10.1109/icme.2014.6890303］
Wang Z， Bovik A C， Sheikh H R and Simoncelli E P. 2004. Image quality assessment： from error visibility to structural similarity. IEEE Transactions on Image Processing， 13（4）： 600-612 ［DOI： 10.1109/TIP.2003.819861http://dx.doi.org/10.1109/TIP.2003.819861］
Wang Z and Li Q. 2011. Information content weighting for perceptual image quality assessment. IEEE Transactions on Image Processing， 20（5）： 1185-1198 ［DOI： 10.1109/tip.2010.2092435http://dx.doi.org/10.1109/tip.2010.2092435］
Yang J C， Sim K， Lu W and Jiang B. 2019. Predicting stereoscopic image quality via stacked auto-encoders based on stereopsis formation. IEEE Transactions on Multimedia， 21（7）： 1750-1761 ［DOI： 10.1109/tmm.2018.2889562http://dx.doi.org/10.1109/tmm.2018.2889562］
Yasakethu S L P， Hewage C T E R， Fernando W A C and Kondoz A M. 2008. Quality analysis for 3D video using 2D video quality models. IEEE Transactions on Consumer Electronics， 54（4）： 1969-1976 ［DOI： 10.1109/tce.2008.4711260http://dx.doi.org/10.1109/tce.2008.4711260］
Yue G H， Hou C P， Jiang Q P and Yang Y. 2018. Blind stereoscopic 3D image quality assessment via analysis of naturalness， structure， and binocular asymmetry. Signal Processing， 150： 204-214 ［DOI： 10.1016/j.sigpro.2018.04.019http://dx.doi.org/10.1016/j.sigpro.2018.04.019］
相关文章
相关作者
相关机构
京公网安备11010802024621