浏览全部资源
扫码关注微信
Published： 16 May 2022 ，
Accepted： 22 December 2021
移动端阅览
Keyan Wang, Shirui Huang, Yunsong Li. An optical reconstruction based underwater image analysis. [J]. Journal of Image and Graphics 27(5):1337-1358(2022)
水下光学图像可以提供直观丰富的海洋信息，近年来在海洋资源开发、环境保护和海洋工程等诸多领域发挥越来越重要的作用。但是受恶劣复杂的水下成像环境影响，水下光学图像普遍存在对比度低、图像模糊以及颜色失真等质量退化问题，严重制约水下智能处理系统的性能和应用。如何清晰地重建水下光学图像是国内外广泛关注的、具有挑战性的难点问题。随着深度学习技术的蓬勃发展，利用深度学习来提升水下图像质量成为当前的研究热点。鉴于目前国内在水下光学图像重建方面的研究综述较少，本文全面综述其研究进展。分析了水下图像退化机理，总结了现有水下成像模型以及水下图像重建的挑战；梳理了水下光学图像重建方法的发展历程，根据是否采用深度学习以及是否基于成像模型，将现有方法分为4大类，并按照研究发展顺序，依次介绍4类方法的基本思想，分析其优缺点；归纳了目前公开的水下图像数据集以及常用的水下图像质量评价方法，并对8种典型的水下图像重建方法进行了性能评测和对比分析；总结了该领域目前仍存在的问题，展望了后续研究方向，以便于相关研究人员了解该领域的研究现状，促进该领域的技术发展。
Underwater optical images have played a key role in related to marine resource development
marine environment protection and marine engineering. Due to the harsh and complex underwater environment
the raw underwater optical images are challenged to the quality degradation
which are mainly introduced via an underwater scenario of light selective absorption and scattering. The degraded underwater images have low contrast
blurred details and color distortion
which severely restrict its performance and applications of underwater intelligent processing systems. In particular
the current deep learning based underwater image recovering has been facilitated currently. Our review first analyzes the mechanism of underwater image degradation
as well as describes the existing underwater imaging models and summarizes the challenges of underwater image reconstruction. Then
we trace the evolving of underwater optical image reconstruction methods. In accordance with the deep learning technology and the physical imaging models contextual
the existing algorithms are segmented into four categories for underwater images recovering methods
which are traditional image enhancement
traditional image restoration
deep-learning-based image enhancement and deep-learning-based image restoration. These four types of existing timescale methods are then discussed and analyzed
including their theories
features
pros and cons
respectively. Specifically
the traditional image enhancement methods tend to deliver unnatural results with color deviations and under-enhancement or over-enhancement of local area to improve the visibility of the underwater images effectively. Conversely
the traditional image restoration methods are based on the mechanism of underwater image degradation and a challenging issue is constrained of the limitations of imaging models and priori knowledge in diverse underwater scenarios. The deep learning based restoration methods is facilitated to nonlinear fitting capabilities of neural networks with an error stacking barrier and insufficient restoration results. The deep-learning-based enhancement methods are comparatively robust and suitable to recover diverse underwater images with flexible neural networks. However
they are relatively difficult to converge
and their generalization capability is insufficient. We introduce the current public underwater image datasets
which can be divided into two categories
and summarize their features and applications
respectively. Moreover
we also demonstrate more evaluation metrics for the quality of underwater images. To evaluate their performance quantitatively and qualitatively
we conduct experiments on eight typical underwater image reconstruction methods further. Three benchmarks are optioned for color cast removal
contrast improvement and comprehensive test. Our demonstrated results indicate that none of these methods are robust enough to recover diverse types of underwater images. The reconstruction of better underwater optical images is a big challenge issue of improving the robustness and generalization of reconstruction
developing more lightweight networks or processing algorithms of real-time applications
utilizing underwater image reconstruction to harness vision tasks ability
as well as establishing an underwater images quality assessment system.
水下图像退化深度学习图像增强图像复原水下数据集水下图像质量评价
underwater degraded imagedeep learningimage enhancementimage restorationunderwater benchmarkunderwater image quality assessment
Akkaynak D and Treibitz T. 2018. A revised underwater image formation model//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, USA: IEEE: 6723-6732[DOI: 10.1109/CVPR.2018.00703http://dx.doi.org/10.1109/CVPR.2018.00703]
Ancuti C, Ancuti C O, Haber T and Bekaert P. 2012. Enhancing underwater images and videos by fusion//Proceedings of 2012 IEEE Conference on Computer Vision and Pattern Recognition. Providence, USA: IEEE: 81-88[DOI: 10.1109/CVPR.2012.6247661http://dx.doi.org/10.1109/CVPR.2012.6247661]
Ancuti C O, Ancuti C, De Vleeschouwer C and Bekaert P. 2018. Color balance and fusion for underwater image enhancement. IEEE Transactions on Image Processing, 27(1): 379-393[DOI: 10.1109/TIP.2017.2759252]
Ancuti C O, Ancuti C, De Vleeschouwer C and Sbert M. 2020. Color channel compensation (3C): a fundamental pre-processing step for image enhancement. IEEE Transactions on Image Processing, 29: 2653-2665[DOI: 10.1109/TIP.2019.2951304]
Anwar S, Li C Y and Porikli F. 2018. Deep underwater image enhancement[EB/OL]. [2021-08-13].https://arxiv.org/pdf/1807.03528.pdfhttps://arxiv.org/pdf/1807.03528.pdf
Barbosa W V, Amaral H G B, Rocha T L and Nascimento E R. 2018. Visual-quality-driven learning for underwater vision enhancement//Proceedings of the 25th IEEE International Conference on Image Processing (ICIP). Athens, Greece: IEEE: 3933-3937[DOI: 10.1109/ICIP.2018.8451356http://dx.doi.org/10.1109/ICIP.2018.8451356]
Berman D, Levy D, Avidan S and Treibitz T. 2021. Underwater single image color restoration using haze-lines and a new quantitative dataset. IEEE Transactions on Pattern Analysis and Machine Intelligence, 43(8): 2822-2837[DOI: 10.1109/TPAMI.2020.2977624]
Cai B L, Xu X M, Jia K, Qing C and Tao D C. 2016. DehazeNet: an end-to-end system for single image haze removal. IEEE Transactions on Image Processing, 25(11): 5187-5198[DOI: 10.1109/TIP.2016.2598681]
Carlevaris-Bianco N, Mohan A and Eustice R M. 2010. Initial results in underwater single image dehazing//OCEANS 2010 MTS/IEEE SEATTLE. Seattle, USA: IEEE: 1-8[DOI: 10.1109/OCEANS.2010.5664428http://dx.doi.org/10.1109/OCEANS.2010.5664428]
Chiang J Y and Chen Y C. 2012. Underwater image enhancement by wavelength compensation and dehazing. IEEE Transactions on Image Processing, 21(4): 1756-1769[DOI: 10.1109/TIP.2011.2179666]
Cong R M, Zhang Y M, Zhang C, Li C Y and Zhao Y. 2020. Research progress of deep learning driven underwater image enhancement and restoration. Journal of Signal Processing, 36(9): 1377-1389
从润民, 张禹墨, 张晨, 李重仪, 赵耀. 2020. 深度学习驱动的水下图像增强与复原研究进展. 信号处理, 36(9): 1377-1389)[DOI: 10.16798/j.issn.1003-0530.2020.09.001]
Desai C, Tabib R A, Reddy S S, Patil U and Mudenagudi U. 2021. RUIG: realistic underwater image generation towards restoration//Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). Nashville, USA: IEEE: 2181-2189[DOI: 10.1109/CVPRW53098.2021.00247http://dx.doi.org/10.1109/CVPRW53098.2021.00247]
Drews P Jr, do Nascimento E, Moraes F, Botelho S and Campos M. 2013. Transmission estimation in underwater single images//Proceedings of 2013 IEEE International Conference on Computer Vision Workshops. Sydney, Australia: IEEE: 825-830[DOI: 10.1109/ICCVW.2013.113http://dx.doi.org/10.1109/ICCVW.2013.113]
Duarte A, Codevilla F, De O Gaya J and Botelho S S C. 2016. A dataset to evaluate underwater image restoration methods//OCEANS 2016. Shanghai, China: IEEE: 1-6[DOI: 10.1109/OCEANSAP.2016.7485524http://dx.doi.org/10.1109/OCEANSAP.2016.7485524]
Fu X P, Shang X D, Sun X D, Yu H Y, Song M P and Chang C I. 2020. Underwater hyperspectral target detection with band selection. Remote Sensing, 12(7): #1056[DOI: 10.3390/rs12071056]
Fu X Y, Zhuang P X, Huang Y, Liao Y H, Zhang X P and Ding X H. 2014. A retinex-based enhancing approach for single underwater image//Proceedings of 2014 IEEE International Conference on Image Processing (ICIP). Paris, France: IEEE: 4572-4576[DOI: 10.1109/ICIP.2014.7025927http://dx.doi.org/10.1109/ICIP.2014.7025927]
Galdran A, Pardo D, Picón A and Alvarez-Gila A. 2015. Automatic red-channel underwater image restoration. Journal of Visual Communication and Image Representation, 26: 132-145[DOI: 10.1016/j.jvcir.2014.11.006]
Guo J C, Li C Y, Guo C L and Chen S J. 2017. Research progress of underwater image enhancement and restoration methods. Journal of Image and Graphics, 22(3): 273-287
郭继昌, 李重仪, 郭春乐, 陈善继. 2017. 水下图像增强和复原方法研究进展. 中国图象图形学报, 22(3): 273-287)[DOI: 10.11834/jig.20170301]
Guo Y C, Li H Y and Zhuang P X. 2020. Underwater image enhancement using a multiscale dense generative adversarial network. IEEE Journal of Oceanic Engineering, 45(3): 862-870[DOI: 10.1109/JOE.2019.2911447]
Guo Y J, Wu Q, Yuan J J, Hou J C and Lyu W H. 2021. Research progress on underwater optical image processing. Journal of Electronics and Information Technology, 43(2): 426-435
郭银景, 吴琪, 苑娇娇, 侯佳辰, 吕文红. 2021. 水下光学图像处理研究进展. 电子与信息学报, 43(2): 426-435)[DOI: 10.11999/JEIT190803]
Han J L, Shoeiby M, Malthus T, Botha E, Anstee J, Anwar S, Wei R, Armin M A, Li H D and Petersson L. 2021. Underwater image restoration via contrastive learning and a real-world dataset[EB/OL]. [2021-08-13].https://arxiv.org/pdf/2106.10718.pdfhttps://arxiv.org/pdf/2106.10718.pdf
He K M, Sun J and Tang X O. 2009. Single image haze removal using dark channel prior//Proceedings of 2009 IEEE Conference on Computer Vision and Pattern Recognition. Miami, USA: IEEE: 1956-1963[DOI: 10.1109/CVPR.2009.5206515http://dx.doi.org/10.1109/CVPR.2009.5206515]
Hitam M S, Awalludin E A, Yussof W N J H W and Bachok Z. 2013. Mixture contrast limited adaptive histogram equalization for underwater image enhancement//Proceedings of 2013 International Conference on Computer Applications Technology (ICCAT). Sousse, Tunisia: IEEE: 1-5[DOI: 10.1109/ICCAT.2013.6522017http://dx.doi.org/10.1109/ICCAT.2013.6522017]
Hou M J, Liu R S, Fan X and Luo Z X. 2018. Joint residual learning for underwater image enhancement//Proceedings of the 25th IEEE International Conference on Image Processing (ICIP). Athens, Greece: IEEE: 4043-4047[DOI: 10.1109/ICIP.2018.8451209http://dx.doi.org/10.1109/ICIP.2018.8451209]
Huang D M, Wang Y, Song W, Sequeira J and Mavromatis S. 2018. Shallow-water image enhancement using relative global histogram stretching based on adaptive parameter acquisition//Proceedings of the 24th International Conference on MultiMedia Modeling. Bangkok, Thailand: Springer: 453-465[DOI: 10.1007/978-3-319-73603-7_37http://dx.doi.org/10.1007/978-3-319-73603-7_37]
Iqbal K, Odetayo M, James A, Salam R A and Talib A Z H. 2010. Enhancing the low quality images using unsupervised colour correction method//Proceedings of 2010 IEEE International Conference on Systems, Man and Cybernetics. Istanbul, Turkey: IEEE: 1703-1709[DOI: 10.1109/ICSMC.2010.5642311http://dx.doi.org/10.1109/ICSMC.2010.5642311]
Iqbal K, Salam R A, Osman A and Talib A Z. 2007. Underwater image enhancement using an integrated colour model. IAENG International Journal of Computer Science, 34(2): 239-244[DOI: 10.1016/S0031-3203(01)00040-1]
Islam M J, Xia Y Y and Sattar J. 2020. Fast underwater image enhancement for improved visual perception. IEEE Robotics and Automation Letters, 5(2): 3227-3234[DOI: 10.1109/LRA.2020.2974710]
Jaffe J S. 1990. Computer modeling and the design of optimal underwater imaging systems. IEEE Journal of Oceanic Engineering, 15(2): 101-111[DOI: 10.1109/48.50695]
Kar A, Dhara S K, Sen D and Biswas P K. 2021. Zero-shot single image restoration through controlled perturbation of Koschmieder's model//Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Nashville, USA: IEEE: 16200-16210[DOI: 10.1109/CVPR46437.2021.01594http://dx.doi.org/10.1109/CVPR46437.2021.01594]
Li C Y, Anwar S, Hou J H, Cong R M, Guo C L and Ren W Q. 2021. Underwater image enhancement via medium transmission-guided multi-color space embedding. IEEE Transactions on Image Processing, 30: 4985-5000[DOI: 10.1109/TIP.2021.3076367]
Li C Y, Guo J C, Cong R M, Pang YW and Wang B. 2016. Underwater image enhancement by dehazing with minimum information loss and histogram distribution prior. IEEE Transactions on Image Processing, 25(12): 5664-5677[DOI: 10.1109/TIP.2016.2612882]
Li C Y, Guo C L, Ren W Q, Cong R M, Hou J H, Kwong S and Tao D C. 2020. An underwater image enhancement benchmark dataset and beyond. IEEE Transactions on Image Processing, 29: 4376-4389[DOI: 10.1109/TIP.2019.2955241]
Li J, Skinner K A, Eustice R M and Johnson-Roberson M. 2018. WaterGAN: unsupervised generative network to enable real-time color correction of monocular underwater images. IEEE Robotics and Automation Letters, 3(1): 387-394[DOI: 10.1109/LRA.2017.2730363]
Lin W H, Zhong J X, Liu S, Li T and Li G. 2020. ROIMIX: proposal-fusion among multiple images for underwater object detection//Proceedings of 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). Barcelona, Spain: IEEE: 2588-2592[DOI: 10.1109/ICASSP40776.2020.9053829http://dx.doi.org/10.1109/ICASSP40776.2020.9053829]
Liu R S, Fan X, Zhu M, Hou M J and Luo Z X. 2020. Real-world underwater enhancement: challenges, benchmarks, and solutions under natural light. IEEE Transactions on Circuits and Systems for Video Technology, 30(12): 4861-4875[DOI: 10.1109/TCSVT.2019.2963772]
Liu R S, Hou M J, Liu J Y, Fan X and Luo Z X. 2019. Compounded layer-prior unrolling: a unified transmission-based image enhancement framework//Proceedings of 2019 IEEE International Conference on Multimedia and Expo (ICME). Shanghai, China: IEEE: 538-543[DOI: 10.1109/ICME.2019.00099http://dx.doi.org/10.1109/ICME.2019.00099]
Liu X D, Gao Z and Chen B M. 2021. IPMGAN: integrating physical model andgenerative adversarial network for underwater image enhancement. Neurocomputing, 453: 538-551[DOI: 10.1016/j.neucom.2020.07.130]
Lu H M, Li Y J, Zhang L F and Serikawa S. 2015. Contrast enhancement for images in turbid water. Journal of the Optical Society of America A, 32(5): 886-893[DOI: 10.1364/JOSAA.32.000886]
Lu J Y, Li N, Zhang S Y, Yu Z B, Zheng H Y and Zheng B. 2019. Multi-scale adversarial network for underwater image restoration. Optics and Laser Technology, 110: 105-113[DOI: 10.1016/j.optlastec.2018.05.048]
Luo M R, Cui G and Rigg B. 2001. The development of the CIE 2000 colour-difference formula: CIEDE2000. Color Research and Application, 26(5): 340-350[DOI: 10.1002/col.1049]
McGlamery B L. 1980. A computer model for underwater camera systems//Proceedings Volume 0208, Ocean Optics VI. Monterey, USA: SPIE: 221-231[DOI: 10.1117/12.958279http://dx.doi.org/10.1117/12.958279]
Mittal A, Moorthy A K and Bovik A C. 2012. No-reference image quality assessment in the spatial domain. IEEE Transactions on Image Processing, 21(12): 4695-4708[DOI: 10.1109/TIP.2012.2214050]
Panetta K, Gao C and Agaian S. 2016. Human-visual-system-inspired underwater image quality measures. IEEE Journal of Oceanic Engineering, 41(3): 541-551[DOI: 10.1109/JOE.2015.2469915]
Peng Y T, Cao K M and Cosman P C. 2018. Generalization of the dark channel prior for single image restoration. IEEE Transactions on Image Processing, 27(6): 2856-2868[DOI: 10.1109/TIP.2018.2813092]
Peng Y T and Cosman P C. 2017. Underwater image restoration based on image blurriness and light absorption. IEEE Transactions on Image Processing,26(4): 1579-1594[DOI: 10.1109/TIP.2017.2663846]
Risholm P, Thielemann J T, Moore R and Haugholt K H. 2018. A scatter removal technique to enhance underwater range-gated 3D and intensity images//Proceedings of OCEANS 2018 MTS/IEEE Charleston. Charleston, USA: IEEE: 1-6[DOI: 10.1109/OCEANS.2018.8604613http://dx.doi.org/10.1109/OCEANS.2018.8604613]
Schechner Y Y and Karpel N. 2004. Clear underwater vision//Proceedings of 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE: 536-543[DOI: 10.1109/CVPR.2004.1315078http://dx.doi.org/10.1109/CVPR.2004.1315078]
Shin Y S, Cho Y, Pandey G and Kim A. 2016. Estimation of ambient light and transmission map with common convolutional architecture//OCEANS 2016 MTS/IEEE Monterey. Monterey, USA: IEEE: 1-7[DOI: 10.1109/OCEANS.2016.7761342http://dx.doi.org/10.1109/OCEANS.2016.7761342]
Silberman N, Hoiem D, Kohli P and Fergus R. 2012. Indoor segmentation and support inference from RGBD images//Proceedings of the 12th European Conference on Computer Vision—ECCV 2012. Florence, Italy: Springer: 746-760[DOI: 10.1007/978-3-642-33715-4_54http://dx.doi.org/10.1007/978-3-642-33715-4_54]
Sun X, Liu L P, Li Q, Dong J Y, Lima E and Yin R Y. 2019. Deep pixel-to-pixel network for underwater image enhancement and restoration. IET Image Processing, 13(3): 469-474[DOI: 10.1049/iet-ipr.2018.5237]
Tian H, Zhu J P, Tan S W, Zhang Y Y, Zhang Y, Li Y C and Hou X. 2018. Rapid underwater target enhancement method based on polarimetric imaging. Optics and Laser Technology, 108: 515-520[DOI: 10.1016/j.optlastec.2018.07.057]
Vasamsetti S, Mittal N, Neelapu B C and Sardana H K. 2017. Wavelet based perspective on variational enhancement technique for underwater imagery. Ocean Engineering, 141: 88-100[DOI: 10.1016/j.oceaneng.2017.06.012]
Wang K Y, Hu Y, Chen J, Wu X Y, Zhao X and Li Y S. 2019. Underwater image restoration based on a parallel convolutional neural network. Remote Sensing, 11(13): #1591[DOI: 10.3390/rs11131591]
Wang N, Zheng H Y and Zheng B. 2017a. Underwater image restoration via maximum attenuation identification. IEEE Access, 5: 18941-18952[DOI: 10.1109/ACCESS.2017.2753796]
Wang S Q, Ma K D, Yeganeh H, Wang Z and Lin W S. 2015. A patch-structure representation method for quality assessment of contrast changed images. IEEE Signal Processing Letters, 22(12): 2387-2390[DOI: 10.1109/LSP.2015.2487369]
Wang Y, Li N, Li Z Y, Gu Z R, Zheng H Y, Zheng B and Sun M N. 2018. An imaging-inspired no-reference underwater color image quality assessment metric. Computers and Electrical Engineering, 70: 904-913[DOI: 0.1016/j.compeleceng.2017.12.006]
Wang Y, Song W, Fortino G, Qi L Z, Zhang W Q and Liotta A. 2019. An experimental-based review of image enhancement and image restoration methods for underwater imaging. IEEE Access, 7: 140233-140251[DOI: 10.1109/ACCESS.2019.2932130]
Wang Y, Zhang J, Cao Y and Wang Z F. 2017b. A deep CNN method for underwater image enhancement//Proceedings of 2017 IEEE International Conference on Image Processing (ICIP). Beijing,China: IEEE: 1382-1386[DOI: 10.1109/ICIP.2017.8296508http://dx.doi.org/10.1109/ICIP.2017.8296508]
Wang Z, Bovik A C, Sheikh H R and Simoncelli E P. 2004. Image quality assessment: from error visibility to structural similarity. IEEE Transactions on Image Processing, 13(4): 600-612[DOI: 10.1109/TIP.2003.819861]
Xie J, Hou G J, Wang G D and Pan Z K. 2021. Blind restoration for underwater image based on sparse prior of red channel. Laser and Optoelectronics Progress, 58(16): #1610014
谢骏, 侯国家, 王国栋, 潘振宽. 2021. 基于红通道稀疏先验的水下图像盲复原. 激光与光电子学进展, 58(16): #1610014)[DOI: 10.3788/LOP202158.1610014]
Yang M and Sowmya A. 2015. An underwater color image quality evaluation metric. IEEE Transactions on Image Processing, 24(12): 6062-6071[DOI: 10.1109/TIP.2015.2491020]
Zhang S, Wang T, Dong J Y and Yu H. 2017. Underwater image enhancement via extended multi-scale Retinex. Neurocomputing, 245: 1-9[DOI: 10.1016/j.neucom.2017.03.029]
Zuiderveld K. 1994. Contrast limited adaptive histogram equalization// Graphics Gems IV. USA: Academic Press Professional: 474-485[DOI: 10.5555/180895.180940http://dx.doi.org/10.5555/180895.180940]
相关文章
相关作者
相关机构
京公网安备11010802024621