Print

发布时间: 2020-07-16
摘要点击次数:
全文下载次数:
DOI: 10.11834/jig.190482
2020 | Volume 25 | Number 7




    图像处理和编码    




  <<上一篇 




  下一篇>> 





基于透射率修正的湍流模型与动态调整retinex的水下图像增强
expand article info 汤新雨, 李敏, 徐灵丽, 郝真, 张学武
河海大学物联网工程学院, 常州 213022

摘要

目的 为提高水下获取的结构物表面缺陷图像的对比度和清晰度,便于缺陷区域的分割、提取和识别工作,提出了一种基于改进的湍流模型和引导滤波平滑的retinex的图像增强方法。方法 将光照不均的水下图像转换到Lab空间,对亮度空间进行自适应直方图均衡的匀光处理,根据暗通道先验理论估算匀光图像的透射率,结合大气湍流通用模型模拟退化图像,通过调整透射率系数获得退化图像。采用维纳滤波过滤图像噪声,将滤波后的图像作为导向图,利用导向滤波细化获得边缘保持的图像。根据3σ准则对3通道多尺度retinex(multi-scale retinex,MSR)的反射分量进行色彩矫正,获取最终增强后的水下结构物表面缺陷图像。结果 选取多组在不同湍流环境下采集的图像为研究对象,采用本文提出的方法进行实验,并与经典的暗通道算法、直方图均衡算法以及单尺度retinex算法对比,使用信噪比、信息熵、标准差和平均梯度等指标进行评估。实验结果表明,本文方法的信息熵、标准差相较直方图均衡算法和单尺度retinex分别提高了11.7%和25.6%,分割准确率上升了3.1%。从主观效果上看,本文算法图像细节更为丰富,视觉效果自然。结论 本文算法改善了退化模型的自适应问题,在信息熵、标准差、平均梯度等综合指标上均有优异表现,与暗通道先验方法相比,信噪比、平均梯度大幅提升,同时实现了缺陷的边缘保持效果,为下阶段的图像处理提供了良好的信息源。

关键词

水下图像增强; 湍流模型; 透射率; 维纳滤波; 多尺度retinex; 色彩校正

Underwater image enhancement based on turbulence model corrected by transmittance and dynamically adjusted retinex
expand article info Tang Xinyu, Li Min, Xu Lingli, Hao Zhen, Zhang Xuewu
College of Internet of Things Engineering, Hohai University, Changzhou 213022, China
Supported by: National Natural Science Foundation of China(61671202, 61701169, 61273170); National Key Research and Development Program of China(2018YFC0407101, 2016YFC0401606)

Abstract

Objective The attenuation of light is typically serious even in the purest water after filtration. Experiments show that the attenuation of water is caused by two unrelated physical processes, namely, absorption and scattering. Water has obvious selectivity for light absorption, and the absorption abilities of different spectral regions vary. Such variation leads to the loss of light energy, which makes underwater imaging difficult with such phenomena as low definition and color distortion. The scattering of light by water can be divided into forward and backward scattering. The suspended particles in water cause the attenuation of light in the forward and reverse directions of propagation, which limits the detection range and distance of underwater optical imaging. This limitation results in a decrease in image contrast, blurred details, and degradation. A series of degradation problems has weakened the detection rate and accuracy of defects. Therefore, the denoising and enhancement of images with surface defects of underwater structures are significant. An improved image enhancement algorithm based on turbulence model and multiscale retinex (MSR) is proposed to improve the contrast and sharpness of surface defect images acquired underwater and facilitate the subsequent segmentation, extraction, and recognition of the defect region. The proposed method combines physical model with the nonphysical model method. Thus, this method improves the considerably simple model parameters and poor versatility. The color cast of the enhanced image is considered, and the image noise is suppressed. Method An underwater image with uneven illumination is converted from RGB space to Lab space, and histogram equalization (HE) is performed on the luminance space. After the incident light is reflected from the structure surface, it will be affected by the suspended particles from the water before it reaches the imaging device, and the scattering phenomenon will generate noise. The absorption of the spectrum by the water will attenuate the light intensity, and this condition will result in low image contrast. The process is similar to the imaging model of a foggy-degraded image. Thus, the reconstructed method of the degraded image can be used to process the blurred underwater image. The light intensity distribution in the imaged scene after homogenization is similar, which can be approximated as a fixed value. The transmittance of the homogenized image can be estimated in accordance with dark channel prior theory. It describes the medium transmission that is unscattered and reaches the light portion of the imaging system, and it is also the degree of blur at each pixel. It reflects the degree of transparency of the light source components in the scene, which indicates the extent to which the image is affected by the scattering of water. In consideration of the same degraded underwater images and remote sensing images, the two exhibit similarities in optical properties, fluid media, and external forms. The atmospheric turbulence model can accordingly be applied to simulate the degradation process of underwater images. The transmittance obtained above is combined with a general model of atmospheric turbulence to simulate an underwater degradation image by adjusting the transmittance coefficient. The image noise is filtered by Wiener filter, and the filtered image is used as a guide image. An edge-preserved image is obtained using a guide filter to refine. Image enhancement is performed in accordance with retinex theory. The MSR result is color-corrected on the basis of the 3σ criterion to obtain an enhanced underwater image. Multiple images collected under different turbulent environments are selected as the research object. The method proposed in this study is used and compared with classic methods, such as the dark channel, HE, and single-scale retinex (SSR) algorithms. The indicators of signal-to-noise ratio (SNR), peak SNR, information entropy (IE), standard deviation (SD), and average gradient (AG) are evaluated. Result Experimental results show that the IE and SD of the proposed algorithm are 11.7% and 25.6% higher than those of HE algorithm and SSR, respectively. The AG is 31.2% higher than that of HE algorithm, and the segmentation accuracy is increased by 3.1%. The oversegmentation rate is the lowest among similar algorithms. From the perspective of subjective visual effects, the image after restoration by the dark channel algorithm is rich in color and prominent in detail, but its gray value is mostly distributed in the low-gray area. Such distribution results in a low image recognition rate and an unsatisfactory segmentation effect. The visual effect of HE algorithm is the closest to the original image, but the problem that the gray scale distribution is excessively concentrated remains. The segmentation accuracy is slightly improved compared with that of the dark channel algorithm. The image after SSR shows a color cast phenomenon, and the overall gray value of the image after the dynamically adjusted MSR decreases. The range of gray scale is enlarged, and the color cast effect is improved. The color and detail of the image are the richest, and the visual effect is the most natural among all the tested algorithms. Conclusion On the basis of the analysis of an underwater image degradation model and the difference among pixels, the image transmittance is estimated, and the gray scale distribution of the enhanced image is expanded by the 3σ criterion. An enhanced underwater image with high contrast, high definition, and balanced color is obtained. The algorithm improves the adaptive problem of the degenerate model with excellent performance in comprehensive indicators, such as IE, SD, and AG. Compared with the dark channel prior method, the proposed method exhibits greatly improved SNR and AG. The edge information of the image is preserved, which provides a good information source for image segmentation and recognition in the next stage.

Key words

underwater image enhancement; turbulence model; transmittance; Wiener filtering; multi-scale retinex(MSR); color correction

0 引言

通常情况下,即便是过滤后最纯净的水,对光的衰减也十分严重。实验表明,光的衰减是由两个互不相关的物理过程即吸收和散射引起,水对光的吸收存在明显的选择性,在不同的光谱区域吸收程度也不相同,由此导致的光能量损失使得水下成像变得困难,出现清晰度低和颜色失真等现象。同时,水对光的散射现象分为前向和后向散射,水中的悬浮颗粒造成光在正向和反向传播方向上的衰减,限制了水下光学成像的探测范围和距离,从而引起图像对比度下降、细节模糊和降质等一系列退化问题,降低了缺陷的检出率和目标检测准确率。因此,开展针对水下结构物表面缺陷图像的复原和增强工作具有重要意义。

水下图像处理方法大致可分为基于图像增强的方法和基于物理模型的方法。基于图像增强的方法主要针对水下图像质量下降的现象,选取相应的图像增强技术,改善图像质量(郭继昌等,2017)。包括基于空域增强的方法,如直方图均衡(histogram equalization, HE)算法(许廷发等,2017)以及不同颜色模型下的自适应直方图拉伸(黄冬梅等,2018);基于频域增强的方法,如àTrous算法(黄允浒等,2018)、同态滤波(王永鑫等,2018)、小波变换(王红茹等,2017)以及小波颜色校正(Singh等,2015);基于颜色恒常性理论的方法,如retinex算法(Jobson等,1997a)、色彩补偿(代成刚等,2018)等。这类方法可以根据现有的图像增强技术灵活构建处理方案,但存在一定弊端,HE算法通过分散灰度值、修改直方图分布增强图像对比度(Wan等,2018),虽然处理速度快,但是增强后的图像容易失真,图像亮度分布不均,主观评价差(张驰等,2019)。retinex算法是从图像中估计出光照分量,得到反射图像,主要包括单尺度retinex(single-scale retinex, SSR)、多尺度retinex(multi-scale retinex, MSR)以及带颜色恢复的多尺度retinex(multi-scale retinex with color restoration, MSRCR)算法(谢娜,2017)等,但由于没有考虑水下图像降质的原因,增强后的图像往往存在偏色现象,无法反映图像的真实原貌。

基于物理模型的方法是从水下图像的成像机理出发,通过研究水体对光线的散射及吸收作用,建立适当的水下成像模型,反演出未降质的图像。若模型建立合理,能得到接近真实图像的复原图像。张赫等人(2010)通过分析水下图像退化过程,提出利用大气湍流模型获取水下退化函数的方法,并以频域滤波算法为基础实现了对水下退化图像的复原,该算法具有良好的抗噪性能,能够在一定程度上提升水下图像的质量,但也受限于退化模型的参数选择问题,面对不同湍流程度的成像环境需要不断调整湍流参数,无法做到与实际图像退化程度的准确契合;Wen等人(2013)通过探索大气与水中光衰减的差异,推导出一种新的水下光学模型来描述真实物理过程中水下图像的形成,提出一种有效方法来估计水下光学模型中的背景光,该算法可以很好地处理从浑浊水域捕获的深海图像,但同时也面临着算法适应性和灵活性欠缺的问题。李昌利等人(2019)提出了一种多通道均衡化的水下图像增强算法,对HIS空间的亮度分量利用McCann retinex算法在4个方向(纵横)实现增强,该算法在改善水下图像照度信息的同时,保留了饱和度和色度信息,但也存在着灰度拉伸带来的红色分量过度增强的缺陷;李向春等人(2019)提出一种基于透射率优化和颜色修正的水下图像增强方法,能够有效去除由后向散射引起的模糊,但受图像过曝区域的影响,导致水下背景光估计值偏大。范新南等人(2018)提出一种基于结构相似性的水下偏振图像复原方法,有效解决了水下偏振图像存在的雾状模糊和场景细节不明显问题,但该算法对图像的获取要求较高,且存在参数迭代过程,降低了算法运行效率。

针对水下图像增强算法中存在的适应性差、色彩过矫正以及图像过增强等问题,本文提出一种结合改进的湍流模型与多尺度retinex算法的增强算法。根据暗通道先验理论估计图像的透射率,建立以透射率为基础的湍流退化模型,通过维纳滤波过滤图像中的噪声,获得导向图,并对匀光图像进行导向滤波,从而保留原图像的边缘信息。利用MSR进行图像的色彩恢复,获得最终的增强图像。最后通过实验与暗通道算法、HE算法以及SSR算法的处理效果进行对比,验证了本文算法的可行性和有效性。

1 本文方法

1.1 本文流程

水下图像增强算法大多存在不同程度的自适应差、复杂度高以及鲁棒性差等问题。为此,本文提出一种结合物理模型与非物理模型的水下图像增强算法,既充分考虑增强图像的偏色情况,抑制图像噪声,又改善大多模型存在的参数简单、通用性差的问题。主要步骤如下:1)水下图像$\mathit{\boldsymbol{I}}$转换到Lab空间,对亮度分量作光照补偿,获得匀光后的图像${\mathit{\boldsymbol{I}}_1}$;2)进行基于暗通道先验理论的水下图像透射率$\bar t\left(x \right)$估计,结合湍流模型获得模拟的退化图像,维纳滤波抑制图像噪声;3)将去噪后图像$\mathit{\boldsymbol{w}}$作为导向图,对原图进行导向滤波,得到保留边缘信息的复原图像$\mathit{\boldsymbol{S}}$;4)使用MSR对复原图像进行颜色恢复,最终获得增强图像$\mathit{\boldsymbol{r}}$。结构流程图如图 1所示。

图 1 本文算法流程图
Fig. 1 Algorithm flow chart

1.2 匀光预处理

在自然光无法到达的深水区,往往需要人工光源辅助照明,光在水下传播时,经过水体散射的光呈喇叭状展开,且中间的光子密度大,向四周逐渐减小。光照不均是水下图像普遍存在的通病,而光照补偿作为水下图像预处理的第1步能够调整光场使之均匀分布。后文在透射率估计中将光场视为固定值。

Lab颜色空间由一个亮度通道L和两个颜色通道a、b组成,其中L分量与人类视觉的亮度感知有着密切的匹配,控制图像的整体明暗度。直接对图像3通道采用HE算法容易导致图像色彩失真,而在Lab空间中对L通道进行限制对比度的直方图均衡(contrast limited adaptive histogram equalization, CLAHE)既能增强图像的对比度,改善非均匀光场分布,又不影响图像的色相。同时CLAHE用预先定义的阈值来裁剪直方图,克服了HE算法过度放大噪声的问题。

考虑到清晰的水下图像通常呈现瑞利分布,本文采用瑞利分布函数对输入图像重分布,匀光后的图像过饱和区域明显减少,对比度显著提高,图像的视觉效果更自然。图 2图 3展示了匀光前后图像及灰度图对比、匀光前后图像及其灰度直方图分布对比。

图 2 匀光前后图像及灰度图对比
Fig. 2 Comparison of the distribution of images before and after homogenization ((a) original image; (b) uniform image; (c) original grayscale; (d) uniform grayscale)
图 3 匀光前后图像的灰度直方图分布对比
Fig. 3 Comparison of the gray histograms of images before and after homogenization((a) original histogram; (b) uniform histogram)

1.3 基于改进的湍流模型的水下图像去噪

入射光线从结构物表面反射后,在到达成像设备之前,会受到来自水中悬浮粒子的影响而发生散射现象使图像产生噪声。另一方面水体对光谱的吸收会使光强度产生衰减,造成图像对比度偏低。这与雾天降质图像的成像模型非常相似,因此可以利用霾降质图像的复原方法处理模糊的水下图像(Yang和Sowmya,2015)。计算机视觉中常用于描述雾天图像的模型为

$ {I_1}(x) = J(x)\bar t(x) + \mathit{\boldsymbol{A}}(1 - \bar t(x)) $ (1)

应用到水下退化图像时,${\mathit{\boldsymbol{I}}_1}$为1.2节匀光后的图像,$\mathit{\boldsymbol{J}}$为要恢复的复原图像,$\mathit{\boldsymbol{A}}$为人工光源成分。由1.2节可知,匀光后的图像场景中各处光强近似相同,因此光源成分$\mathit{\boldsymbol{A}}$可近似为固定值。取图像暗通道中亮度前0.1%的点,在匀光图像中找到对应的像素,其中所有通道的最大值即为$\mathit{\boldsymbol{A}}$的近似值。$\bar t\left(x \right)$为水下图像的透射率,描述的是未散射并到达成像系统的光部分的介质传输,即每个像素处的模糊程度,反映了场景中光源成分的通透程度,表示图像受水体散射影响的退化程度。

暗通道${\mathit{\boldsymbol{J}}^{{\rm{dark}}}}$定义为

$ {J^{{\rm{ dark }}}}(x) = \mathop {{\rm{min}}}\limits_{y{\kern 1pt} \in {\kern 1pt} \mathit{\boldsymbol{ \boldsymbol{\varOmega} }}(x)} (\mathop {{\rm{min}}}\limits_{c \in \{ r,g,b\} } {J^c}(y)) = 0 $ (2)

式中,$c$为图像的RGB三通道,则图像的透射率$\bar t\left(x \right)$

$ \bar t(x) = 1 - \mathop {{\rm{min}}}\limits_{y{\kern 1pt} \in {\kern 1pt} \mathit{\boldsymbol{ \boldsymbol{\varOmega} }}(x)} \left( {\mathop {{\rm{min}}}\limits_c \frac{{{I^c}(y)}}{{{\mathit{\boldsymbol{A}}^c}}}} \right) $ (3)

Hufnagel和Stanley(1964)提出一种与大气湍流的时间平均降质图像相对应的光学传递函数,旨在解决大气湍流对遥感技术成像的干扰,考虑同样退化严重的水下图像和遥感图像,二者在光学特性、流体介质和外在形式上均呈现出一定的相似性,可将大气湍流模型应用于模拟水下图像的退化过程,该模型的通用公式为

$ T(u,v) = {{\rm{e}}^{ - k{{({u^2} + {v^2})}^{5/6}}}} $ (4)

式中,$\left({u, v} \right)$为水平和垂直方向上的频域变量,$k$为湍流系数,与湍流强度的大小有关,控制图像的降质程度。如前所述,图像透射率可依据式(3)求出,则依据透射率可求得图像深度分布$\mathit{\boldsymbol{D}}$,具体为

$ D(i,j) = - {\rm{lg}}(1 - {\bar t_{(i,j)}}(x)) $ (5)

式中,$\left({i, j} \right)$为像素坐标。

图 4展示的是匀光图像及其深度图像。图 4(b)是根据式(5)所求的匀光后图像的深度图。对比图 3(a)(b)两幅图像,可以看出深度图中高亮区对应的是原图中光照散射较为严重的区域,即易产生过曝的区域。由式(4)可以进一步看出,透射率$\bar t\left(x \right)$越大的区域,深度图像亮度越大,$\bar t\left(x \right)$分布反映了图像像素之间退化程度的比例关系。

图 4 匀光图像及其深度图像
Fig. 4 Uniform image and its depth image ((a) uniform image; (b) depth image)

通常情况下,$k$=0.001时为中等湍流,$k$≤0.000 25时为微弱湍流(Huang等,2019),式(3)计算的透射率$\bar t\left(x \right)$值分布在[0.1, 0.9]之间,本文采用$\alpha \cdot \bar t\left(x \right)$替代常数$k$($\alpha$为透射率系数)来表示各个像素处的湍流强度大小,并将透射率系数$\alpha$控制在[0.001, 0.01]之间,控制湍流强度为中等湍流,同时保证退化图像满足原图像的退化比例关系,这里选取$\alpha$=0.001 5。

为得到中心化的传递函数,便于滤波,即保证频谱图中心为低频,四周为高频分量,则式(4)可表示为

$ T(u,v) = {{\rm{e}}^{ - \alpha \cdot \bar t(x){{\left[ {{{\left( {u - \frac{m}{2}} \right)}^2} + {{\left( {v - \frac{n}{2}} \right)}^2}} \right]}^{5/6}}}} $ (6)

式中,$\left({m, n} \right)$为图像的高度和宽度,$\alpha$为透射率系数,$\bar t\left(x \right)$为透射率。将改进后的湍流模型与匀光图像做卷积,并采用维纳滤波进行去噪处理,则

$ \hat G(u,v) = \left[ {\frac{1}{{T(u,v)}}\frac{{|T(u,v){|^2}}}{{|T(u,v){|^2} + Q}}} \right]G(u,v) $ (7)

式中,$\mathit{\boldsymbol{G}}$为原始水下图像频谱,$\mathit{\boldsymbol{T}}$为传递函数,即改进的湍流模型,${\mathit{\boldsymbol{\hat G}}}$为复原图像的频谱,$Q$为图像信噪比的倒数,由于噪声功率谱未知,通常$Q$的选取根据实验决定,适当增加$Q$的值,复原效果增强(廖秋香等,2019),但只适合在图像要求不高的情况下使用。廖秋香等人(2019)使用噪声和图像的自相关函数代替功率谱,本文通过图像空域中的平坦区域的灰度方差和均值来估计$Q$。该区域的信噪比倒数为

$ Q = \frac{{\sigma _p^2(i,j)}}{{\mu _p^2(i,j)}} $ (8)

式中,${\sigma _p}\left({i, j} \right)$为以像素$p\left({i, j} \right)$为中心的区域的灰度方差,${\mu _p}\left({i, j} \right)$为灰度均值。确定了$Q$值大小后,通过傅里叶反变换可以求出滤波后的图像$\mathit{\boldsymbol{w}}$

滤波后的图像噪声减弱,但图像细节也退化严重,采用具有边缘保持效果的导向滤波器,以滤波图像$\mathit{\boldsymbol{w}}$为引导图对匀光图像${\mathit{\boldsymbol{I}}_1}$细化,能够获得具有去噪和保边效果的图像$\mathit{\boldsymbol{S}}$,作为下阶段增强的有效信息源。

图 5图 6分别是维纳滤波复原退化图像及其梯度图分布、改进前后图像梯度分布矢量图对比。结合图 5图 6可以看出,原模型通过统一湍流系数来模拟退化效果,复原后的图像加强了边缘信息,但干扰部分也被放大,而改进后的湍流模型因为考虑到像素间差异,滤波后更好地还原了图像细节,同时过滤了部分噪声,表 1给出了3种算法在信噪比(signal-to-noise ratio,SNR)、峰值信噪比(peak signal-to-noise ratio,PSNR)和平均梯度(average gradient, AG)等数据上的对比,改进后的湍流模型各项指标均优于原模型,能够根据图像区域间不同的退化程度实现有针对性的复原,从而提高了模型的适应性。

图 5 维纳滤波复原退化图像及其梯度图分布
Fig. 5 Degraded image restored by Wiener filtering and its gradient map((a) uniform image; (b) restored image before improvement; (c) restored image after improvement; (d) gradient map of uniform image; (e) gradient map of restored image before improvement; (f) gradient map of restored image after improvement)
图 6 改进前后图像梯度分布矢量图和局部放大图对比
Fig. 6 Comparison between images of gradient distribution vector and local amplification from it before and after improvement ((a) uniform image; (b) before improvement; (c) after improvement)

表 1 各算法图像复原的视觉质量评价参数
Table 1 Visual quality evaluation parameters of different algorithms for image restoration

下载CSV
算法SNR/dBPSNR/dBAG
退化复原退化复原退化复原
原模型26.749 833.058 234.762 341.001 20.250 10.314 8
改进后31.595 936.829 737.854 643.067 90.327 00.443 9
暗通道1.535 912.315 51.440 4
注:加粗字体表示最优结果,“—”表示没有退化图像。

1.4 基于retinex理论的水下图像增强

retinex理论是基于人类视觉感知的图像处理算法,认为物体的颜色是由物体对三原色光线的反射能力决定的,因此物体的色彩具有一致性,由物体本身决定,不受光照不均的影响,即颜色恒常性,retinex模型将一幅图像视作照射分量$\mathit{\boldsymbol{L}}$与反射分量$\mathit{\boldsymbol{R}}$乘积的形式(王峰萍等,2017),目的就是从原始图像$\mathit{\boldsymbol{S}}$中估计出光照分量,从而求解出反射分量,以改善视觉效果。即

$ {S_c}(x,y) = {L_c}(x,y) \cdot {R_c}(x,y) $ (9)

式中,$c$$\mathit{\boldsymbol{S}}$的RGB三通道,对式(9)两边同时取对数,则将乘积的形式转化为和的形式,即

$ {\rm{lg}}({R_c}(x,y)) = {\rm{lg}}({S_c}(x,y)) - {\rm{lg}}({L_c}(x,y)) $ (10)

观察式(10)可以发现,retinex算法的关键就是从原始图像中估计出照射分量,在Land(1986)提出的单尺度retinex(SSR)算法中,通常选取高斯滤波函数与原图像卷积的方式来估计照射分量,即

$ {L_c}(x,y) = F(x,y) * {S_c}(x,y) $ (11)

式中,$F(x, y) = f\exp \left({ - \frac{{{x^2} + {y^2}}}{{{\sigma ^2}}}} \right)$$f$为归一化因子,满足$\iint{f}(x, y)\text{d}x\text{d}y=1$$\sigma $为高斯滤波函数的标准偏差。SSR的效果与$\sigma $的选取密切相关,小尺寸的$\sigma $可以提升图像暗部亮度,但可能导致过度补偿使图像的动态范围过小,也会出现光晕(halo)现象;而大尺寸的$\sigma $可以提升图像动态范围,但可能使模型丧失局部特性,导致图像阴影处无法得到补偿(宋巍等,2018)。因此,针对SSR可能出现的各种问题,Jobson等人(1997b)提出多尺度加权平均的retinex(MSR)算法,即

$ \begin{array}{*{20}{c}} {{r_c}(x,y) = \sum\limits_{i = 1}^n {{\omega _i}} ({\rm{lg}}({S_c}(x,y)) - }\\ {{\rm{lg}}({S_c}(x,y) * {F_i}(x,y)))} \end{array} $ (12)

式中,$n$为尺度参数的个数,通常取$n$=3,即使用3个不同尺度的高斯滤波器对原始图像做滤波,${\omega _i}$是第$i$个尺度在做加权时的权重系数,需满足

$ {\sum\limits_{i = 1}^n {{\omega _i}} }=1 $ (13)

大量实验发现,${\omega _i}$取平均即${\omega _1}$=${\omega _2}$=${\omega _3}$=1/3时适用于大部分低照度图像,且运算简单。对$\mathit{\boldsymbol{S}}$的三通道图像分别求出$r\left({x, y} \right)$后,则可以求出反射分量,得到增强图像,即

$ {R_c}(x,y) = {\rm{exp}}({r_c}(x,y)) $ (14)

由式(14)获得的增强图像色彩丰富,细节增强,但因为图像中存在噪声,使得图像的局部细节色彩失真,经过指数变换后的图像容易出现偏色现象,视觉效果变差,导致初步分割的效果不理想,因此需要对灰度分布重新调整,根据正态分布的3$\sigma $法则,正态曲线横轴区间内的面积反映了变量值落在该区间的概率,横轴区间($\mu $-2$\sigma $$\mu $+2$\sigma $)下的面积约为95.45%,($\mu $-3$\sigma $$\mu $+3$\sigma $)为99.73%,本文将3$\sigma $区间作为灰度值的取值区间,通过对$r\left({x, y} \right)$作灰度范围拉伸处理来获得增强图像,具体为

$ \begin{array}{l} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} r(x,y) = \\ \left\{ {\begin{array}{*{20}{l}} 0&{r(x,y) < {r_{\rm{L}}}}\\ {\frac{{r(x,y) - {\rm{min}}(r(x,y))}}{{{\rm{max}}(r(x,y)) - {\rm{min}}(r(x,y))}}}&{{r_{\rm{L}}} \le r(x,y) \le {r_{\rm{H}}}}\\ {255}&{r(x,y) > {r_{\rm{H}}}} \end{array}} \right. \end{array} $ (15)

式中,取图像灰度均值为${\mu _r}$,灰度标准差为${\sigma _r} $,动态调整参数$d$=3,则有

$ \left\{ {\begin{array}{*{20}{l}} {{r_{\rm{L}}} = {\mu _r} - d \cdot {\sigma _r}}\\ {{r_{\rm{H}}} = {\mu _r} + d \cdot {\sigma _r}} \end{array}} \right. $ (16)

2 实验结果与分析

实验基于MATLAB R2018b平台,运行环境为Lenovo PC端,Core i7处理器,3.4 GHz主频,8 GB内存。实验图像选取在各深度采集的不同流速下地涵的表面缺陷图像,缺陷类型包括结构缝开裂、填料脱空以及砌石破裂等,摄像机分辨率为3 840×2 160像素,摄像模式为24 p,图像大小为1 920×740像素。

首先对图像进行匀光处理,对匀光后的图像使用改进后的湍流模型进行复原,进而对复原图像进行增强。当前大多数水下复原方法都基于图像成像模型和暗通道先验或它的变体(谢凤英等,2019),本文选用暗通道(dark channel)去雾、HE、SSR等算法进行对比。

图 7图 8分别为同一幅复原图像在不同算法下的增强效果对比及其对应的灰度直方图分布,图 9为对各增强结果采用最大类间方差算法(OTSU)的初步分割结果。最大类间方差算法基于图像的灰度特性,图像背景与目标之间的类间方差越大,分割结果越准确。综合3类图像可以看出暗通道算法复原后的图像色彩丰富,细节突出,但因其灰度值大多分布在低灰度区,导致图像识别率低,分割效果不理想;HE算法的视觉效果与原图最为接近,但也存在灰度分布过于集中的问题,分割准确率较暗通道算法稍有提升;SSR后的图像出现了偏色现象,而经过动态调整后的MSR图像整体灰度值有所下降,同时灰度分布范围扩大,偏色效果得到了改善。

图 7 各算法增强效果
Fig. 7 Enhancement effects of different algorithms ((a) original image; (b) restored image; (c) dark channel prior; (d) HE; (e) SSR; (f) ours)
图 8 各算法增强图像直方图
Fig. 8 Histograms of different enhancement algorithms ((a) original image; (b) restored image; (c) dark channel prior; (d) HE; (e) SSR; (f) ours)
图 9 各算法采用OTSU的分割结果
Fig. 9 Segmentation results of different algorithms with OSTU ((a) original image; (b) restored image; (c) dark channel prior; (d) HE; (e) SSR; (f) ours)

为更客观地比较不同算法的优劣性,本文选取信噪比(SNR)、峰值信噪比(PSNR)、信息熵(information entropy,IE)、标准差(standard deviation,SD)以及平均梯度(average gradient,AG)等指标衡量图像的增强效果。其中,标准差反映了图像的对比度和清晰度,标准差越大,图像的视觉效果越好;信息熵是衡量图像信息量大小的指标,信息熵越大图像的细节信息越丰富,信息熵的定义为

$ E = \sum\limits_{i = 0}^{255} p (i){\rm{lg}}{\kern 1pt} {\kern 1pt} p(i) $ (17)

式中,$p\left(i \right)$是灰度级为$i$的像素在图像中所占的比例。平均梯度是反映图像模糊程度的指标(温海滨等,2016),即

$ \left\{ {\begin{array}{*{20}{l}} {{g_x}(i,j) = I(i,j) - I(i + 1,j)}\\ {{g_y}(i,j) = I(i,j) - I(i,j + 1)}\\ {g = \frac{1}{{(m - 1)(n - 1)}} \cdot }\\ {\sum\limits_{i = 1}^{m - 1} {\sum\limits_{j = 1}^{n - 1} {\sqrt {\frac{1}{2}[g_x^2(i,j) + g_y^2(i,j)]} } } } \end{array}} \right. $ (18)

考虑到大多分割算法都会出现过分割的现象,本文引入衡量指标分割准确率(segmentation accuracy,SA)和过分割率(over-segmentation rate,OR)考察各算法初步分割的效果。表 2图 7中不同算法增强效果的各项指标,可以看出本文的增强算法在IE、SD、AG等指标上均有较优表现,分割准确率SA接近90%,过分割率OR为同类算法中最低。综合视觉效果、分割结果及数据表现,本文算法具有一定的优越性。

表 2 不同算法增强效果的视觉质量评价参数
Table 2 Visual quality evaluation parameters of enhancement effects of different algorithms

下载CSV
算法SNR/dBPSNR/dBIESDAGSA/%OR/%
暗通道先验1.535 912.315 57.146 038.041 61.440 443.5836.07
HE16.006 622.751 06.733 425.900 91.046 581.3715.70
SSR11.840 617.008 77.177 736.502 41.430 484.3313.55
本文11.463 416.373 07.392 245.748 21.141 989.489.52
注:加粗字体表示最优效果。

图 10列出了5幅不同程度湍流环境下采集到的退化图像(自上至下编号依次为#A,#B,#C,#D,#E)的增强结果,各算法评价指标如文后附表所示。值得注意的是,由于HE算法处理后的图像整体会出现偏色现象,而信噪比指标并未考虑到人眼特性,所以会出现指标相对较高,但与实际人眼观察效果不一致的情况。

图 10 不同图像的增强效果
Fig. 10 Enhancement results of different images ((a) original images; (b) restored images; (c) dark channel prior; (d) HE; (e) SSR; (f) ours; (g) our segmentation results)

表   图 10中不同图像的视觉质量评价参数
Table   Visual quality evaluation parameters of different images in Fig. 10

下载CSV
图像编号算法SNR/dBPSNR/dBIESDAGSA/%
#A暗通道先验4.438 413.809 17.408 046.592 11.041 786.52
HE16.942 123.905 96.718 925.887 00.802 178.71
SSR12.634 518.828 36.983 832.119 31.201 097.41
本文12.229 917.685 17.377 442.964 41.483 698.52
#B暗通道先验7.087 316.272 97.363 743.189 01.201 827.55
HE16.956 724.135 86.793 727.021 80.972 892.27
SSR11.683 817.897 77.182 835.721 51.261 991.55
本文10.261 215.435 07.434 244.498 41.608 095.75
#C暗通道先验5.038 713.983 57.510 950.792 41.099 249.28
HE17.129 124.059 06.720 325.770 60.800 769.54
SSR12.418 218.282 06.970 232.156 41.237 982.10
本文11.964 317.433 87.332 042.844 91.559 687.05
#D暗通道先验4.209 714.780 97.177 539.727 51.101 227.33
HE19.483 826.968 06.530 922.953 70.694 453.38
SSR11.351 417.323 67.311 139.123 81.294 581.07
本文10.610 215.981 67.442 743.473 11.391 383.25
#E暗通道先验5.289 615.955 57.147 538.492 01.182 589.56
HE18.656 726.394 46.383 720.418 00.594 598.69
SSR11.241 617.860 57.107 133.877 61.118 498.87
本文9.598 015.162 07.416 842.404 61.263 299.70
注:加粗字体表示各图像的各项指标中的最优值。

3 结论

本文针对水下图像存在的因光的散射和吸收产生的严重退化现象展开研究,提出了基于改进的湍流模型的水下模型模拟退化图像,使用维纳滤波过滤图像,获得导向图,并引入导向滤波对原图像进行细化,去除噪声干扰的同时较好地还原了图像细节信息,而后结合动态调整的MSR算法对增强图像实现色彩校正,较好地实现了水下图像的增强。实验结果表明,本文算法在一定程度上增强了图像的对比度、清晰度和色彩均衡性,为后续的分割工作提供了良好的信息源。

本文算法从透射率出发,结合水体散射和吸收对图像退化的影响,引入的新参数充分考虑到场景中各处的像素间差异,提高了退化模型的参数适应性,但也增加了算法的计算量,相比其他算法,运行时间有待缩短,将通过简化计算流程来提高运算效率。下一步的研究重点将集中在增强水下图像的目标轮廓检测与识别上。

参考文献

  • Dai C G, Lin M X, Wang Z, Zhang D, Guan Z G. 2018. Color compensation based on bright channel and fusion for underwater image enhancement. Acta Optica Sinica, 38(11): 86-95 (代成刚, 林明星, 王震, 张东, 管志光. 2018. 基于亮通道色彩补偿与融合的水下图像增强. 光学学报, 38(11): 86-95) [DOI:10.3788/AOS201838.1110003]
  • Fan X N, Chen J Y, Zhang X W, Shi P F, Zhang Z. 2018. Underwater polarized images restoration algorithm based on structural similarity. Journal of Image and Graphics, 23(7): 1033-1041 (范新南, 陈建跃, 张学武, 史朋飞, 张卓. 2018. 结构相似性的水下偏振图像复原. 中国图象图形学报, 23(7): 1033-1041) [DOI:10.11834/jig.180009]
  • Guo J C, Li C Y, Guo C L, Chen S J. 2017. Research progress of underwater image enhancement and restoration methods. Journal of Image and Graphics, 22(3): 273-287 (郭继昌, 李重仪, 郭春乐, 陈善继. 2017. 水下图像增强和复原方法研究进展. 中国图像图形学报, 22(3): 273-287) [DOI:10.11834/jig.20170301]
  • Huang D M, Wang Y, Song W, Wang Z H, Du Y L. 2018. Underwater image enhancement method using adaptive histogram stretching in different color models. Journal of Image and Graphics, 23(5): 640-651 (黄冬梅, 王龑, 宋巍, 王振华, 杜艳玲. 2018. 不同颜色模型下自适应直方图拉伸的水下图像增强. 中国图象图形学报, 23(5): 640-651) [DOI:10.11834/jig.170610]
  • Huang H, Zhou H, Yang X, Zhang L, Qi L, Zang A Y. 2019. Faster R-CNN for marine organisms detection and recognition using data augmentation. Neurocomputing, 337: 372-384 [DOI:10.1016/j.neucom.2019.01.084]
  • Huang Y H, Han Y, Turghunjan·Abdukirim. 2018. An underwater image enhancement algorithm based on àTrous algorithm. Electronic Design Engineering, 26(16): 6-11, 18 (黄允浒, 韩燕, 吐尔洪江·阿布都克力木. 2018. 一种基于àTrous算法的水下图像增强算法. 电子设计工程, 26(16): 6-11, 18) [DOI:10.3969/j.issn.1674-6236.2018.16.002]
  • Hufnagel R E, Stanley N R. 1964. Modulation transter function associated with image transmission through turbulent media. Journal of the Optical Society of America, 54(1): 52-60 [DOI:10.1364/JOSA.54.000052]
  • Jobson D J, Rahman Z, Woodell G A. 1997a. Properties and performance of a center/surround retinex. IEEE Transactions on Image Processing, 6(3): 451-462 [DOI:10.1109/83.557356]
  • Jobson D J, Rahman Z, Woodell G A. 1997b. A multiscale retinex for bridging the gap between color images and the human observation of scenes. IEEE Transactions on Image Processing, 6(7): 965-976 [DOI:10.1109/83.597272]
  • Land E H. 1986. An alternative technique for the Computation of the designator in the Retinex theory of color vision. Proceedings of the National Academy of Sciences of the United States of America, 83(10): 3078-3080 [DOI:10.1073/pnas.83.10.3078]
  • Li C L, Sun Y W, Yan J W, Fan T H. 2019. Underwater color image enhancement algorithm based on multi-channel equalization. Journal of Huazhong University of Science and Technology (Nature Science Edition), 47(6): 1-5, 29 (李昌利, 孙亚伟, 闫敬文, 樊棠怀. 2019. 基于多通道均衡化的水下彩色图像增强算法. 华中科技大学学报(自然科学版), 47(6): 1-5, 29) [DOI:10.13245/j.hust.190601]
  • Li X C, Zhang H, Liu X Y, Zong F Y, Liu J L. 2019. A method of underwater image enhancement based on transmittance optimization and color calibration. Shandong Science, 32(2): 7-14 (李向春, 张浩, 刘晓燕, 宗芳伊, 刘军礼. 2019. 基于透射率优化和颜色修正的水下图像增强方法. 山东科学, 32(2): 7-14) [DOI:10.3976/j.issn.1002-4026.2019.02.002]
  • Liao Q X, Lu Z S, Peng J H. 2019. Research on PSF parameter estimation and image restoration of motion blurred image. Chinese High Technology Letters, 29(4): 338-343 (廖秋香, 卢在盛, 彭金虎. 2019. 运动模糊图像PSF参数估计与图像复原研究. 高技术通讯, 29(4): 338-343) [DOI:10.3772/j.issn.1002-0470.2019.04.004]
  • Singh G, Jaggi N, Vasamsetti S, Sardana H K, Kumar S and Mittal N. 2015. Underwater image/video enhancement using wavelet based color correction (WBCC) method//Proceedings of 2015 IEEE Underwater Technology. Chennai, India: IEEE: 23-25 (DOI: 10.1109/UT.2015.7108303]
  • Song W, Wang Y, Huang D M, He Q, Wang Z H. 2018. Combining background light fusion and underwater dark channel prior with color balancing for underwater image enhancement. Pattern Recognition and Artificial Intelligence, 31(9): 856-868 (宋巍, 王龑, 黄冬梅, 贺琪, 王振华. 2018. 结合背景光融合及水下暗通道先验和色彩平衡的水下图像增强. 模式识别与人工智能, 31(9): 856-868) [DOI:10.16451/j.cnki.issn1003-6059.201809008]
  • Wan M J, Gu G H, Qian W X, Ren K, Chen Q. 2018. Infrared small target enhancement:grey level mapping based on improved sigmoid transformation and saliency histogram. Journal of Modern Optics, 65(10): 1161-1179 [DOI:10.1080/09500340.2018.1426796]
  • Wang F P, Wang W X, Yang N, Sui L C. 2017. An urban traffic image enhancement method based on modified retinex. Journal of Transportation Systems Engineering and Information Technology, 17(5): 53-59 (王峰萍, 王卫星, 杨楠, 隋立春. 2017. 基于改进Retinex的城市交通图像增强. 交通运输系统工程与信息, 17(5): 53-59) [DOI:10.16097/j.cnki.1009-6744.2017.05.008]
  • Wang H R, Zhu Z J, Wang J. 2017. An efficient underwater image enhancement algorithm for unmanned underwater vehicle. Ship Engineering, 39(12): 95-101 (王红茹, 朱振杰, 王佳. 2017. 一种面向水下无人航行器的实效图像增强算法. 船舶工程, 39(12): 95-101) [DOI:10.13788/j.cnki.cbgc.2017.12.095]
  • Wang Y X, Diao M, Han C. 2018. Underwater image enhancement and color correction model based on homomorphic filter. Computer Engineering and Applications, 54(11): 30-34, 80 (王永鑫, 刁鸣, 韩闯. 2018. 基于同态滤波的水下图像增强与色彩校正模型. 计算机工程与应用, 54(11): 30-34, 80) [DOI:10.3778/j.issn.1002-8331.1801-0131]
  • Wen H C, Tian Y H, Huang T J and Gao W. 2013. Single underwater image enhancement with a new optical model//Proceedings of 2013 IEEE International Symposium on Circuits and Systems. Beijing, China: IEEE: 753-756[DOI: 10.1109/ISCAS.2013.6571956]
  • Xie F Y, Tang M, Zhang R. 2019. Review of image enhancement algorithms based on Retinex. Journal of Data Acquisition and Processing, 34(1): 1-11 (谢凤英, 汤萌, 张蕊. 2019. 基于Retinex的图像增强方法综述. 数据采集与处理, 34(1): 1-11) [DOI:10.16337/j.1004-9037.2019.01.001]
  • Wen H B, Bi D Y, Ma S P, He L Y. 2016. Variational Retinex algorithm for infrared image enhancement with staircase effect suppression and detail enhancement. Acta Optica Sinica, 36(9): 122-131 (温海滨, 毕笃彦, 马时平, 何林远. 2016. 消除阶梯效应与增强细节的变分Retinex红外图像增强算法. 光学学报, 36(9): 122-131) [DOI:10.3788/AOS201636.0911005]
  • Xie N. 2017. Research on the image defogging algorithm based on image enhancement. Machine Design and Manufacturing Engineering, 46(12): 31-33 (谢娜. 2017. 基于图像增强的图像去雾算法研究. 机械设计与制造工程, 46(12): 31-33) [DOI:10.3969/j.issn.2095-509X.2017.12.007]
  • Xu T F, Su C, Luo X, Bian Z Y. 2017. Enhancement of the underwater range-gated degraded image. Transactions of Beijing Institute of Technology, 37(8): 853-857, 862 (许廷发, 苏畅, 罗璇, 卞紫阳. 2017. 水下距离选通降质图像的增强. 北京理工大学学报, 37(8): 853-857, 862) [DOI:10.15918/j.tbit1001-0645.2017.08.015]
  • Yang M, Sowmya A. 2015. An underwater color image quality evaluation metric. IEEE Transactions on Image Processing, 24(12): 6062-6071 [DOI:10.1109/TIP.2015.2491020]
  • Zhang C, Tan N L, Li X, Li G Z, Su S Q. 2019. Foggy image enhancement technology based on improved Retinex algorithm. Journal of Beijing University of Aeronautics and Astronautics, 45(2): 309-316 (张驰, 谭南林, 李响, 李国正, 苏树强. 2019. 基于改进型Retinex算法的雾天图像增强技术. 北京航空航天大学学报, 45(2): 309-316) [DOI:10.13700/j.bh.1001-5965.2018.0316]
  • Zhang H, Xu Y R, Wan L, Tang X D, Cai H P. 2010. Processing method for underwater degenerative image. Journal of Tianjin University, 43(9): 827-833 (张赫, 徐玉如, 万磊, 唐旭东, 蔡昊鹏. 2010. 水下退化图像处理方法. 天津大学学报, 43(9): 827-833) [DOI:10.3969/j.issn.0493-2137.2010.09.012]