Current Issue Cover
沙尘图像色彩恢复及增强卷积神经网络

石争浩1, 刘春月1, 任文琦2, 都双丽1, 赵明华1(1.西安理工大学计算机科学与工程学院, 西安 710048;2.中山大学网络空间安全学院, 深圳 518107)

摘 要
目的 在沙尘天气条件下,由于大气中悬浮微粒对入射光线的吸收和散射,户外计算机视觉系统所采集图像通常存在颜色偏黄失真和低对比度等问题,严重影响户外计算机视觉系统的性能。为此,提出一种带色彩恢复的沙尘图像卷积神经网络增强方法,由一个色彩恢复子网和一个去尘增强子网组成。方法 采用提出的色彩恢复子网(sand dust color correction, SDCC)校正沙尘图像的偏色,将颜色校正后的图像作为条件,输入到由自适应实例归一化残差块组成的去尘增强子网中,对沙尘图像进行增强处理。本文还提出一种基于物理光学模型的沙尘图像合成方法,并采用该方法构建了大规模的配对沙尘图像数据集。结果 对大量沙尘图像的实验结果表明,所提出的沙尘图像增强方法能很好地去除图像中的偏色和沙尘,获得正常的视觉颜色和细节清晰的图像。进一步的对比实验表明,该方法能取得优于对比方法的增强图像。结论 本文所提出的沙尘图像增强方法能很好地消除整体的黄色色调和尘霾现象,获得正常的视觉色彩和细节清晰的图像。
关键词
Convolutional neural networks for sand dust image color restoration and visibility enhancement

Shi Zhenghao1, Liu Chunyue1, Ren Wenqi2, Du Shuangli1, Zhao Minghua1(1.School of Computer Science and Engineering, Xi'an University of Technology, Xi'an 710048, China;2.School of Cyber Science and Technology, Sun Yat-sen University, Shenzhen 518107, China)

Abstract
Objective The quality of captured images tends to yellowish color distortion, reduced contrast, and detailed information loss in the sand dust atmosphere due to the suspended particles derived incident light absorption and scattering. The issues of outdoor computer vision systems like video surveillance, video navigation and intelligent transportation are severely constrained. Traditional sand dust image enhancement methods are originated from visual perception based sand dust image enhancement and physical model based sand dust image restoration. The visual perception based method is not restricted of the physical imaging model. The visual quality is based on color correction and contrast enhancement. The recovered image still has insufficient color distortion, image brightness and image contrast. Physical models based sand dust image restoration is related to additional assumptions and less prior robustness, complex parameters calculation. Nowadays, existing deep learning-based sand dust image enhancement methods are migrated from the deep learning based haze images methods. Although these methods has achieved good results for haze image processing, the color of the output image still has different degrees of distortion, and the sharpness of the image is also relatively poor in terms of transferred and enhanced sand dust images. An enhanced convolutional neural network (CNN) method of restored color sand dust images can be used to resolve and improve the issues mentioned above. Method Our proposed network structure consists of a sand dust color correction subnet and a dust removal enhancement subnet. We illustrated a novel sand dust color correction network structure to improve gray world algorithm. First, the proposed sand dust color correction subnet (SDCC) is used to correct the color cast of the sand dust image. The sand dust image is de-composed into 3 channels of R, G, and B. For each channel, a convolutional layer with a convolution kernel size of 3 is used to conducted, and each feature map is processed to obtain color correction image via gray world algorithm. To enhance the sand dust images quality, a benched color-corrected image is transmitted into the dust removal enhancement subnet in the context of adaptive instance normalized-residual blocks (AIN-ResBlock). The dust removal enhancement subnet takes the sand image and the color correction image as input, and uses the adaptive instance normalization module to adaptively restore the color distortion issues in the feature mapping in the dust removal enhancement subnet, and realizes the image sand removal through the residual block. Our AIN-ResBlock is capable to resolve the blurred details and missing image content for the natural color factors of the restored image. Additionally, in view of the difficulty in obtaining pairs of sand dust images and their corresponding clear images as training samples for deep learning, a sand dust image synthesis method is illustrated based on a physical imaging model. Absorb and scatter light to attenuate, and the attenuation degree of light of different colors is different. We optioned 15 color marks close to the color of the sand dust image, and simulate the sand dust image under 15 different conditions, and a large-scale dataset of clear image and sand dust images is finally constructed. Our loss function used is composed of L1 loss function, perceptual loss function and gradient loss function in training the network. In order to validate the targeted image ground truth, we use L1 loss in color correction subnet and a dust removal enhancement subnet; A perceptual loss is used to narrow the difference between the perceptual features of the sand dust image enhancement network results and the perceptual features of the real image; In order to better restore the details and structure of the image, we use the horizontal and vertical gradient loss in the network function. Result The performance of the method is verified by synthetic images and real images. The experimental results illustrated that our sand dust image enhancement method can remove the color cast and dust of the sand dust image, and obtain normal visual colors and details clear image. The performance of our method is based on the peak signal-to-noise ratio (PSNR), structural similarity (SSIM), natural image quality evaluator (NIQE), perception-based image quality evaluator (PIQE), blind/referenceless image spatial quality evaluator (BRISQUE), the percentage of newly visible edges e, the contrast restoration quality r and saturationσ are estimated each. Compared with the existing methods, our method obtains the highest average PSNR and average SSIM on the composite image, which are 18.705 7 dB and 0.669 5 respectively. Our method can also significantly improve the quality of the enhanced image on real sand dust images, and obtain enhanced images with good visual effects. Conclusion We propose a CNN-based enhancement method for sand dust images with color restoration. This method can restore the color cast of the sand dust image, improve the contrast of the image, restore the detailed information of the image, and obtain normal visual colors and clear details image. The synthetic sand dust image and the real sand dust image have their priorities in visual effects and facilitate quantitative evaluation indicators structure further.
Keywords

订阅号|日报