Current Issue Cover
自然场景图像去雨的可学习混合MAP网络

马龙1,2, 刘日升1,2, 姜智颖1,2, 王怡洋3, 樊鑫1,2, 李豪杰1,2(1.大连理工大学-立命馆大学国际信息与软件学院, 大连 116621;2.辽宁省泛在网络与服务软件重点实验室, 大连 116621;3.大连理工大学数学科学学院, 大连 116024)

摘 要
目的 近年来,人工智能成为新兴研究领域,得到越来越多研究机构的关注。图像恢复问题一直是低层次计算机视觉领域的一个研究热点,其中,图像去雨由于其雨线分布的未知性及其求解的病态性,导致难以解决。现有方法存在雨线和背景之间的估计具有依赖性,难以平衡雨线去除效果与估计背景的清晰程度之间的关系;局限性比较大,训练数据很难涵盖各种场景下的雨图,而测试结果受训练数据的影响,导致难于泛化。针对上述不足,借鉴一般图像恢复问题思路,将模型与以数据驱动的网络相结合,凸显网络与模型各自的优势,提出可学习的混合MAP网络有效地解决图像去雨问题。方法 首先基于最大后验估计(MAP)建立含有隐式先验的能量模型,然后通过优化算法将模型分解为背景估计模型和雨线估计模型两部分,以减少背景估计和雨线估计之间的依赖性。对于背景估计模型,通过对模型及优化目标分析采用以数据驱动的去噪残差网络进行建模,保证估计出的背景更清晰;对于雨线估计模型,为避免直接对未知的雨线建模失去准确性,利用高斯混合模型实时刻画输入雨图的雨线先验。结果 在合成数据集Rain12及真实雨图上进行实验,通过综合考虑定量分析和定性分析,并与3种基于模型的方法及两种基于深度网络的方法相比,本文方法在去除雨线的同时能够损失的背景信息最少,合成数据集上的平均结构相似性(SSIM)值达到0.92。结论 本文通过将基于模型的方法与基于深度网络的方法相结合,既去除了雨线又保证了估计背景的清晰程度,同时也验证了将传统模型与深度网络相结合是一种解决图像恢复问题的有效途径。
关键词
Rain streak removal using learnable hybrid MAP network

Ma Long1,2, Liu Risheng1,2, Jiang Zhiying1,2, Wang Yiyang3, Fan Xin1,2, Li Haojie1,2(1.DUT-RU International School of Information Science & Engineering, Dalian University of Technology, Dalian 116621, China;2.Key Laboratory for Ubiquitous Network and Service Software of Liaoning Province, Dalian 116621, China;3.School of Mathematical Sciences, Dalian University of Technology, Dalian 116024, China)

Abstract
Objective In recent years, artificial intelligence has become the emerging area and attract the attention of many research institutions. Computer vision is the force at the core of the development in the area of artificial intelligence. Image restoration remains an active research topic in low-level computer vision. Because the distribution of the rain streaks is irregular and the problem is ill-posed, rain removal for the single image is one of the difficult problems to be solved. So far, the basic idea to solve the problem is building the rational rain image generative model, and find the solution by different measures. There are two representative ways. One is by describing the priors of rain streaks and background with conventional models and then solve it by the optimization algorithm, but the shortcoming is due to the interdependent connection between rain streaks and the background, it is hard to balance the relationship between the performance of rain removal and the clarity of background. The other is deep neural network based framework to build the network structure and loss function, then learning the network with large amounts of data, but the shortcoming is the difficulty of generalization because of test results are subject to the training data which is hard to cover the rain in the different scene. Method For the shortcomings mentioned above, inspired by general image restoration, this paper combines the network by data-driven with the empirical model, and the process displays the advantages of the network and empirical model, then learnable mixture MAP network is proposed so that image deraining problem can be solved effectively. The specific process is to build the energy model which contains the implicit prior item based on maximum a posterior (MAP) framework, then transfer the model to two parts that is estimating background and estimating rain streaks by the optimization algorithm, and that reduce the dependency of background and rain streaks. For the first part, it can be solved by the residual network for denoising by analysis the model and the optimization objective and in this way, the estimated image is more clearly. For the second part, the rain streaks prior to the input image can be described real-time by gaussian mixture model to improve the accuracy of estimating rain streaks. Result In the end, compared with the three empirical model-based algorithms and two deep learning-based algorithms, the experiments on the synthetic dataset and real-world image demonstrate the good performance of our model on removing rain streaks meanwhile reducing the information loss of the background, in addition, the SSIM value reaches to 0.92 on the synthetic dataset. Conclusion This paper proposes to combine model-based method with deep network-based method, assimilates the advantages from each other. Not only can the majority of visible rain streaks be removed, but also remains detail information of background. Achieving a harmonious of the two aspects and standing out from state-of-the-art methods. At the same time, this paper makes a verification, which is the effective way to solve the image restoration problem by combining the conventional model and deep network.
Keywords

订阅号|日报