浏览全部资源
扫码关注微信
Published： 16 October 2022 ，
Accepted： 12 August 2021
移动端阅览
Zhiqing Meng, Jing Zhang, Jianshu Qiu. Multi-supervision loss function based smoothed super-resolution image reconstruction. [J]. Journal of Image and Graphics 27(10):2972-2983(2022)
目的
2
将低分辨率(low-resolution，LR)图像映射到高分辨率(high-resolution，HR)图像是典型的不适定恢复问题，即输出的HR图像和输入的LR图像之间的映射是多对一的，这意味着仅通过增加网络深度来确定HR图像与LR图像之间的特定映射关系是非常困难的。针对该问题，本文提出一种基于多监督光滑化损失函数的图像超分辨率方法。
方法
2
该方法主体由LR图像上采样通道和HR图像下采样通道两部分组成。各通道分为两个阶段，每个阶段均包括浅层特征提取模块、基于迭代采样错误反馈机制的采样模块、全局特征融合模块和图像重建模块。将LR图像上采样通道第1阶段结果与HR图像下采样通道第1阶段结果对比，然后将HR原图像和HR图像下采样通道第2阶段结果作为约束构成多监督，使映射函数空间尽可能精确，并将多监督损失函数光滑化保证梯度在全局范围内传递。
结果
2
在基准测试集Set5、Set14、BSD100(Berkeley segmentation dataset)、Urban100(urban scenes dataset)、Manga109(109 manga volumes dataset)数据集上进行测试，并与Bicubic、SRCNN(super-resolution convolutional neural network)、FSRCNN(fast super-resolution convolutional neural network)、LapSRN(Laplacian pyramid super-resolution network)、VDSR(very deep super-resolution convolutional networks)、DBPN(deep back-projection networks for super-resolution)和DRN(dual regression networks)等方法的实验结果进行对比。当放大因子为4时，本文算法的峰值信噪比分别为32.29 dB、28.85 dB、27.61 dB、26.16 dB和30.87 dB；在重建图像的可视化分析方面，本文算法相较于对比算法具有更加丰富的纹理和清晰的轮廓。
结论
2
实验结果表明，基于多监督光滑化损失函数方法的图像重建结果与其他超分辨率主流算法相比，在重建图像质量和高频细节处理方面均有所提高。
Objective
2
The issue of images interpretation can be used in human cancer diagnosis contexts. It can help law enforcement officers to forensics via license plate images as well. Single super-resolution image process restores the high-resolution image based on a low-resolution image. It is the key to mapping reconstruction to clarify the relationship between the low-resolution image and the high-resolution image. Emerging neural networks have developed super-resolution image contexts. To improve the clarity of reconstructed images
theses novel networks are used to learn information feature between low-resolution images and high-resolution images. It is challenged that the mapping is many-to-one between the output high-resolution image and the input low-resolution image
which tends to determine the high-resolution image via more network depth only. The specific mapping relationship is difficult to be identified between low-resolution images. We facilitate a reconstructed super-resolution image method based on multi-supervised smoothing loss function
this method can clarify the possible mapping space between the low-resolution image and the high-resolution image in terms of the added constraints to the objective function
and improves the quality of the reconstructed image.
Method
2
Our method is composed of two channels like low-resolution (LR) image up-sampling and high-resolution (HR) image down-sampling. Each channel is divided into two stages
and each stage includes 4 modules in the context of a shallow feature extraction
an iterative sampling error feedback mechanism based sampling
a global feature fusion
and an image reconstruction
respectively. Our analysis compares the results of the first stage of the LR image upsampling channel to the results of the first stage of the HR image downsampling channel. To map the function space accurately
the HR original image and the second stage result of the HR image downsampling channel are as the constraints to form multi-supervision
and the smoothed multi-supervised loss function ensures that the gradient is transmitted in the global scope
the channel attention module is added to the method in order to reduce the computational complexity of the network and get more effective features. The channel attention module can extract useful features and suppress invalid features to speed up network convergence. In order to improve the network ability to reconstruct image detail information
we introduce a global feature fusion module
which improves the degree of information fusion in the network
and utilizes shallow information and detailed texture effectively.
Result
2
We use the DIV2K (diverse 2k resolution high quality images)dataset as the training dataset. The DIV2K dataset is organized by 800 training images
100 verification images and 100 test images. To verify the algorithm performance
we carry out quantitative and qualitative analyses both. The quantitative tests are performed on benchmark test sets
such as Set5
Set14
Berkeley segmentation dataset (BSD100)
urban scenes dataset (Urban100)
and 109 manga volumes dataset (Manga109). The results are compared to Bicubic
super-resolution convolutional neural network (SRCNN)
fast super-resolution convolutional neural network (FSRCNN)
Laplacian pyramid super-resolution network (LapSRN)
very deep super-resolution convolutional networks (VDSR)
deep back-projection networks for super-resolution (DBPN)
and dual regression networks (DRN)
our peak signal-to-noise ratio of the algorithm is 32.29 dB
28.85 dB
27.61 dB
26.16 dB and 30.87 dB
respectively when the magnification factor is 4; the reconstructed image of this algorithm has richer texture in terms of the qualitative analysis and the reconstructed image is closer to the real image with a clear outline.
Conclusion
2
Our multi-supervision smoothing loss function is used as the objective function
and the objective function mapping space is clear. In addition
the network calculation complexity is reduced through adding the channel attention module and the global feature fusion module. The network convergence speed is accelerated
and the shallow features are fully utilized. Our experimental results show that the multi-supervised smoothing loss function based image reconstruction results have improved the quality of the reconstructed image and the processing of high-frequency detail in comparison with some typical super-resolution algorithms.
超分辨率重建迭代采样多监督映射空间光滑化损失函数
super-resolution reconstructioniterative samplingmulti-supervisionmapping spacesmoothing loss function
Agustsson E and Timofte R. 2017. NTIRE 2017 challenge on single image super-resolution: dataset and study//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops. Honolulu, USA: IEEE: 1122-1131 [DOI: 10.1109/CVPRW.2017.150http://dx.doi.org/10.1109/CVPRW.2017.150]
Cheong J Y and Park I K. 2017. Deep CNN-based super-resolution using external and internal examples. IEEE Signal Processing Letters, 24(8): 1252-1256 [DOI: 10.1109/LSP.2017.2721104]
Christensen-Jeffries K, Couture O, Dayton P A, Eldar Y C, Hynynen K, Kiessling F, O'Reilly M, Pinton G F, Schmitz G, Tang M X, Tanter M and Van Sloun R J G. 2020. Super-resolution ultrasound imaging. Ultrasound in Medicine and Biology, 46(4): 865-891 [DOI: 10.1016/j.ultrasmedbio.2019.11.013]
Dogan B, Gu S H and Timofte R. 2019. Exemplar guided face image super-resolution without facial landmarks//Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. Long Beach, USA: IEEE: 1814-1823 [DOI: 10.1109/CVPRW.2019.00232http://dx.doi.org/10.1109/CVPRW.2019.00232]
Dong C, Loy C C, He K M and Tang X O. 2016a. Image super-resolution using deep convolutional networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 38(2): 295-307 [DOI: 10.1109/TPAMI.2015.2439281]
Dong C, Loy C C and Tang X O. 2016b. Accelerating the super-resolution convolutional neural network//Proceedings of the 14th European Conference on Computer Vision. Amsterdam, the Netherlands: Springer: 391-407 [DOI: 10.1007/978-3-319-46475-6_25http://dx.doi.org/10.1007/978-3-319-46475-6_25]
Guo Y, Chen J, Wang J D, Chen Q, Cao J Z, Deng Z S, Xu Y W and Tan M K. 2020. Closed-loop matters: dual regression networks for single image super-resolution//Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle, USA: IEEE: 5406-5415 [DOI: 10.1109/CVPR42600.2020.00545http://dx.doi.org/10.1109/CVPR42600.2020.00545]
Haris M, Shakhnarovich G and Ukita N. 2018. Deep back-projection networks for super-resolution//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, USA: IEEE: 1664-1673 [DOI: 10.1109/CVPR.2018.00179http://dx.doi.org/10.1109/CVPR.2018.00179]
He K M, Zhang X Y, Ren S Q and Sun J. 2016. Deep residual learning for image recognition//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, USA: IEEE: 770-778 [DOI: 10.1109/CVPR.2016.90http://dx.doi.org/10.1109/CVPR.2016.90]
Hu J, Shen L and Sun G. 2018. Squeeze-and-excitation networks//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, USA: IEEE: 7132-7141 [DOI: 10.1109/CVPR.2018.00745http://dx.doi.org/10.1109/CVPR.2018.00745]
Huang J B, Singh A and Ahuja N. 2015. Single image super-resolution from transformed self-exemplars//Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition. Boston, USA: IEEE: 5197-5206 [DOI: 10.1109/CVPR.2015.7299156http://dx.doi.org/10.1109/CVPR.2015.7299156]
Hui Z, Wang X M and Gao X B. 2018. Fast and accurate single image super-resolution via information distillation network//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, USA: IEEE: 723-731 [DOI: 10.1109/CVPR.2018.00082http://dx.doi.org/10.1109/CVPR.2018.00082]
Kim J, Lee J K and Lee K M. 2016a. Accurate image super-resolution using very deep convolutional networks//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, USA: IEEE: 1646-1654 [DOI: 10.1109/CVPR.2016.182http://dx.doi.org/10.1109/CVPR.2016.182]
Kim J, Lee J K and Lee K M. 2016b. Deeply-recursive convolutional network for image super-resolution//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, USA: IEEE: 1637-1645 [DOI: 10.1109/CVPR.2016.181http://dx.doi.org/10.1109/CVPR.2016.181]
Lai W S, Huang J B, Ahuja N and Yang M H. 2017. DeepLaplacian pyramid networks for fast and accurate super-resolution//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, USA: IEEE: 5835-5843 [DOI: 10.1109/CVPR.2017.618http://dx.doi.org/10.1109/CVPR.2017.618]
Laroca R, Severo E, Zanlorensi L A, Oliveira L S, Gonçalves G R, Schwartz W R and Menotti D. 2018. A robust real-time automatic license plate recognition based on the YOLO detector//Proceedings of 2018 International Joint Conference on Neural Networks (IJCNN). Rio de Janeiro, Brazil: IEEE: 1-10 [DOI: 10.1109/IJCNN.2018.8489629http://dx.doi.org/10.1109/IJCNN.2018.8489629]
Lei P C, Liu C, Tang J G and Peng D L. 2020. Hierarchical feature fusion attention network for image super-resolution reconstruction. Journal of Image and Graphics, 25(9): 1773-1786
雷鹏程, 刘丛, 唐坚刚, 彭敦陆. 2020. 分层特征融合注意力网络图像超分辨率重建. 中国图象图形学报, 25(9): 1773-1786 [DOI: 10.11834/jig.190607]
Li B, Wang P and Zhao S Y. 2021. Image super-resolution reconstruction based on dual attention mechanism. Journal of Graphics, 42(2): 206-215
李彬, 王平, 赵思逸. 2021. 基于双重注意力机制的图像超分辨重建算法. 图学学报, 42(2): 206-215 [DOI: 10.11996/JG.j.2095-302X.2021020206]
Lim B, Son S, Kim H, Nah S and Lee K M. 2017. Enhanced deep residual networks for single image super-resolution//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops. Honolulu, USA: IEEE: 1132-1140 [DOI: 10.1109/CVPRW.2017.151http://dx.doi.org/10.1109/CVPRW.2017.151]
Ma W, Pan Z X, Yuan F and Lei B. 2019. Super-resolution of remote sensing images via a dense residual generative adversarial network. Remote Sensing, 11(21): #2578 [DOI: 10.3390/rs11212578]
Onaran I, Ince N F and Cetin A E. 2013. Sparse spatial filter via a novel objective function minimization with smoothℓ1regularization. Biomedical Signal Processing and Control, 8(3): 282-288 [DOI: 10.1016/j.bspc.2012.10.003].
Shi W Z, Caballero J, Huszár F, Totz J, Aitken A P, Bishop R, Rueckert D and Wang Z H. 2016. Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, USA: IEEE: 1874-1883 [DOI: 10.1109/CVPR.2016.207http://dx.doi.org/10.1109/CVPR.2016.207]
Tai Y, Yang J and Liu X M. 2017. Image super-resolution via deep recursive residual network//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, USA: IEEE: 2790-2798 [DOI: 10.1109/CVPR.2017.298http://dx.doi.org/10.1109/CVPR.2017.298]
Tang R, Chen L H, Zhang R Z, Ahmad A, Albertini M K and Yang X M. 2022. Medical image super-resolution with Laplacian dense network. Multimedia Tools and Applications, 81(3): 3131-3144 [DOI: 10.1007/s11042-020-09845-y]
Xie L L, Ahmad T, Jin L W, Liu Y L and Zhang S. 2018. A new CNN-based method for multi-directional car license plate detection. IEEE Transactions on Intelligent Transportation Systems, 19(2): 507-517 [DOI: 10.1109/TITS.2017.2784093]
Youm G Y, Bae S H and Kim M. 2016. Image super-resolution based on convolution neural networks using multi-channel input//Proceedings of the 12th IEEE Image, Video, and Multidimensional Signal Processing Workshop (IVMSP). Bordeaux, France: IEEE: 1-5 [DOI: 10.1109/IVMSPW.2016.7528224http://dx.doi.org/10.1109/IVMSPW.2016.7528224]
Zhang Z J. 2018. Improved ADAM optimizer for deep neural networks//Proceedings of the 26th IEEE/ACM International Symposium on Quality of Service (IWQoS). Banff, Canada: IEEE: 1-2 [DOI: 10.1109/IWQoS.2018.8624183http://dx.doi.org/10.1109/IWQoS.2018.8624183]
Zhong Z, Zheng L, Kang G L, Li S Z and Yang Y. 2020. Random erasing data augmentation. Proceedings of the AAAI Conference on Artificial Intelligence, 34(7): 13001-13008 [DOI: 10.1609/aaai.v34i07.7000]
相关文章
相关作者
相关机构
京公网安备11010802024621