Contour perception based on multi-path convolution neural network

Mingming Tan; Yingle Fan; Wei Wu; Qingshan She; Haitao Gan

doi:10.11834/jig.190014

Image Analysis and Recognition | Views : 0 下载量: 5 CSCD: 0

PDF
Export
Share
Collection
Album

Contour perception based on multi-path convolution neural network
Vol. 24, Issue 10, Pages: 1750-1760(2019)
Received：12 January 2019，

Revised：2019-4-15，

Accepted：22 April 2019，

Published：16 October 2019
DOI： 10.11834/jig.190014
稿件说明：

移动端阅览

DOI：

Mingming Tan, Yingle Fan, Wei Wu, Qingshan She, Haitao Gan. Contour perception based on multi-path convolution neural network[J]. Journal of Image and Graphics, 2019, 24(10): 1750-1760. DOI： 10.11834/jig.190014.

摘要

目的

引入视觉信息流的整体和局部处理机制，提出了一种多路径卷积神经网络的轮廓感知新方法。

方法

利用高斯金字塔尺度分解获得低分辨率子图，用来表征视觉信息中的整体轮廓；通过2维高斯导函数模拟经典感受野的方向选择性，获得描述细节特征的边界响应子图；构建多路径卷积神经网络，利用具有稀疏编码特性的子网络（Sparse-Net）实现对整体轮廓的快速检测；利用具有冗余度增强编码特性的子网络（Redundancy-Net）实现对局部细节特征提取；对上述多路径卷积神经网络响应进行融合编码，以实现轮廓响应的整体感知和局部检测融合，获取轮廓的精细化感知结果。

结果

以美国伯克利大学计算机视觉组提供的数据集BSDS500图库为实验对象，在GTX1080Ti环境下本文Sparse-Net对整体轮廓的检测速度达到42幅/s，为HFL方法1.2幅/s的35倍；而Sparse-Net和Redundancy-Net融合后的检测指标数据集尺度上最优（ODS）、图片尺度上最优（OIS）、平均精度（AP）分别为0.806、0.824、0.846，优于HED（holistically-nested edge detection）方法和RCF（richer convolution features for edge detection）方法，结果表明本文方法能有效突出主体轮廓并抑制纹理背景。

结论

多路径卷积神经网络的轮廓感知应用，将有助于进一步理解视觉感知机制，并对减弱卷积神经网络的黑盒特性有着重要的意义。

Abstract

Objective

This study aims to introduce the global and local processing mechanism of visual information flow by constructing a visual information encoding and decoding model based on the correlation between visual nerve coding and contour perception and propose a contour perception method based on multi-path convolution neural network.

Method

The Gauss pyramid scale decomposition was used to obtain low-resolution molecular images to characterize the whole contour of visual information. Two-dimensional Gauss derivative was used to simulate the directional selectivity of classical receptive fields to obtain boundary response sub-graphs describing details. A multi-path convolution neural network was constructed

and a sparse encoding sub-network (Sparse-Net) was used to realize the fast processing of the whole contour detection. Redundancy enhanced coding (Redundancy-Net) was used to extract local details. The response of the multi-path convolution neural network was fused and coded to integrate global perception and local detection of contour responses and obtain the fine perception results of the contour.

Result

With the BSDS500 image database provided by Berkely Computer Vision Group as the experimental object

the detection speed of Sparse-Net in GTX1080Ti environment reached 42 frame/s

which was 35 times higher than that of HFL method (1.2 frame/s). The detection index data set of Sparse-Net and Redundancy-Net after fusion was the best in scale (ODS) and picture scale (OIS) and AP are 0.806

0.824

and 0.846 respectively

which are better than the holistically-nested edge detection (HED) and richer convolution features for edge detection (RCF) methods

which are based on the analysis of the lateral output feature map

progressive encoding and decoding

and feature fusion from the shallow to the deep layer of the network

learning fine contour features and achieving end-to-end contour detection. The proposed method cannot only effectively highlight the main contour and suppress the texture background but also improve the detection efficiency of contour.

Conclusion

Convolution neural network can be explained by visual mechanism in some dimensions

such as convolution operation corresponding to the topological mapping of retinal visual information. Pooling operation is related to complex cells and simple cells in visual pathway. As such

convolution neural network is still a black box model which depends heavily on massive samples on the whole. Considering that the actual visual pathway is not simply a serial transmission of information but a fusion of the local and global characteristics of multi-channel visual information flow in the visual cortex

a Gauss pyramid decomposition model was constructed for sparse encoding of the spatial scale of visual information and obtaining low-resolution molecular maps representing the overall characteristics. Lateral suppression of non-classical receptive fields was used in the lateral geniculate region. A classical receptive field with directional selection characteristics was set up for isotropic suppression of background information and considering the ability of primary visual cortex for information processing in the visual radiation region. A two-dimensional Gauss derivative model was constructed to process the visual information by directional selection. The boundary response sub-graph representing local features was obtained. A multi-path convolution neural network was constructed considering the local details of external excitation and the layer-by-layer perception of overall information in the primary and advanced visual cortex. In the network

the fast detection path was composed of a sub-network Sparse-Net containing a pooling unit for sparse coding of the overall image contour. The detail detection path was composed of a sub-network Redundancy-Net containing a void convolution unit to realize image bureau. Redundancy enhanced the coding of part details. Finally

the feedback and fusion process of high-level visual cortex to visual information flow was simulated

and the above-mentioned multi-path convolution neural network response was fused and coded for overall perception and local detection fusion of the contour response. Finally

the fine perception results of the contour were obtained. Contour perception based on multi-path convolution neural network is helpful to further understand the mechanism of visual perception and is of great significance to weaken the black-box characteristics of the convolution neural network. Taking the natural scene image subject contour perception under complex texture background as an example

simulating the neural coding mechanism of multi-path cooperative work in primary visual pathway will help understand the intrinsic mechanism of visual system and its specific application in visual perception. This works provides a new idea for subsequent image understanding and analysis based on visual mechanism.

关键词

Keywords

references

Gonzalez C I, Melin P, Castro J R, et al. An improved sobel edge detection method based on generalized type-2 fuzzy logic[J]. Soft Computing, 2016, 20(2):773-784.[DOI:10.1007/s00500-014-1541-0]

Yang H Y, Wu J F, Yu Y J, et al. Content based image retrieval using color edge histogram in HSV color space[J]. Journal of Image and Graphics, 2018, 13(10):2035-2038.

杨红颖, 吴俊峰, 于永健, 等.一种基于HSV空间的彩色边缘图像检索方法[J].中国图象图形学报, 2018, 13(10):2035-2038. [DOI:10.11834/jig.20081054]

Lin C, Xu G L, Cao Y J. Contour detection model based on neuron behaviour in primary visual cortex[J]. IET Computer Vision, 2018, 12(6):863-872.[DOI:10.1049/iet-cvi.2017.0661]

Zhou T, Fan Y L, Zhu Y P, et al. Fast contour detection method based on hierarchical response model of primary visual pathway[J]. Space Medicine&Medical Engineering, 2018, 31(3):75-83.

周涛, 范影乐,朱亚萍, 等.基于主视通路层级响应模型的轮廓检测方法[J].航天医学与医学工程, 2018, 31(3):75-83. [DOI:10.16289/j.cnki.1002-0837.2018.03.012]

Chen H, Qi X J, Yu L Q, et al. DCAN:deep contour aware networks for object instance segmentation from histology images[J]. Medical Image Analysis, 2017, 36:135-146.[DOI:10.1016/j.media.2016.11. 004]

Yang W H, Feng J S, Yang J C, et al. Deep edge guided recurrent residual learning for image super resolution[J]. IEEE Transactions on Image Processing, 2017, 26(12):5895-5907.[DOI:10.1109/TIP.2017.2750403]

Dollár P, Zitnick C L. Fast edge detection using structured forests[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 37(8):1558-1570.[DOI:10.1109/TPAMI.2014.2377715]

Hallman S, Fowlkes C C. Oriented edge forests for boundary detection[C]//Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition. Boston, MA, USA: IEEE, 2015: 1732-1740.[ DOI: 10.1109/CVPR.2015.7298782 http://dx.doi.org/10.1109/CVPR.2015.7298782 ]

Xie S N, Tu Z W. Holistically-nested edge detection[J]. International Journal of Computer Vision, 2017, 125(1-3):3-18.[DOI:10.1007/s11263-017-1004-z]

Liu Y, Cheng M M, Hu X W, et al. Richer convolutional features for edge detection[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, Hawaii, USA: IEEE, 2017: 5872-5881.[ DOI: 10.1109/CVPR.2017.622 http://dx.doi.org/10.1109/CVPR.2017.622 ]

Shao J, Gao J. Visual selective attention computational model based on synergetic perception[J]. Journal of Image and Graphics, 2008, 13(1):129-136.

邵静, 高隽.基于协同感知的视觉选择注意计算模型[J].中国图象图形学报, 2008, 13(1):129-136. [DOI:10.11834/jig.20080124]

Qian L L, Gao J, Xie Z. An improved hierarchical generic object recognition algorithm based on neural sparse coding[J]. Journal of Image and Graphics, 2010, 15(10):1521-1529.

钱乐乐, 高隽, 谢昭.一种融合神经稀疏编码机制的层次目标识别算法[J].中国图象图形学报, 2010, 15(10):1521-1529. [DOI:10.11834/jig.20101004]

Li Z M, Zhou C C, Gong Y H, et al. Saliency object detection based on domain transform and contour detection[J]. Journal of Computer-Aided Design&Computer Graphics, 2018, 30(8):1457-1465.

李宗民, 周晨晨, 宫延河, 等.结合域变换和轮廓检测的显著性目标检测[J].计算机辅助设计与图形学学报, 2018, 30(8):1457-1465. [DOI:10.3724/SP.J.1089.2018. 16778]

Ren Z L, Shakhnarovich G. Image segmentation by cascaded region agglomeration[C]//Proceedings of 2013 IEEE Conference on Computer Vision and Pattern Recognition. Portland, OR, USA: IEEE, 2013: 2011-2018.[ DOI: 10.1109/CVPR.2013.262 http://dx.doi.org/10.1109/CVPR.2013.262 ]

Arbelaez P, Maire M, Fowlkes C, et al. Contour detection and hierarchical image segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2011, 33(5):898-916.[DOI:10.1109/TPAMI.2010.161]

Dollár P, Zitnick C L. Structured forests for fast edge detection[C]//Proceedings of 2013 IEEE International Conference on Computer Vision. Sydney, Australia: IEEE, 2013: 1841-1848.[ DOI: 10.1109/ICCV.2013.231 http://dx.doi.org/10.1109/ICCV.2013.231 ]

Arbeláez P, Pont-Tuset J, Barron J, et al. Multiscale combinatorial grouping[C]//Proceedings of 2014 IEEE Conference on Computer Vision and Pattern Recognition. Columbus, OH, USA: IEEE, 2014: 328-335.[ DOI: 10.1109/CVPR.2014.49 http://dx.doi.org/10.1109/CVPR.2014.49 ]

Bertasius G, Shi J B, Torresani L. DeepEdge: a multi-scale bifurcated deep network for top-down contour detection[C]//Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition. Boston, MA, USA: IEEE, 2015: 4380-4389.[ DOI: 10.1109/CVPR.2015.7299067 http://dx.doi.org/10.1109/CVPR.2015.7299067 ]

Bertasius G, Shi J B, Torresani L. High-for-Low and Low-for-High: efficient boundary detection from deep object features and its applications to high-level vision[C]//Proceedings of 2015 IEEE International Conference on Computer Vision. Santiago, Chile: IEEE, 2015: 504-512.[ DOI: 10.1109/ICCV.2015.65 http://dx.doi.org/10.1109/ICCV.2015.65 ]

Maninis K K, Pont-Tuset J, Arbeláez P, et al. Convolutional oriented boundaries[C]//Proceedings of the 14th European Conference on Computer Vision. Amsterdam, The Netherlands: Springer, 2016: 580-596.[ DOI: 10.1007/978-3-319-46448-0_35 http://dx.doi.org/10.1007/978-3-319-46448-0_35 ]

Felzenszwalb P F, Huttenlocher D P. Efficient graph-based image segmentation[J]. International Journal of Computer Vision, 2004, 59(2):167-181.[DOI:10.1023/b:visi.0000022288.19776.77]

Dollar P, Tu Z, Belongie S. Supervised learning of edges and object boundaries[C]//Proceedings of 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. IEEE, 2006, 2: 1964-1971.[ DOI: 10.1109/CVPR.2006.298 http://dx.doi.org/10.1109/CVPR.2006.298 ]

Alert me when the article has been cited

提交

Ancient mural classification with AlexNet model using feature fusion

Scene classification algorithm of fusing visual perception

Cross-modal feature fusion and detail-enhanced RGB-D salient object detection

Two-stage vision transformer for fusing global and local features in distracted driving behavior recognition

MoLiNet： a local and global information interactive fusion network for improving multi-classification of pathological image artifacts