3D object detection based on domain attention and dilated convolution

Juan Yan; Zhijun Fang; Yongbin Gao

doi:10.11834/jig.190378

Image Understanding and Computer Vision | Views : 0 下载量: 17 CSCD: 3

PDF
Export
Share
Collection
Album

3D object detection based on domain attention and dilated convolution
Vol. 25, Issue 6, Pages: 1221-1234(2020)
Received：16 August 2019，

Revised：2019-11-8，

Published：16 June 2020
DOI： 10.11834/jig.190378
稿件说明：

移动端阅览

DOI：

Juan Yan, Zhijun Fang, Yongbin Gao. 3D object detection based on domain attention and dilated convolution[J]. Journal of Image and Graphics, 2020, 25(6): 1221-1234. DOI： 10.11834/jig.190378.

摘要

目的

通过深度学习卷积神经网络进行3维目标检测的方法已取得巨大进展，但卷积神经网络提取的特征既缺乏不同区域特征的依赖关系，也缺乏不同通道特征的依赖关系，同时难以保证在无损空间分辨率的情况下扩大感受野。针对以上不足，提出了一种结合混合域注意力与空洞卷积的3维目标检测方法。

方法

在输入层融入空间域注意力机制，变换输入信息的空间位置，保留需重点关注的区域特征；在网络中融入通道域注意力机制，提取特征的通道权重，获取关键通道特征；通过融合空间域与通道域注意力机制，对特征进行混合空间与通道的混合注意。在特征提取器的输出层融入结合空洞卷积与通道注意力机制的网络层，在不损失空间分辨率的情况下扩大感受野，根据不同感受野提取特征的通道权重后进行融合，得到全局感受野的关键通道特征；引入特征金字塔结构构建特征提取器，提取高分辨率的特征图，大幅提升网络的检测性能。运用基于二阶段的区域生成网络，回归定位更准确的3维目标框。

结果

KITTI（A project of Karlsruhe Institute of Technology and Toyota Technological Institute at Chicago）数据集中的实验结果表明，在物体被遮挡的程度由轻到高时，对测试集中的car类别，3维目标检测框的平均精度

值分别为83.45%、74.29%、67.92%，鸟瞰视角2维目标检测框的平均精度

BEV

值分别为89.61%、87.05%、79.69%；对pedestrian和cyclist类别，

和

BEV

值同样比其他方法的检测结果有一定优势。

结论

本文提出的3维目标检测网络，一定程度上解决了3维检测任务中卷积神经网络提取的特征缺乏视觉注意力的问题，从而使3维目标检测更有效地运用于室外自动驾驶。

Abstract

Objective

With the continuous development of convolutional neural network (CNN) used in deep learning in recent years

3D object detection networks based on deep learning have also made outstanding development. 3D object detection aims to identify the class

location

orientation

and size of a target object in 3D space. It is widely used in the visual field

such as autonomous driving

intelligent monitoring

and medical analysis. The feature extracted by a deep learning network is important in detection accuracy. The detection task is similar to human vision; that is

it also needs to distinguish the difference between the background and the objects. In human vision

attention is given to target objects

while the background is disregarded. Therefore

paying more attention to the target area and less attention to the background area is better when performing object detection in an image. However

a CNN does not distinguish which areas and channels in an image should be given more and less attention. Thus

the features extracted by a CNN not only lack the dependence relationship between different regions but also the dependence relationship between different channels. The current 3D object detection method based on a deep learning network uses a combination of pooling layers behind the multilayer convolution layer. These network structures generally use maximum or averaging pooling in feature maps. They aim to adjust the receptive field size of the extracted features. However

transforming the receptive field of the features of the pooling layers must be performed by removing some information

causing a considerable loss of feature information. Information loss may result in detected errors. Therefore

a CNN should expand the receptive field without losing information

obtaining good detection results. To address the shortcomings of the aforementioned 3D target detection methods

this study proposes a two-stage 3D object detection network that combines mixed domain attention and dilated convolution.

Method

In this study

a 3D object detection network based on a deep learning network is built. Integrating the spatial domain attention mechanism into the input layer of the network transforms the spatial position of the input information

preserving regional features that require more attention. Incorporating the channel domain attention mechanism into the network computes the channel weights of the extracted features

obtaining the key channel features. The features are mixed by combining the aforementioned spatial and channel domain attention mechanisms. Second

the output layer of the feature extractor integrates the network layer that is combined with the dilated convolution and the channel domain attention mechanism

and thus

our network can expand the receptive field of the extracted features without losing spatial resolution. In accordance with the different obtained receptive fields

the features can determine their channel weights and then fuse these feature weights through different schemes to obtain the channel weights of their global receptive fields and identify key channel features. In addition

the feature pyramid network structure is introduced to construct the feature extractor of our network

through which our network can extract high-resolution feature maps

considerably improving the detection performance of our network. Lastly

our network architecture is based on a two-stage region proposal network

which can regress to accurate 3D bounding boxes.

Result

A series of experiments has been conducted on the KITTI(A project of Karlsruhe Institute of Technology and Toyota Technological Institute at Chicago) dataset using the method proposed in this study. Cases wherein the object is slightly to severely occluded are denoted as "easy"

"moderate"

and "hard" in the tables. In the car class in the test set

the values of

that represents average accuracy of 3D object detection box obtained are 83.45%

74.29%

and 67.92%; and the values of

BEV

that represents average accuracy of 2D detection box from bird's eye view obtained are 89.61%

87.05%

and 79.69%. In the pedestrian class

the values of

obtained are 52.23%

44.91%

and 41.64%; and the values of

BEV

obtained are 59.73%

53.97%

and 49.62%. In the cyclist class

the values of

obtained are 65.02%

54.38%

and 47.97%; and the values of

BEV

obtained are 69.13%

59.69%

and 52.11%. We also perform ablation experiments on the test set. The experiment results show that in the car class and relative to the proposed method

the average value of

obtained after removing the pyramid structure is reduced by approximately 6.09%

the average value of

obtained after removing the mixed domain attention structure is reduced by approximately 0.99%

and the average value of

obtained after removing the dilated convolution structure is reduced by approximately 0.71%.

Conclusion

For the research on 3D object detection task

we propose a two-stage 3D object detection network that combines dilated convolution and mixed domain attention. The experiment results show that the proposed method outperforms several existing state-of-the-art 3D object detection methods and obtains accurate detection results

and it can be effectively applied to outdoor automatic driving.

关键词

Keywords

references

Charles R Q, SuH, Kaichun M and Guibas L J. 2017. PointNet: deep learning on point sets for 3D classification and segmentation//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu, HI: IEEE: 77-85[DOI: 10.1109/CVPR.2017.16 http://dx.doi.org/10.1109/CVPR.2017.16 ]

Chen X Z, Ma H M, Wan J, Li B and Xia T. 2017. Multi-view 3D object detection network for autonomous driving//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, HI: IEEE: 6526-6534[DOI: 10.1109/CVPR.2017.691 http://dx.doi.org/10.1109/CVPR.2017.691 ]

Dalal N and Triggs B. 2005. Histograms of oriented gradients for human detection//Proceedings of 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05), San Diego, CA, USA: IEEE: 886-893[DOI: 10.1109/CVPR.2005.177 http://dx.doi.org/10.1109/CVPR.2005.177 ]

Geiger A, Lenz P, Stiller C and Urtasun R. 2013. Vision meets robotics:the KITTI dataset. International Journal of Robotics Research, 32(11):1231-1237[DOI:10.1177/0278364913491297]

Hu J, Shen L, Albanie S, Sun G and Wu E H. 2017. Squeeze-and-Excitation Networks. IEEE Transactions on Pattern Analysis and Machine Intelligence: 1-13[DOI: 10.1109/TPAMI.2019.2913372 http://dx.doi.org/10.1109/TPAMI.2019.2913372 ]

Ku J, Mozifian M, Lee J, Harakeh A and Waslander S L. 2018. Joint 3D proposal generation and object detection from view aggregation//Proceedings of 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). Madrid: IEEE: 1-8[DOI: 10.1109/IROS.2018.8594049 http://dx.doi.org/10.1109/IROS.2018.8594049 ]

Li B, Zhang T L and Xia T. 2016. Vehicle detection from 3d lidar using fully convolutional network. Proceedings of Robotics: Science and Systems: #42[DOI: 10.15607/RSS.2016.XII.042 http://dx.doi.org/10.15607/RSS.2016.XII.042 ]

Li B. 2017. 3D fully convolutional network for vehicle detection in point cloud//Proceedings of 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). Vancouver, BC: IEEE: 1513-1518[DOI: 10.1109/IROS.2017.8205955 http://dx.doi.org/10.1109/IROS.2017.8205955 ]

Lin T Y, Dollár P, Girshick R, He K M, Hariharan B and Belongie S. 2017. Feature pyramid networks for object detection//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, HI: IEEE: 936-944[DOI: 10.1109/CVPR.2017.106 http://dx.doi.org/10.1109/CVPR.2017.106 ]

Lowe D G. 1999. Object recognition from local scale-invariant features//Proceedings of the 7th IEEE International Conference on Computer Vision. Kerkyra, Greece: IEEE: 1150-1157[DOI: 10.1109/ICCV.1999.790410 http://dx.doi.org/10.1109/ICCV.1999.790410 ]

Neubeck A and Van Gool L J. 2006. Efficient Non-Maximum Suppression//Proceedings of the 18th International Conference on Pattern Recognition, Hong Kong, China: IEEE: 850-855[DOI: 10.1109/ICPR.2006.479 http://dx.doi.org/10.1109/ICPR.2006.479 ]

Papageorgiou C P, Oren M and Poggio T. 1998. A general framework for object detection//Proceedings of the 6th International Conference on Computer Vision. Bombay, India: IEEE: 555-562[DOI: 10.1109/ICCV.1998.710772 http://dx.doi.org/10.1109/ICCV.1998.710772 ]

Qi C R, Liu W, Wu C X, Su H and Guibas L J. 2018. Frustum PointNets for 3D object detection from RGB-D data//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, UT: IEEE: 918-927[DOI: 10.1109/CVPR.2018.00102 http://dx.doi.org/10.1109/CVPR.2018.00102 ]

Redmon J, Divvala S, Girshick R and Farhadi A. 2016. You only look once: unified, real-time object detection//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas, NV: IEEE: 779-788[DOI: 10.1109/CVPR.2016.91 http://dx.doi.org/10.1109/CVPR.2016.91 ]

Ren S Q, He K M, Girshick R and Sun J. 2017. Faster R-CNN:towards real-time object detection with region proposal networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(6):1137-1149[DOI:10.1109/TPAMI.2016.2577031]

Simon M, Amende K, Kraus A, Honer J, Sämann T, Kaulbersch H, Milz S and Gross H M. 2019. Complexer-YOLO: real-time 3D object detection and tracking on semantic point clouds[EB/OL].[2019-10-24] . https://arxiv.org/pdf/1904.07537.pdf https://arxiv.org/pdf/1904.07537.pdf

Simonyan K and Zisserman A. 2014. Very deep convolutional networks for large-scale image recognition[EB/OL].[2019-10-24] . https://arxiv.org/pdf/1409.1556.pdf https://arxiv.org/pdf/1409.1556.pdf

Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez A N, KaiserŁand Polosukhin I. 2017. Attention is all you need[EB/OL].[2019-10-24] . https://arxiv.org/pdf/1706.03762.pdf https://arxiv.org/pdf/1706.03762.pdf

Yu F and Koltun V. 2015. Multi-scale context aggregation by dilated convolutions[EB/OL].[2019-10-24] . https://arxiv.org/pdf/1511.07122 https://arxiv.org/pdf/1511.07122 . pdf

Zhou J, Tan X, Shao Z W and Ma L Z. 2019. FVNet: 3D front-view proposal generation for real-time object detection from point clouds//Proceedings of the 12th International Congress on Image and Signal Processing, BioMedical Engineering and Informatics (CISP-BMEI). Suzhou, China: IEEE: 1-8[DOI: 10.1109/CISP-BMEI48845.2019.8965844 http://dx.doi.org/10.1109/CISP-BMEI48845.2019.8965844 ]

Zhou Y and Tuzel O. 2018. Voxelnet: end-to-end learning for point cloud based 3d object detection//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, UT: IEEE: 4490-4499[DOI: 10.1109/CVPR.2018.00472 http://dx.doi.org/10.1109/CVPR.2018.00472 ]

Alert me when the article has been cited

提交

Face age synthesis fusing channel-coordinate attention mechanism and parallel dilated convolution

Deep learning based backlight image enhancement method derived of zero-reference samples

Single image rain removal based on multi scale progressive residual network

Region-level channel attention for single image super-resolution combining high frequency loss

Object tracking using enhanced second-order network modulation