Scale-aware EfficientDet: real-time pedestrian detection algorithm for automated driving

Xinkai Xu; Yan Ma; Xu Qian; Yan Zhang

doi:10.11834/jig.200445

Object Detection and Tracking | Views : 0 下载量: 43 CSCD: 6

PDF
Export
Share
Collection
Album

Scale-aware EfficientDet: real-time pedestrian detection algorithm for automated driving
Vol. 26, Issue 1, Pages: 93-100(2021)
Received：03 August 2020，

Revised：2020-10-23，

Accepted：30 October 2020，

Published：16 January 2021
DOI： 10.11834/jig.200445
稿件说明：

移动端阅览

DOI：

Xinkai Xu, Yan Ma, Xu Qian, Yan Zhang. Scale-aware EfficientDet: real-time pedestrian detection algorithm for automated driving[J]. Journal of Image and Graphics, 2021, 26(1): 93-100. DOI： 10.11834/jig.200445.

摘要

目的

行人检测是目标检测中的一个基准问题，在自动驾驶等场景有着较大的实用价值，在路径规划和智能避障方面发挥着重要作用。受限于现实的算法功耗和运行效率，在自动驾驶场景下行人检测存在检测速度不佳、遮挡行人检测精度不足和小尺度行人漏检率高等问题，在保证实时性的前提下设计一种适合行人检测的算法，是一项挑战性的工作。

方法

本文旨在解决自动驾驶场景中耗时长、行人遮挡和小尺度行人检测结果精度低的问题，提出了一种尺度注意力并行检测算法（scale-aware and efficient object detection，Scale-aware EfficientDet）：在特征提取与检测中使用了EfficientDet的主干网络，保证算法效率和功耗的平衡；在行人遮挡方面，为了提高模型对遮挡现象的检测精度，引入了可以增强行人与其他物体之间特征差异的损失函数；在提高小目标行人检测精度方面，采用scale-aware双路网络算法来增加对小目标行人的检测精度。

结果

本文选择Caltech行人数据集作为对比数据集，选取YOLO（you only look once）、YOLOv3、SA-FastRCNN（scale-aware fast region-based convolutional neural network）等算法进行对比，在运行效率方面，本文算法在连续输入单帧图像的情况下达到了35帧/s，多图像输入时达到了70帧/s的工作效率；在模型精度测试中，本文算法也略胜一筹。本文算法应用于2020年中国智能汽车大赛中，在安全避障环节皆获得满分。

结论

本文设计的尺度感知的行人检测算法，在EfficientDet高性能检测器的基础上，通过结合损失函数、scale-aware双路子网络的改进，进一步提升了本文检测器的鲁棒性。

Abstract

Objective

Pedestrian detection is a crucial safety factor in autonomous driving scenarios. Consistent pedestrian detection results play a particular role in path planning and pedestrian collision avoidance. In recent years

pedestrian detection algorithms have become a research hotspot in the field of autonomous driving. For the pedestrian detection task

several problems need to be solved. 1) Pedestrian occlusion in traffic scenes. Pedestrian occlusion is a challenging driving safety problem in autonomous driving scenarios. Pedestrians who are obscured by other objects (such as buildings

vehicles

and other pedestrians) are difficult to detect. 2) Small pedestrian detection accuracy needs to be improved. In an autonomous driving environment

the accuracy of pedestrian detection plays a crucial role in vehicle control systems based on vision algorithms. When the vehicle speed is fast

the pedestrians at a long distance need to be detected accurately. With the need for low algorithm power consumption and good operating efficiency

designing an algorithm suitable for pedestrian detection to maintain excellent detection performance under the premise of achieving real-time performance is a difficult problem.

Method

This paper proposed a real-time pedestrian detection algorithm called scale-aware and efficient object detection (Scale-aware EfficientDet) based on EfficientDet

which achieves state-of-the-art performance in object detection. Our approach aimed to solve the problems of high time consumption

pedestrian occlusion

and low accuracy of small pedestrian detection results in autonomous driving scenarios. Most of the computing power and running time of the existing object detection algorithms are consumed in the visual feature extraction stage

so the use of a lightweight feature extraction network is a crucial factor in improving the efficiency of the algorithm. Our method uses EfficientDet in feature extraction to ensure the algorithm's computational efficiency and power consumption balance. Our approach aimed to observe occluded pedestrians precisely. The loss function was introduced to improve the model's detection accuracy of occlusion phenomena. The function can enhance the feature difference between pedestrians and other objects

and reduce the feature difference between occlude pedestrians and normal pedestrians. In terms of improving the accuracy of small target pedestrian detection

we use the scale-aware mechanism to enhance the algorithm's detection accuracy for small target pedestrians.

Result

The Caltech pedestrian dataset was used for model comparison. You only look once (YOLO)

YOLOv3 scale-aware fast region-based convolutional neural network (fast R-CNN)

and other algorithms are selected for comparison. In terms of operating efficiency

our algorithm achieves 35 frame/s with continuous input of a single frame image and a working efficiency of 70 frame/s with multi-image input. In the test of model accuracy

our algorithm is more accurate than YOLOv3

SA-FastRCNN(scale-aware fast region-based convolutional neural network)

EfficientDet

and other algorithms. In the preliminaries and finals of the China Intelligent Vehicle Championship(CIVC) 2020

the safety and obstacle avoidance links all received full marks.

Conclusion

To address the problems of detection speed in pedestrian detection in autonomous driving

this paper designs the scale-aware EfficientDet real-time pedestrian detector

which is based on the efficient and high-precision EfficientDet. Our method solved the insufficient detection accuracy for occluded pedestrians and the high missed detection rate of small-scale pedestrians. In accordance with the occlusion characteristics of pedestrians

the loss function with repulsive force is used to solve the problem of pedestrian occlusion. Considering the significant differences in visual appearance and extracted feature maps between small-scale and large-scale pedestrians

scale-aware networks are used separately to minimize the missed detection rate of small-scale pedestrians. The improvements in these two aspects further improve the robustness of the designed detector. In future work

our methods can be adjusted to improve detection performance

find optimization methods

and improve neural networks. The detection performance and detection accuracy can be further improved to promote its better application in the field of autonomous driving.

关键词

Keywords

references

Ahmed Z, Iniyavan R and Madhan M P. 2019. Enhanced vulnerable pedestrian detection using deep learning//Proceedings of International Conference on Communication and Signal Processing (ICCSP). Chennai, India: IEEE: 971-974[ DOI:10.1109/ICCSP.2019.8697978 http://dx.doi.org/10.1109/ICCSP.2019.8697978 ]

Broggi A, Fascioli A, Fedriga I, Tibaldi A and Rose M D. 2003. Stereo-based preprocessing for human shape localization in unstructured environments//IEEE IV2003 Intelligent Vehicles Symposium. Columbus, USA: IEEE: 410-415[ DOI:10.1109/IVS.2003.1212946 http://dx.doi.org/10.1109/IVS.2003.1212946 ]

Dalal N and Triggs B. 2005. His tograms of oriented gradients for human detection//Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05). San Diego, USA: IEEE: 886-893[ DOI:10.1109/CVPR.2005.177 http://dx.doi.org/10.1109/CVPR.2005.177 ]

Dalal N, Triggs B and Schmid C. 2006. Human detection using oriented histograms of flow and appearance//Proceedings of the 9th European Conference on Computer Vision. Graz, Austria: Springer: 428-441[ DOI:10.1007/11744047_33 http://dx.doi.org/10.1007/11744047_33 ]

Dollar P, Wojek C, Schiele B and Perona P. 2009. Pedestrian detection: a benchmark//Proceedings of 2009 IEEE Conference on Computer Vision and Pattern Recognition. Miami, USA: IEEE: 304-311[ DOI:10.1109/CVPR.2009.5206631 http://dx.doi.org/10.1109/CVPR.2009.5206631 ]

Dollar P, Wojek C, Schiele B and Perona P. 2012. Pedestrian detection:an evaluation of the state of the art. IEEE Transactions on Pattern Analysis and Machine Intelligence, 34(4):743-761[DOI:10.1109/TPAMI. 2011.155]

Gavrila D M and Munder S. 2007. Multi-cue pedestrian detection and tracking from a moving vehicle. International Journal of Computer Vision, 73(1):41-59[DOI:10.1007/s11263-006-9038-7]

Guo J. 2017. Research on Radar Modeling For Vehicle Intelligence. Changchun: Jilin University

郭姣. 2017.面向汽车智能化仿真的雷达模拟研究.长春: 吉林大学

Li J N, Liang X D, Shen S M, Xu T F, Feng J S and Yan S C. 2018. Scale-aware fast R-CNN for pedestrian detection. IEEE Transactions on Multimedia, 20(4):985-996[DOI:10.1109/TMM.2017.2759508]

Liu J, Gao X K, Bao N Y, Tang J and Wu G S. 2017. Deep convolutional neural networks for pedestrian detection with skip pooling//Proceedings of International Joint Conference on Neural Networks (IJCNN). Anchorage, USA: IEEE: 2056-2063[ DOI:10.1109/IJCNN.2017.7966103 http://dx.doi.org/10.1109/IJCNN.2017.7966103 ]

Liu W, Anguelov D, Erhan D, Szegedy C, Reed S, Fu C Y and Berg A C. 2016. SSD: single shot MultiBox detector//Proceedings of the 14th European Conference Computer Vision. Amsterdam, the Netherlands: Springer: 21-37[ DOI:10.1007/978-3-319-46448-0_2 http://dx.doi.org/10.1007/978-3-319-46448-0_2 ]

Milan A, Leal-Taixe L, Reid I, Roth S and Schindler K. 2016. MOT16: a benchmark for multi-object tracking[EB/OL ] .[2020-07-23 ] . https://arxiv.org/pdf/1603.00831.pdf https://arxiv.org/pdf/1603.00831.pdf

Milton A A. 2019. Towards pedestrian detection using RetinaNet in ECCV 2018 wider pedestrian detection challenge[EB/OL ] .[2020-07-23 ] . https://arxiv.org/pdf/1902.01031.pdf https://arxiv.org/pdf/1902.01031.pdf

Nam W, Dollár P and Han J H. 2014. Local decorrelation for improved pedestrian detection//Proceedings of the 27th International Conference on Neural Information Processing Systems. Montreal, Canada: ACM: 424-432[ DOI:10.5555/2968826.2968874 http://dx.doi.org/10.5555/2968826.2968874 ]

Redmon J, Divvala S, Girshick R and Farhadi A. 2016. You only look once: unified, real-time object detection//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, USA: IEEE: 779-788[ DOI:10.1109/CVPR.2016.91 http://dx.doi.org/10.1109/CVPR.2016.91 ]

Sermanet P, Kavukcuoglu K, Chintala S and Lecun Y. 2013. Pedestrian detection with unsupervised multi-stage feature learning//Proceedings of 2013 IEEE Conference on Computer Vision and Pattern Recognition. Portland, USA: IEEE: 3626-3633[ DOI:10.1109/CVPR.2013.465 http://dx.doi.org/10.1109/CVPR.2013.465 ]

Tan M X and Le Q V. 2019. EfficientNet: rethinking model scaling for convolutional neural networks[EB/OL ] .[2020-07-23 ] . https://arxiv.org/pdf/1905.11946.pdf https://arxiv.org/pdf/1905.11946.pdf

Tan M X, Pang R M and Le Q V. 2020. EfficientDet: scalable and efficient object detection//Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle, USA: IEEE: 10781-10790[ DOI:10.1109/cvpr42600.2020.01079 http://dx.doi.org/10.1109/cvpr42600.2020.01079 ]

Tian Y L, Luo P, Wang X G and Tang X O. 2015. Deep learning strong parts for pedestrian detection//Proceedings of 2015 IEEE International Conference on Computer Vision. Santiago, Chile: IEEE: 1904-1912[ DOI:10.1109/iccv.2015.221 http://dx.doi.org/10.1109/iccv.2015.221 ]

Wang X L, Xiao T T, Jiang Y N, Shao S, Sun J and Shen C H. 2018. Repulsion loss: detecting pedestrians in a crowd//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, USA: IEEE: 7774-7783[ DOI:10.1109/CVPR.2018.00811 http://dx.doi.org/10.1109/CVPR.2018.00811 ]

Yang F, Choi W and Lin Y Q. 2016. Exploit all the layers: fast and accurate C NN object detector with scale dependent pooling and cascaded rejection classifiers//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, USA: IEEE: 2129-2137[ DOI:10.1109/CVPR.2016.234 http://dx.doi.org/10.1109/CVPR.2016.234 ]

Zhang L L, Lin L, Liang X D and He K M. 2016. Is faster R-CNN doing well for pedestrian detection?//Proceedings of the 14th European Conference on Computer Vision-ECCV 2016. Amsterdam, The Netherlands: Springer: 443-457[ DOI:10.1007/978-3-319-46475-6_28 http://dx.doi.org/10.1007/978-3-319-46475-6_28 ]

Zhang S S, Benenson R and Schiele B. 2017. CityPersons: a diverse dataset for pedestrian detection//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, USA: IEEE: 4457-4465[ DOI:10.1109/CVPR.2017.474 http://dx.doi.org/10.1109/CVPR.2017.474 ]

Zhou C L, Yang M and Yuan J S. 2019. Discriminative feature transformation for occluded pedestrian detection//Proceedings of 2019 IEEE/CVF International Conference on Computer Vision. Seoul, South Korea: IEEE: 9557-9566[ DOI:10.1109/ICCV.2019.00965 http://dx.doi.org/10.1109/ICCV.2019.00965 ]

Zhuang C B, Lei Z and Li S Z. 2020. SADet: learning an efficient and accura te pedestrian detector[EB/OL ] .[2020-07-26 ] . https://arxiv.org/pdf/2007.13119.pdf https://arxiv.org/pdf/2007.13119.pdf

Alert me when the article has been cited

提交

Review of deep learning algorithms for small object detection in optical remote sensing images

Integrated generation-detection approach for pedestrian recognition in dark-light conditions

Improved YOLOv7-tiny with long short range dependency feature pyramid network

Adaptive slicing-assisted enhanced method for small object detection

Research progress of unmanned mobile vision technology for complex dynamic scenes