浏览全部资源
扫码关注微信
纸质出版日期： 2023-02-16 ，
录用日期： 2022-01-19
移动端阅览
鲍文霞, 孙强, 梁栋, 胡根生, 杨先军. 结合感受野模块与并联RPN网络的火焰检测[J]. 中国图象图形学报, 2023,28(2):418-429.
Wenxia Bao, Qiang Sun, Dong Liang, Gensheng Hu, Xianjun Yang. Flame detection combined with receptive field and parallel RPN[J]. Journal of Image and Graphics, 2023,28(2):418-429.
目的
2
准确快速的火焰检测技术在早期火灾预警中具有重要的实际应用价值。为了降低伪火类物体引起的误警率以及早期小火焰的漏检率，本文设计了一种结合感受野(receptive field，RF)模块与并联区域建议网络(parallel region proposal network，PRPN)的卷积神经网络(receptive field and parallel region proposal convolutional neural network，R-PRPNet)用于火焰检测。
方法
2
R-PRPNet主要由特征提取模块、并联区域建议网络和分类器3部分组成。特征提取模块在MobileNet卷积层的基础上，通过嵌入感受野RF模块扩大感受野捕获更丰富的上下文信息，从而提取更具鉴别性的火焰特征，降低伪火类物体引起的误警率；并联区域建议网络与特征提取模块后端的多尺度采样层连接，使用3×3和5×5的全卷积进一步拓宽多尺度锚点的感受野宽度，提升PRPN对不同尺度火焰的检测能力，解决火灾发生初期的小火焰漏检问题；分类器由softmax和smooth L1分别实现分类与回归。在R-PRPNet训练过程中，将伪火类物体作为负样本进行负样本微调，以更好区分伪火类物体。
结果
2
在包括室内、建筑物、森林和夜晚等场景火焰数据以及包括灯光、晚霞、火烧云和阳光等伪火类数据的自建数据集上对所提方法进行测试，在火焰检测任务中，准确度为98.07%，误警率为4.2%，漏检率为1.4%。消融实验结果表明，R-PRPNet较基线网络在漏检率和误警率上分别降低了4.9%和21.72%。与传统火焰检测方法相比，R-PRPNet在各项指标上均优于边缘梯度信息和聚类等方法。性能较几种目标检测算法有所提升，其中相较于YOLOX-L，误警率和漏检率分别降低了22.2%和5.2%。此外，本文在不同场景火焰下进行测试，都有较稳定的表现。
结论
2
本文方法有效降低了火焰检测中的误警率和漏检率，并可以满足火焰检测的实时性和准确性需求。
Objective
2
Early flame detection is essential for quick response event through minimizing casualties and damage. Smoky and flaming alarms have been using in indoor-scenario in common. However
the challenging issue of most traditional physical sensors is limited to the fire source-near problem and cannot be meet the requirement of outdoor-scene flame detection. Real-time image detection has been developing in terms of image processing and machine learning technique. However
the flame shape
size
and color can be varied intensively
and there are a plenty of pseudo-fire objects (very similar to the features of the flame color) in the natural environment. To distinguish real flames from pseudo flames precisely
the detection model has been developing dramatically. Image processing and machine learning methods can be divided into three categories: 1) traditional image processing
2) machine learning
and 3) deep learning. Traditional image processing and machine learning is often concerned of design-manual of flame features
which is not quantitative and poor matching to complex background images. Thanks to the self-learning and deep learning techniques
current flame-based detection and interpretation has been facilitating. First
convolution-depth can be used to interpret small-scale areas less than 32×32 missing information on the feature map. Second
deep learning models can be applied to detect color features similar to the object-targeted and the misjudgment-caused
while it is restricted of small target and color-similar feature in flame-detection interpretation. In order to alleviate the pseudo-fire-objects-derived false alarm rate and the missed detection rate of early small flames
we develop a receptive field module based (RF-module-based) convolutional neural network (CNN) and the parallel region proposal network (PRPN) is designed for flame detection
called R-PRPNet.
Method
2
The R-PRPNet is mainly composed of three parts: 1) feature extraction module
2) region-parallel network
and 3) classifier. The feature extraction module is focused on convolutional layers of lightweight MobileNet
which makes our algorithm-proposed run faster without the performance loss of flame detection. To extract more discriminative flame features and alleviate the high pseudo-fire-objects-derived false alarm rate
the RF module is embedded into this module for receptive field-expanded and richer context information-captured. The features of multi-scale flames in flaming duration are combined with the region-parallel network. To connect the PRPN
a multi-scale sampling layer is established at the back of the feature extraction module. Furthermore
we use 3×3 and 5×5 full convolution to broaden the receptive field width of multi-scale anchor points
which can improve multi-scale flames detection ability
and resolve the problem of detection-missed of small flames in the early stage of a fire. To achieve classification and regression
the classifier is implemented through softmax and smooth L1
and the final flame category and position information is produced in the image.
Result
2
Our method is tested on the multiple datasets-self-built
such as indoor
building
forest and night scene flame data and scenario-based pseudo fire data like light
sunset glow
burning cloud and sunshine. The Faster region CNN (R-CNN) with MobileNet as the backbone network is used as the benchmark. The RF module-based network is compared to the network-benched
which can be used to learn more flame features-discriminative
and the detection-missed rate and alarm-false rate are lower by 1.1% and 0.43% of each. The network-benched is melted into parallel RPN (PRPN) on the basis of the RF module
which improves the networks recognition rate of multi-scale flames effectively. The recall rate is increased by 1.7%
and the detection-missed rate is decreased by 1.7%. The RF module is compared via the negative sample fine-tuning strategy. The pseudo-fire features are enriched through negative sample fine-tuning strategy
and the network classification performance is interpreted and improved for real flames and pseudo-fire objects. The false alarm rate can be decreased by 21% as well based on the two components mentioned above. The comparative analysis is carried out with three detection methods as following: 1) for traditional flame detection methods : R-PRPNet is better than edge gradient information and clustering methods in all indexes. 2) For classical target detection algorithms: the performance is improved as well. 3) For YOLOX-L
the false alarm rate and detection-missed rate are reduced by 22.2% and 5.2%
respectively. The final results are reached to 98.07% (accuracy)
4.2% (alarm-false rate) and 1.4% (detection-missed rate) of each.
Conclusion
2
We design a CNN for flame detection in related to a receptive field module and the parallel RPN. To expand the receptive field and extract more contextual information-captured discriminative flame features
the RF module is embedded into the feature extraction module of the network
and flame features are melted into through splicing
downsampling and elements-added. Comparing the proposed network with some classic convolutional neural networks and traditional methods
the experimental results show that our network-proposed can extract complex image flame features for multiple scenarios automatically
and it can be filtered pseudo-fire objects accurately.
火焰检测深度学习感受野(RF)并联区域建议网络(PRPN)负样本微调
flame detectiondeep learningreceptive field (RF)parallel region proposal network (PRPN)negative sample fine-tuning
Bochkovskiy A, Wang C Y and Liao H Y M. 2020. YOLOv4: optimal speed and accuracy of object detection [EB/OL]. [2021-09-21].https://arxiv.org/pdf/2004.10934.pdfhttps://arxiv.org/pdf/2004.10934.pdf
Celik T. 2010. Fast and efficient method for fire detection using image processing. ETRI Journal, 32(6): 881-890 [DOI: 10.4218/etrij.10.0109.0695]
Çelik T and Demirel H. 2009. Fire detection in video sequences using a generic color model. Fire Safety Journal, 44(2): 147-158 [DOI: 10.1016/j.firesaf.2008.05.005]
Chakraborty I and Paul T K. 2010. A hybrid clustering algorithm for fire detection in video and analysis with color based thresholding method//Proceedings of 2010 International Conference on Advances in Computer Engineering. Bangalore, India: IEEE: 277-280 [DOI: 10.1109/ACE.2010.12http://dx.doi.org/10.1109/ACE.2010.12]
Chen T H, Wu P H and Chiou Y C. 2004. An early fire-detection method based on image processing//Proceedings of 2004 International Conference on Image Processing. Singapore, Singapore: IEEE: 1707-1710 [DOI: 10.1109/ICIP.2004.1421401http://dx.doi.org/10.1109/ICIP.2004.1421401]
Foggia P, Saggese A and Vento M. 2015. Real-time fire detection for video-surveillance applications using a combination of experts based on color, shape, and motion. IEEE Transactions on Circuits and Systems for Video Technology, 25(9): 1545-1556 [DOI: 10.1109/TCSVT.2015.2392531]
Frizzi S, Kaabi R, Bouchouicha M, Ginoux J M, Moreau E and Fnaiech F. 2016. Convolutional neural network for video fire and smoke detection//Proceedings of the IECON 2016-42nd Annual Conference of the IEEE Industrial Electronics Society. Florence, Italy: IEEE: 877-882 [DOI: 10.1109/IECON.2016.7793196http://dx.doi.org/10.1109/IECON.2016.7793196]
Ge Z, Liu S T, Wang F, Li Z M and Sun J. 2021. YOLOX: exceeding YOLO series in 2021 [EB/OL]. [2021-08-06].https://arxiv.org/pdf/2107.08430.pdfhttps://arxiv.org/pdf/2107.08430.pdf
Habiboǧlu Y H, Güenay O and Çetin A E. 2012. Covariance matrix-based fire and flame detection method in video. Machine Vision and Applications, 23(6): 1103-1113 [DOI: 10.1007/s00138-011-0369-1]
Han X F, Jin J S, Wang M J, Jiang W, Gao L and Xiao L P. 2017. Video fire detection based on Gaussian mixture model and multi-color features. Signal, Image and Video Processing, 11(8): 1419-1425 [DOI: 10.1007/s11760-017-1102-y]
Hashemzadeh M and Zademehdi A. 2019. Fire detection for video surveillance applications using ICA K-medoids-based color model and efficient spatio-temporal visual features. Expert Systems with Applications, 130: 60-78 [DOI: 10.1016/j.eswa.2019.04.019]
He K M, Zhang X Y, Ren S Q and Sun J. 2016. Deep residual learning for image recognition//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, USA: IEEE: 770-778 [DOI: 10.1109/CVPR.2016.90http://dx.doi.org/10.1109/CVPR.2016.90]
Howard A G, Zhu M L, Chen B, Kalenichenko D, Wang W J, Weyand T, Andreetto M and Adam H. 2017. MobileNets: efficient convolutional neural networks for mobile vision applications [EB/OL]. [2021-04-17].https://arxiv.org/pdf/1704.04861.pdfhttps://arxiv.org/pdf/1704.04861.pdf
Khatami A, Mirghasemi S, Khosravi A, Lim C P and Nahavandi S. 2017. A new PSO-based approach to fire flame detection using K-medoids clustering. Expert Systems with Applications, 68: 69-80 [DOI: 10.1016/j.eswa.2016.09.021]
Kim B and Lee J. 2019. A video-based fire detection using deep learning models. Applied Sciences, 9(14): #2862 [DOI: 10.3390/app9142862]
Kim J J, Ryu J K, Kwak D K and Byun S J. 2018. A study on flame detection using faster R-CNN and image augmentation techniques. Journal of IKEEE, 22(4): 1079-1087 [DOI: 10.7471/ikeee.2018.22.4.1079]
Kong S G, Jin D L, Li S Z and Kim H. 2016. Fast fire flame detection in surveillance video using logistic regression and temporal smoothing. Fire Safety Journal, 79: 37-43 [DOI: 10.1016/j.firesaf.2015.11.015]
Liu S T, Huang D and Wang Y H. 2018. Receptive field block net for accurate and fast object detection//Proceedings of the 15th European Conference on Computer Vision. Munich, Germany: Springer: 404-419 [DOI: 10.1007/978-3-030-01252-6_24http://dx.doi.org/10.1007/978-3-030-01252-6_24]
Liu W, Anguelov D, Erhan D, Szegedy C, Reed S, Fu C Y and Berg A C. 2016. SSD: single shot MultiBox detector//Proceedings of the 14th European Conference on Computer Vision. Amsterdam, the Netherlands: Springer: 21-37 [DOI: 10.1007/978-3-319-46448-0_2http://dx.doi.org/10.1007/978-3-319-46448-0_2]
Ouyang J N, Bu L P, Yang Z K and Wang T. 2018. An early flame identification method based on edge gradient feature//Proceedings of the 2nd IEEE Advanced Information Management, Communicates, Electronic and Automation Control Conference. Xi′an, China: IEEE: 642-646 [DOI: 10.1109/IMCEC.2018.8469419http://dx.doi.org/10.1109/IMCEC.2018.8469419]
Redmon J and Farhadi A. 2018. YOLOv3: an incremental improvement [EB/OL]. [2012-04-08].https://arxiv.org/pdf/1804.02767.pdfhttps://arxiv.org/pdf/1804.02767.pdf
Ren S Q, He K M, Girshick R and Sun J. 2017. Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(6): 1137-1149 [DOI: 10.1109/TPAMI.2016.2577031]
Shen D Q, Chen X, Nguyen M and Yan W Q. 2018. Flame detection using deep learning//Proceedings of the 4th International Conference on Control, Automation and Robotics. Auckland, New Zealand: IEEE: 416-420 [DOI: 10.1109/ICCAR.2018.8384711http://dx.doi.org/10.1109/ICCAR.2018.8384711]
Szegedy C, Vanhoucke V, Ioffe S, Shlens J and Wojna Z. 2016. Rethinking the inception architecture for computer vision//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, USA: IEEE: 2818-2826 [DOI: 10.1109/CVPR.2016.308http://dx.doi.org/10.1109/CVPR.2016.308]
Toreyin B U, Dedeoglu Y and Cetin A E. 2005. Flame detection in video using hidden Markovmodels//Proceedings of 2005 IEEE International Conference on Image Processing 2005. Genova, Italy: IEEE: 1230-1233 [DOI: 10.1109/ICIP.2005.1530284http://dx.doi.org/10.1109/ICIP.2005.1530284]
Truong T X and Kim J M. 2012. Fire flame detection in video sequences using multi-stage pattern recognition techniques. Engineering Applications of Artificial Intelligence, 25(7): 1365-1372 [DOI: 10.1016/j.engappai.2012.05.007]
Zhang Q J, Xu J L, Xu L and Guo H F. 2016. Deep convolutional neural networks for forest fire detection//Proceedings of 2016 International Forum on Management, Education and Information Technology Application. Guangzhou, China: Atlantis Press: 568-575 [DOI: 10.2991/ifmeita-16.2016.105http://dx.doi.org/10.2991/ifmeita-16.2016.105]
Zhao T and Wu X Q. 2019. Pyramid feature attention network for saliency detection//Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach, USA: IEEE: 3080-3089 [DOI: 10.1109/CVPR.2019.00320http://dx.doi.org/10.1109/CVPR.2019.00320]
相关作者
相关机构
京公网安备11010802024621