Smoke detection by trajectories in condensed images for early fire warning

Sheng Luo; Xiang Zhang; Jie Hu; Muchou Wang; Xiaoqing Zhang

doi:10.11834/jig.190217

Fire and Smoke Special Forum | Views : 0 下载量: 243 CSCD: 0

PDF
Export
Share
Collection
Album

Smoke detection by trajectories in condensed images for early fire warning
Vol. 24, Issue 10, Pages: 1648-1657(2019)
Received：22 May 2019，

Revised：2019-7-9，

Accepted：16 July 2019，

Published：16 October 2019
DOI： 10.11834/jig.190217
稿件说明：

移动端阅览

DOI：

Sheng Luo, Xiang Zhang, Jie Hu, Muchou Wang, Xiaoqing Zhang. Smoke detection by trajectories in condensed images for early fire warning[J]. Journal of Image and Graphics, 2019, 24(10): 1648-1657. DOI： 10.11834/jig.190217.

摘要

目的

检测烟雾可以预警火灾。视频监控烟雾比传统的单点探测器监控范围更广、反应更灵敏，对环境和安装的要求也更低。但是目前的烟雾检测算法，无论是利用烟雾的色彩、纹理等静态特征和飘动、形状变化或者频域变化等动态特征的传统方法，还是采用卷积神经网络、循环神经网络等深度学习的方法，准确率和敏感性都不高。

方法

本文着眼于烟雾的升腾特性，根据烟雾运动轨迹的右倾直线特性、连续流线型特性、低频特性、烟源固定特性和比例特性，采用切片的方式用卷积神经网络（CNN）抽取时间压缩轨迹的动态特征，用循环神经网络（RNN）抽取长程的时间关联关系，采用分块的方式提高空间分辨能力，能准确、迅速地识别烟雾轨迹并发出火灾预警。

结果

对比CNN、C3D（3d convolutional networks）、traj+SVM（trajectory by support vector machine）、traj+RNNs（trajectory by recurrent neural network）和本文方法traj+CNN+RNNs（trajectory by convolutional neural networks and recurrent neural network）以验证效果。CNN和C3D先卷积抽取特征，后分类。traj+SVM采用SVM辨识视频时间压缩图像中的烟雾轨迹，traj+RNNs采用RNNs分辨烟雾轨迹，traj+CNN+RNNs结合CNN和RNNs识别轨迹。实验表明，与traj+SVM相比，traj+CNN+RNNs准确率提高了35.2%，真负率提高15.6%。但是深度学习的方法往往计算消耗很大，traj+CNN+RNNs占用内存2.31 GB，网络权重261 MB，前向分析时帧率49帧/s，而traj+SVM帧率为178帧/s。但与CNN、C3D相比，本文方法较轻较快。为了进一步验证方法的有效性，采用一般方法难以识别的数据进一步测试对比这5个方法。实验结果表明，基于轨迹的方法仍然取得较好的效果，traj+CNN+RNNs的准确率、真正率、真负率和帧率还能达到0.853、0.847、0.872和52帧/s，但是CNN、C3D的准确率下降到0.585、0.716。

结论

从视频的时间压缩轨迹可以辨认出烟雾的轨迹，即便是早期的弱小烟雾也能准确识别，因此traj+CNN+RNNs辨识轨迹的方法有助于预警早期火灾。本文方法能够在较少的资源耗费下大幅度提高烟雾检测的准确性和敏感性。

Abstract

Objective

Smoke detection by surveillance cameras is reasonable to warn fire. This technology has many advantages compared with other traditional point detectors. Wide areas could be covered

rapid respondence could be available

and installation and maintenance requirements could be less. However

the current smoke detection algorithms are unsatisfying in terms of accuracy and sensitivity due to the varying colors

shapes

and textures of smoke. The traditional studies focus on designing handcrafted features that extract such static features as colors

shapes

and textures and dynamic ones

including shape deforming

drifting

and frequency shifting. This task is time consuming. Although the algorithm exhibits good characteristics

maintaining its robustness for all environments is difficult. The detection effectiveness often sharply descends when these methods are applied in different environments. The fashionable methods

such as convolution neural network (CNN)

recurrent neural network (RNN)

and other statistical methods

are based on deep learning. However

applying these methods is difficult because the surveillance platforms have limited resources. These networks are also unsatisfying in terms of accuracy and sensitivity.

Method

The proposed method utilizes trajectories in condensed images

which are summed in horizontal and vertical directions for all video pixels. Smoke trajectories in condensed images are always right-leaning

straightly linear

proportional

and streamline-like with low frequencies and fixed starting points. Accordingly

surveillance videos are summed into condensed images

sliced

and then fed into CNN to extract features to find the long-term relationship by RNN. Partitioning strategy is also adopted to improve sensitivity. Therefore

the method uses not only the trajectory shapes but also the short- and long-range relationships in the time domain to detect the existence of smoke in videos.

Result

Controlled experiments of CNN

C3D(3d convolutional networks)

traj + SVM(support vector machine)

traj + RNNs

and traj + CNN + RNNs are conducted. The CNN and C3D methods are typical deep learning networks that initially extract features and then make judgments. The traj + SVM method detects smoke trajectories by traditional SVM algorithm

the traj + RNNs method finds smoke trajectories by RNNs

and the traj + CNN + RNNs method recognizes smoke trajectories by combining CNN and RNNs

which is the proposed method. The accuracy of the traj + CNN + RNNs method is increased by 35.2% compared with that of traj + SVM

and the real negative rate is increased by 15.6%. However

the computing cost of the traj + CNN + RNNs method is relatively high. The frame rate

maximum memory consumption

and network weight are 49 frame/s

2.31 GB

and 261 MB

respectively. By contrast

the frame rate of traj + SVM is 178 frame/s. The computing cost of deep learning networks is generally high. Nevertheless

the traj + CNN + RNNs method is the lightest and fastest among all deep learning networks. Some confusing data for many traditional methods are collected for the second experiment to further compare these methods. The methods based on trajectories

namely

traj + SVM

traj + RNNs

and traj + CNN + RNNs

remain at a good level

and the indexes of ACC(accaracy)

TPR(trure positive rate)

and TNR(true negative rate) and the sensitivity are 0.853

0.847

0.872

and 52/26(frame/s)

respectively. However

the corresponding indexes of CNN and C3D considerably reduced. The accuracies of CNN and C3D are 0.585 and 0.716

respectively.

Conclusion

The proposed method helps improve the accuracy and sensitivity of smoke detection. The smoke trajectories can be identified from the condensed images

even from those of early smoke

which are helpful for early fire warning.

关键词

Keywords

references

Shi J T, Yuan F N, Xia X. Video smoke detection:a literature survey[J]. Journal of Image and Graphics, 2018, 23(3):303-322.

史劲亭, 袁非牛, 夏雪.视频烟雾检测研究进展[J].中国图象图形学报, 2018, 23(3):303-322.[DOI:10.11834/jig.170439]

Hohberg S P. Wildfire smoke detection using convolutional neural networks[D]. Berlin, Germany: Freie Universität Berlin, 2015.

Frizzi S, Kaabi R, Bouchouicha M, et al. Convolutional neural network for video fire and smoke detection[C]//Proceedings of the 42nd Annual Conference of the IEEE Industrial Electronics Society. Florence, Italy: IEEE, 2016: 877-882.[ DOI: 10.1109/IECON.2016.7793196 http://dx.doi.org/10.1109/IECON.2016.7793196 ]

Tao C Y, Zhang J, Wang P. Smoke detection based on deep convolutional neural networks[C] //Proceedings of the International Conference on Industrial Informatics-Computing Technology, Intelligent Technology, Industrial Information Integration. Wuhan, China: IEEE, 2016: 150-153.[ DOI: 10.1109/ICⅡCⅡ.2016.0045 http://dx.doi.org/10.1109/ICⅡCⅡ.2016.0045 ]

Chen J Z, Wang Z J, Chen H H, et al. Dynamic smoke detection using cascaded convolutional neural network for surveillance videos[J]. Journal of University of Electronic Science and Technology of China, 2016, 46(6):992-996.

陈俊周, 汪子杰, 陈洪瀚, 等.基于级联卷积神经网络的视频动态烟雾检测[J].电子科技大学学报, 2016, 46(6):992-996.[DOI:10.3969/j.issn.1001-0548.2016.06.020]

Filonenko A, Kurnianggoro L, Jo K H. Comparative study of modern convolutional neural networks for smoke detection on image data[C]//Proceedings of the 10th International Conference on Human System Interactions. Ulsan, South Korea: IEEE, 2017: 64-68.[ DOI: 10.1109/HSI.2017.8004998 http://dx.doi.org/10.1109/HSI.2017.8004998 ]

Yin M X, Lang C Y, Li Z, et al. Recurrent convolutional network for video-based smoke detection[J]. Multimedia Tools and Applications, 2019, 78(1):237-256.[DOI:10.1007/s11042-017-5561-5]

Yin Z J, Wan B Y, Yuan F N, et al. A deep normalization and convolutional neural network for image smoke detection[J]. IEEE Access, 2017, 5:18429-18438.[DOI:10.1109/ACCESS.2017.2747399]

Zhang Q X, Lin G H, Zhang Y M, et al. Wildland forest fire smoke detection based on faster r-cnn using synthetic smoke images[J]. Procedia Engineering, 2018, 211:441-446.[DOI:10.1016/j.proeng.2017.12.034]

Luo Y M, Zhao L, Liu P Z, et al. Fire smoke detection algorithm based on motion characteristic and convolutional neural networks[J]. Multimedia Tools and Applications, 2018, 77(12):15075-15092.[DOI:10.1007/s11042-017-5090-2]

Dung N M, Kim D, Ro S. A video smoke detection algorithm based on cascade classification and deep learning[J]. KSⅡ Transactions on Internet and Information Systems, 2018, 12(12):6018-6033.[DOI:10.3837/tiis.2018.12.022]

Li X Q, Chen Z X, Wu Q M J, et al. 3D parallel fully convolutional networks for real-time video wildfire smoke detection[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2018, (12):1-15.[DOI:10.1109/TCSVT.2018.2889193]

Yuan F N, Zhang L, Wan B Y, et al. Convolutional neural networks based on multi-scale additive merging layers for visual smoke recognition[J]. Machine Vision and Applications, 2019, 30(2):345-358.[DOI:10.1007/s00138-018-0990-3]

Luo S, Yan C W, Wu K L, et al. Smoke detection based on condensed image[J]. Fire Safety Journal, 2015, 75:23-35.

Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition[EB/OL].[2019-05-21] . https://arxiv.org/pdf/1409.1556.pdf https://arxiv.org/pdf/1409.1556.pdf .

Hochreiter S, Schmidhuber J. Long short-term memory[J]. Neural Computation, 1997, 9(8):1735-1780.[DOI:10.1162/neco.1997.9.8.1735]

Tran D, Bourdev L, Fergus R, et al. Learning spatiotemporal features with 3d convolutional networks[C]//Proceedings of the 2015 IEEE International Conference on Computer Vision. Santiago, Chile: IEEE, 2015: 4489-4497.[ DOI: 10.1109/ICCV.2015.510 http://dx.doi.org/10.1109/ICCV.2015.510 ]

Alert me when the article has been cited

提交

State-of-art of video based smoke detection algorithms

Polyp detection network with multilevel and multiscale feature fusion enhancement based on Mamba architecture

Panoramic visual saliency detection： a survey of principles， methods and applications

Embodied intelligence-driven distracted driving detection： a framework and research prospects

Dougong detection method of Mask R-CNN Swin with an integrated attention mechanism