从传统到深度：视觉烟雾识别、检测与分割

夏雪; 袁非牛; 章琳; 杨龙箴; 史劲亭

doi:10.11834/jig.190230

火情与烟雾专栏 | 浏览量 : 0 下载量: 154 CSCD: 10

PDF
导出
分享
收藏
专辑

从传统到深度：视觉烟雾识别、检测与分割
From traditional methods to deep ones: review of visual smoke recognition, detection, and segmentation
2019年24卷第10期页码：1627-1647
收稿：2019-05-20，

修回：2019-6-12，

录用：2019-6-19，

纸质出版：2019-10-16
DOI： 10.11834/jig.190230
稿件说明：

移动端阅览

夏雪, 袁非牛, 章琳, 杨龙箴, 史劲亭. 从传统到深度：视觉烟雾识别、检测与分割[J]. 中国图象图形学报, 2019,24(10):1627-1647. DOI： 10.11834/jig.190230.

Xue Xia, Feiniu Yuan, Lin Zhang, Longzhen Yang, Jinting Shi. From traditional methods to deep ones: review of visual smoke recognition, detection, and segmentation[J]. Journal of Image and Graphics, 2019, 24(10): 1627-1647. DOI： 10.11834/jig.190230.

摘要

在烟雾检测系统中，采用机器学习的视觉技术暂未广泛替代传感器的主要原因在于其误报与漏报较高。计算力度的提高、存储设备的发展，使得传统视觉技术中存在的问题逐渐得到改善或解决，但也迎来了新的挑战。为反映用于森林火灾预警的烟雾识别、检测等技术的最新研究进展，本文重点对2017—2019年国内外公开发表的相关文献进行梳理和分析。从监控角度出发，基于对此领域的长期研究与广泛文献调研，将利用烟雾的森林火灾预警任务分为烟雾识别、检测、分割这3类不同的粒度，分别介绍实现这些任务的传统方法及深度方法。依照当前研究热度，主要关注视频烟雾检测与分割这两个细粒度任务。其中烟雾区域的粗提取与二次提取方法是检测与分割的关键，因此将探索这些方法如何提取、利用烟雾的动态与静态特征。此外，由于深度学习框架主要实现端对端的任务，无法分离出关键步骤，故对基于深度学习的烟雾监控任务进行单独梳理，不关注单步细节，主要体现文献思路。最后，对实现烟雾识别、检测、分割任务具体方法中的优缺点、烟雾监控任务中常用的指标、研究常用的数据库进行总结，并对发展前景进行展望。为基于烟雾的森林火灾预警技术提供更多的发展方向。

Abstract

Sensor-based smoke detection techniques have been widely used in industrial applications. With the development of artificial intelligence

especially the successful commercial application of deep learning

the number of cases in which computer vision-based techniques are applied to smoke detection for fire alarm has increased. Computer vision techniques have not been used as substitutes of sensors in smoke detection systems because of frequent false and missed alarm. By improving computer capability and storage devices

several shortcomings in traditional video smoke detection have been improved or even solved

but these improvements are accompanied with new challenges. To keep up with the development of and latest research on smoke recognition

detection

and segmentation

this study focuses on related domestic and international literature published from 2017 to 2019. From the perspective of tasks and based on years of studying smoke detection

we divide forest fire alarm relying on smoke into three categories

namely

smoke recognition

detection

and segmentation. The three categories of tasks are of different grains and called smoke surveillance tasks. This study grain-wisely presents the latest methods of achieving the above-mentioned surveillance tasks in different aspects ranging from traditional techniques to deep ones. Concretely

related studies on coarse-grained surveillance tasks based on traditional algorithms are introduced first

followed by those on fine-grained tasks implemented by deep learning frameworks. Among the three surveillance tasks

smoke recognition is adopted as the basis. Hence

regarding smoke recognition

detection

and segmentation as recognition-based tasks in coarse-to-fine grain is reasonable. For instance

smoke recognition is the coarsest-grained task and smoke segmentation is the finest-grained recognition task among the three surveillance tasks. Given that the latest literature focuses more on detection and segmentation than on recognition

this study follows this trend and introduces methods of smoke region rough extraction

which obtains a candidate smoke region

and region refinement

which obtains the final detection or segmentation results. Furthermore

according to research

the most distinguishing characteristics of smoke are dynamic features

such as motion and diffusion

and the most stable and robust characteristics of smoke are static features

such as texture. Therefore

during the introduction to smoke region extraction

the extraction and leveraging of static and motion features are explored in every step to gain discriminative capability and robustness for accurate smoke recognition and location. Meanwhile

because deep learning methods tend to present end-to-end solutions rather than individual steps for surveillance tasks

introducing deep learning-based surveillance tasks step-wisely is difficult. Consequently

deep learning-based methods for surveillance tasks are introduced in another section grain-wisely. The overall frameworks and inner concepts are involved rather than the algorithm steps of deep learning-based smoke surveillance. Lastly

the strengths and weaknesses in smoke surveillance tasks are determined

and widely used evaluation indicators and several available datasets are summarized to allow researchers to search for evaluators and annotated datasets. Future development trends are also predicted. Through a comprehensive literature review of surveillance tasks in coarse-to-fine grain

the key techniques

problems to be solved

and promising research directions are demonstrated. Thus

potential solutions can be provided to surveillance task-based forest fire alarm. Further research based on this review might promote the industrial application of smoke surveillance tasks.

关键词

Keywords

references

Xing Z H, Ye Z C. Resource and environment: 2016 forest fires[M]//National Bureau of Statistics of China. China Statistical Yearbook. Beijing: China Statistics Press, 2017: 260.

邢志宏, 叶植材.资源和环境: 森林火灾情况(2016年)[M]//中华人民共和国国家统计局.中国统计年鉴.北京: 中国统计出版社, 2017: 260.

Jin H, Zhang H Y. Public safety: 2017 basic statistics on fire accidents by region[M]//Statistics Department of Social Science, Technology and Cultural Industries, National Bureau of Statistics. China Social Statistical Yearbook. Beijing: China Statistics Press, 2018: 344.

金红, 张会英.公共安全: 分地区火灾事故情况(2017年)[M]//国家统计局社会科技和文化产业统计司.中国社会统计年鉴.北京: 中国统计出版社, 2018: 344.

Liu W H, Xu B J. Natural disasters&environmental accidents: forest fires by regions(2017)[M]//National Bureau of Statistics, Ministry of Ecology and Environment. China Statistical Yearbook on Environment. Beijing: China Statistics Press, 2018: 78-79.

刘文华, 徐必久.自然灾害及突发事件: 各地区森林火灾情况(2017年)[M]//国家统计局, 生态环境部.中国环境统计年鉴.北京: 中国统计出版社, 2018: 78-79.

Jin B. Forest fire prevention forest fire by months(2017)[M]//State Forestry and Grassland Administration. China Forestry Yearbook. Beijing: China Forestry Publishing House, 2018: 138.

金博.森林防火: 全国森林火灾分月统计(2017)[M]//国家林业和草原局.中国林业年鉴.北京: 中国林业出版社, 2018: 138.

Shi J T, Yuan F N, Xia X. Video smoke detection:a literature survey[J]. Journal of Image and Graphics, 2018, 23(3):303-322.

史劲亭, 袁非牛, 夏雪.视频烟雾检测研究进展[J].中国图象图形学报, 2018, 23(3):303-322.[DOI:10.11834/jig.170439]

Wang Y K. Forest fire prevention: construction of forest fire protection system[M]//State Forestry and Grassland Administration. China Forestry Yearbook. Beijing: China Forestry Publishing House, 2018: 141.

王永坤.森林防火, 森林消防体系建设[M]//国家林业和草原局.中国林业年鉴.北京: 中国林业出版社, 2018: 141.

Yuan C, Liu Z X, Zhang Y M. Learning-based smoke detection for unmanned aerial vehicles applied to forest fire surveillance[J]. Journal of Intelligent&Robotic Systems, 2019, 93(1-2):337-349.[DOI:10.1007/s10846-018-0803-y]

Li H D, Yuan F N. Image based smoke detection using pyramid texture and edge features[J]. Journal of Image and Graphics, 2015, 20(6):772-780.

李红娣, 袁非牛.采用金字塔纹理和边缘特征的图像烟雾检测[J].中国图象图形学报, 2015, 20(6):772-780.[DOI:10.11834/jig.20150606]

Yuan F G. Smoke detection method based on smoke density with video[D]. Wuhan: Huazhong University of Science and Technology, 2013. http://cdmd.cnki.com.cn/Article/CDMD-10487-1014026473.htm .

袁飞阁.基于烟雾浓度的火灾视频检测方法[D].武汉: 华中科技大学, 2013.

Ma T Y. Smoke concertration measurement method based on structural similarity index with video[D]. Wuhan: Huazhong University of Science and Technology, 2017.

马天颖.基于图像结构相似度的烟雾浓度视频测量方法研究[D].武汉: 华中科技大学, 2017.

Lu C, Lu M Q, Lu X B, et al. Forest fire smoke recognition based on multiple feature fusion[J]. IOP Conference Series:Materials Science and Engineering, 2018, 435:#012006.[DOI:10.1088/1757-899x/435/1/012006]

Yao J Y, Raffuse S M, Brauer M, et al. Predicting the minimum height of forest fire smoke within the atmosphere using machine learning and data from the CALIPSO satellite[J]. Remote Sensing of Environment, 2018, 206:98-106.[DOI:10.1016/j.rse.2017.12.027]

Collumeau J F, Laurent H, Hafiane A, et al. Fire scene segmentations for forest fire characterization: a comparative study[C]//Proceedings of the 18th IEEE International Conference on Image Processing. Brussels, Belgium: IEEE, 2011: 2973-2976.[ DOI: 10.1109/ICIP.2011.6116285 http://dx.doi.org/10.1109/ICIP.2011.6116285 ]

Alamgir N, Nguyen K, Chandran V, et al. Combining multi-channel color space with local binary co-occurrence feature descriptors for accurate smoke detection from surveillance videos[J]. Fire Safety Journal, 2018, 102:1-10.[DOI:10.1016/j.firesaf.2018.09.003]

Tian H D, Li W Q, Ogunbona P O, et al. Detection and separation of smoke from single image frames[J]. IEEE Transactions on Image Processing, 2018, 27(3):1164-1177.[DOI:10.1109/TIP.2017.2771499]

Appana D K, Islam R, Khan S A, et al. A video-based smoke detection using smoke flow pattern and spatial-temporal energy analyses for alarm systems[J]. Information Sciences, 2017, 418-419:91-101.[DOI:10.1016/j.ins.2017.08.001]

Yuan F N, Shi J T, Xia X, et al. Encoding pairwise Hamming distances of local binary patterns for visual smoke recognition[J]. Computer Vision and Image Understanding, 2019, 178:43-53.[DOI:10.1016/j.cviu.2018.10.008]

Yuan F N, Xia X, Shi J T. Mixed co-occurrence of local binary patterns and Hamming-distance-based local binary patterns[J]. Information Sciences, 2018, 460-461:202-222.[DOI:10.1016/j.ins.2018.05.033]

Yuan F N, Shi J T, Xia X, et al. Co-occurrence matching of local binary patterns for improving visual adaption and its application to smoke recognition[J]. IET Computer Vision, 2019, 13(2):178-187.[DOI:10.1049/iet-cvi.2018.5164]

Yuan F N, Xia X, Li G, et al. GaborNet for smoke recognition and texture classification[J]. Journal of Image and Graphics, 2019, 24(2):269-281.

袁非牛, 夏雪, 李钢, 等.面向烟雾识别与纹理分类的Gabor网络[J].中国图象图形学报, 2019, 24(2):269-281.[DOI:10.11834/jig.180397]

Yuan F N, Xia X, Shi J T. Holistic learning-based high-order feature descriptor for smoke recognition[J]. International Journal of Wavelets, Multiresolution and Information Processing, 2019, 17(2):1940005.[DOI:10.1142/s0219691319400058]

Yuan F N, Tang T T, Xia X, et al. Dual-encoded features from both spatial and Curvelet domains for image smoke recognition[J]. Ksii Transactions on Internet and Information Systems, 2019, 13(4):2078-2093.[DOI:10.3837/tiis.2019.04.019]

Yuan F N, Xia X, Shi J T, et al. Learning multi-scale and multi-order features from 3D local differences for visual smoke recognition[J]. Information Sciences, 2018, 468:193-212.[DOI:10.1016/j.ins.2018.08.005]

Matlani P, Shrivastava M. A survey on video smoke detection[M]//Mishra D K, Nayak M K, Joshi A. Information and Communication Technology for Sustainable Development. Singapore: Springer, 2018: 211-222.[ DOI: 10.1007/978-981-10-3932-4_22 http://dx.doi.org/10.1007/978-981-10-3932-4_22 ]

Tao H J, Lu X B. Smoky vehicle detection based on range filtering on three orthogonal planes and motion orientation histogram[J]. IEEE Access, 2018, 6:57180-57190.[DOI:10.1109/ACCESS.2018.2873757]

Zhao X, Ji H, Zhang D Y, et al. Fire smoke detection based on contextual object detection[C]//Proceedings of the 3rd IEEE International Conference on Image, Vision and Computing. Chongqing, China: IEEE, 2018: 473-476.[ DOI: 10.1109/icivc.2018.8492823 http://dx.doi.org/10.1109/icivc.2018.8492823 ]

Zen R I M, Widyanto M R, Kiswanto G, et al. Dangerous smoke classification using mathematical model of meaning[J]. Procedia Engineering, 2013, 62:963-971.[DOI:10.1016/j.proeng.2013.08.149]

Wang S D, He Y P, Yang H Y, et al. Video smoke detection using shape, color and dynamic features[J]. Journal of Intelligent&Fuzzy Systems, 2017, 33(1):305-313.[DOI:10.3233/JIFS-161605]

Tang J, Zhou Y, Yang M, et al. A smoke detection algorithm using color mixture model and feature combination[J]. Journal of Optoelectronics·Laser, 2017, 28(7):751-758.

唐杰, 周洋, 杨萌, 等.采用颜色混合模型和特征组合的视频烟雾检测[J].光电子·激光, 2017, 28(7):751-758.[DOI:10.16136/j.joel.2017.07.0425]

Sun R, Chen X C, Chen B Y. Smoke detection for videos based on adaptive learning rate and linear fitting algorithm[C]//Proceedings of 2018 Chinese Automation Congress. Xi'an, China: IEEE, 2018: 1948-1954.[ DOI: 10.1109/CAC.2018.8623369 http://dx.doi.org/10.1109/CAC.2018.8623369 ]

Zhao Y Q, Zhou Z, Xu M M. Forest fire smoke video detection using spatiotemporal and dynamic texture features[J]. Journal of Electrical and Computer Engineering, 2015, 2015:#706187.[DOI:10.1155/2015/706187]

Fang S, Qi L J, Yu L. Video smoke detection with multi-feature analysis[J]. Computer Engineering and Applications, 2016, 52(13):222-227.

方帅, 祁林娟, 于磊.多特征分析的视频烟雾检测方法[J].计算机工程与应用, 2016, 52(13):222-227.[DOI:10.3778/j.issn.1002-8331.1407-0533]

Zhao M, Zhang W, Wang X, et al. A smoke detection algorithm with multi-texture feature exploration under a spatio-temporal background model[J]. Journal of Xi'an Jiaotong University, 2018, 52(8):67-73.

赵敏, 张为, 王鑫, 等.时空背景模型下结合多种纹理特征的烟雾检测[J].西安交通大学学报, 2018, 52(8):67-73.[DOI:10.7652/xjtuxb201808011]

Dimitropoulos K, Barmpoutis P, Grammalidis N. Higher order linear dynamical systems for smoke detection in video surveillance applications[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2017, 27(5):1143-1154.[DOI:10.1109/TCSVT.2016.2527340]

Gao Y, Cheng P L. Forest fire smoke detection based on visual smoke root and diffusion model[J]. Fire Technology, 2019.[DOI:10.1007/s10694-019-00831-x]

Tao H J, Lu X B. Smoke vehicle detection based on robust codebook model and robust volume local binary count patterns[J]. Image and Vision Computing, 2019, 86:17-27.[DOI:10.1016/j.imavis.2019.03.008]

Tao H J, Lu X B. Smoke vehicle detection based on multi-feature fusion and hidden Markov model[J]. Journal of Real-Time Image Processing, 2019.[DOI:10.1007/s11554-019-00856-z]

de Souza B M N, Facon J. A fire color mapping-based segmentation: fire pixel segmentation approach[C]//Proceedings of the 13th IEEE/ACS International Conference of Computer Systems and Applications. Agadir, Morocco: IEEE, 2016: 1-8.[ DOI: 10.1109/AICCSA.2016.7945741 http://dx.doi.org/10.1109/AICCSA.2016.7945741 ]

Nemalidinne S M, Gupta D. Nonsubsampled contourlet domainvisible and infrared image fusion framework for fire detection using pulse coupled neural network and spatial fuzzy clustering[J]. Fire Safety Journal, 2018, 101:84-101.[DOI:10.1016/j.firesaf.2018.08.012]

Sousa M J, Moutinho A, Almeida M. Classification of potential fire outbreaks:a fuzzy modeling approach based on thermal images[J]. Expert Systems with Applications, 2019, 129:216-232.[DOI:10.1016/j.eswa.2019.03.030]

Wang X, Jiang A P, Wang Y L. A segmentation method of smoke in forest-fire image based on FBM and region growing[C]//Proceedings of the 4th International Workshop on Chaos-Fractals Theories and Applications. Hangzhou, China: IEEE, 2011: 390-393.[ DOI: 10.1109/IWCFTA.2011.92 http://dx.doi.org/10.1109/IWCFTA.2011.92 ]

Zhang N, Wang H Q, Hu Y. Smoke image segmentation algorithm based on rough set and region growing[J]. Journal of Frontiers of Computer Science and Technology, 2017, 11(8):1296-1304.

张娜, 王慧琴, 胡燕.粗糙集与区域生长的烟雾图像分割算法研究[J].计算机科学与探索, 2017, 11(8):1296-1304.[DOI:10.3778/j.issn.1673-9418.1606018]

Zhou Z Q, Shi Y S, Gao Z F, et al. Wildfire smoke detection based on local extremal region segmentation and surveillance[J]. Fire Safety Journal, 2016, 85:50-58.[DOI:10.1016/j.firesaf.2016.08.004]

Lin G H, Zhang Y M, Zhang Q X, et al. Smoke detection in video sequences based on dynamic texture using volume local binary patterns[J]. Ksii Transactions on Internet and Information Systems, 2017, 11(11):5522-5536.[DOI:10.3837/tiis.2017.11.019]

Filonenko A, Hernández D C, Jo K H. Fast smoke detection for video surveillance using CUDA[J]. IEEE Transactions on Industrial Informatics, 2018, 14(2):725-733.[DOI:10.1109/TⅡ.2017.2757457]

Hu Y, Wang H Q, Ma Z F. Adaptive smoke image segmentation algorithm based on improved Gaussian mixture model[J]. Journal of Computer-Aided Design&Computer Graphics, 2016, 28(7):1138-1145.

胡燕, 王慧琴, 马宗方.改进混合高斯模型的自适应烟雾图像分割算法[J].计算机辅助设计与图形学学报, 2016, 28(7):1138-1145.[DOI:10.3969/j.issn.1003-9775.2016.07.013]

Tian H D, Li W Q, Wang L, et al. Smoke detection in video:an image separation approach[J]. International Journal of Computer Vision, 2014, 106(2):192-209.[DOI:10.1007/s11263-013-0656-6]

Wellhausen A, Stadler A, Hoppe F. Visual smoke density measurement for video smoke detection[C]//Proceedings of International Conference on Automatic Fire Detection. Hyattsville, MD, USA: EUSAS, 2017.

Bchir O, Ismail M M B, Asiri N. Image based smoke detection using source separation[J]. Spectroscopy and Spectral Analysis, 2019, 39(3):982-989.[DOI:10.3964/j.issn.1000-0593(2019)03-0982-08]

Su Y C, Li J, Plaza A, et al. DAEN:deep autoencoder networks for hyperspectral unmixing[J]. IEEE Transactions on Geoscience and Remote Sensing, 2019, 57(7):4309-4321.[DOI:10.1109/TGRS.2018.2890633]

Jia Y, Lin G H, Wang J J, et al. Early video smoke segmentation algorithm based on saliency detection and Gaussian mixture model[J]. Computer Engineering, 2016, 42(2):206-209, 217.

贾阳, 林高华, 王进军, 等.基于显著性检测和高斯混合模型的早期视频烟雾分割算法[J].计算机工程, 2016, 42(2):206-209, 217.[DOI:10.3969/j.issn.1000-3428.2016.02.037]

Li Y, Vodacek A, Zhu Y S. An automatic statistical segmentation algorithm for extraction of fire and smoke regions[J]. Remote Sensing of Environment, 2007, 108(2):171-178.[DOI:10.1016/j.rse.2006.10.023]

Tao C Y, Zhang J, Wang P. Smoke detection based on deep convolutional neural networks[C]//Proceedings of 2016 International Conference on Industrial Informatics——Computing Technology, Intelligent Technology, Industrial Information Integration. Wuhan, China: IEEE, 2016: 150-153.[ DOI: 10.1109/ICⅡCⅡ.2016.0045 http://dx.doi.org/10.1109/ICⅡCⅡ.2016.0045 ]

Krizhevsky A, Sutskever I, Hinton G E. ImageNet classification with deep convolutional neural networks[C]//Proceedings of the 25th International Conference on Neural Information Processing Systems. Lake Tahoe, Nevada, USA: ACM, 2012: 1097-1105.

Yin Z J, Wan B Y, Yuan F N, et al. A deep normalization and convolutional neural network for image smoke detection[J]. IEEE Access, 2017, 5:18429-18438.[DOI:10.1109/ACCESS.2017.2747399]

Yuan F N, Zhang L, Wan B Y, et al. Convolutional neural networks based on multi-scale additive merging layers for visual smoke recognition[J]. Machine Vision and Applications, 2019, 30(2):345-358.[DOI:10.1007/s00138-018-0990-3]

Wang W B, Xu Q, Han Z B. Fire and smoke detection based on the optimal mass transmission optical flow method and neural network[J]. Journal of Harbin University of Science and Technology, 2017, 22(1):86-90.

王卫兵, 徐倩, 韩再博.基于最优质量传输光流法和神经网络的火焰和烟雾检测[J].哈尔滨理工大学学报, 2017, 22(1):86-90.[DOI:10.15938/j.jhust.2017.01.015]

Zhong Z, Wang M J, Shi Y K, et al. A convolutional neural network-based flame detection method in video sequence[J]. Signal, Image and Video Processing, 2018, 12(8):1619-1627.[DOI:10.1007/s11760-018-1319-4]

Dung N M, Kim D, Ro S. A video smoke detection algorithm based on cascade classification and deep learning[J]. Ksii Transactions on Internet and Information Systems, 2018, 12(12):6018-6033.[DOI:10.3837/tiis.2018.12.022]

Luo Y M, Zhao L, Liu P Z, et al. Fire smoke detection algorithm based on motion characteristic and convolutional neural networks[J]. Multimedia Toolsand Applications, 2018, 77(12):15075-15092.[DOI:10.1007/s11042-017-5090-2]

Pundir A S, Raman B. Deep belief network for smoke detection[J]. Fire Technology, 2017, 53(6):1943-1960.[DOI:10.1007/s10694-017-0665-z]

Kaabi R, Sayadi M, Bouchouicha M, et al. Early smoke detection of forest wildfire video using deep belief network[C]//Proceedings of the 4th International Conference on Advanced Technologies for Signal and Image Processing. Sousse, Tunisia: IEEE, 2018: 1-6.[ DOI: 10.1109/atsip.2018.8364446 http://dx.doi.org/10.1109/atsip.2018.8364446 ]

Khan S, Muhammad K, Mumtaz S, et al. Energy-efficient deep CNN for smoke detection in foggy IoT environment[J]. IEEE Internet of Things Journal, 2019.[DOI:10.1109/jiot.2019.2896120]

Muhammad K, Ahmad J, Baik S W. Early fire detection using convolutional neural networks during surveillance for effective disaster management[J]. Neurocomputing, 2018, 288:30-42.[DOI:10.1016/j.neucom.2017.04.083]

Muhammad K, Ahmad J, Mehmood I, et al. Convolutional neural networks based fire detection in surveillance videos[J]. IEEE Access, 2018, 6:18174-18183.[DOI:10.1109/ACCESS.2018.2812835]

Zhang Q X, Lin G H, Zhang Y M, et al. Wildland forest fire smoke detection based on faster R-CNN using synthetic smoke images[J]. Procedia Engineering, 2018, 211:441-446.[DOI:10.1016/j.proeng.2017.12.034]

Ren S Q, He K M, Girshick R, et al. Faster R-CNN:towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(6):1137-1149.[DOI:10.1109/TPAMI.2016.2577031]

Frizzi S, Kaabi R, Bouchouicha M, et al. Convolutional neural network for video fire and smoke detection[C]//Proceedings of the 42nd Annual Conference of the IEEE Industrial Electronics Society. Florence, Italy: IEEE, 2016: 877-882.[ DOI: 10.1109/IECON.2016.7793196 http://dx.doi.org/10.1109/IECON.2016.7793196 ]

LeCun Y,Bottou L, Bengio Y, et al. Gradient-based learning applied to document recognition[J]. Proceedings of the IEEE, 1998, 86(11):2278-2324.[DOI:10.1109/5.726791]

Lin Z Y, Shen Y. Research on fire warning algorithm based on deep convolutional neural network[J]. Information&Communications, 2018, (5):38-42.

林作永, 谌瑶.基于深度卷积神经网络的火灾预警算法研究[J].信息通信, 2018, (5):38-42.

Szegedy C, Vanhoucke V, Ioffe S, et al. Rethinking the inception architecture for computer vision[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, NV, USA: IEEE, 2016: 2818-2826.[ DOI: 10.1109/CVPR.2016.308 http://dx.doi.org/10.1109/CVPR.2016.308 ]

He K M, Zhang X Y, Ren S Q, et al. Identity mappings in deep residual networks[C]//Proceedings of the 14th European Conference on Computer Vision. Amsterdam, The Netherlands: Springer, 2016: 630-645.[ DOI: 10.1007/978-3-319-46493-0_38 http://dx.doi.org/10.1007/978-3-319-46493-0_38 ]

Howard A G, Zhu M L, Chen B, et al. MobileNets: efficient convolutional neural networks for mobile vision applications[EB/OL].[2019-04-20] . https://arxiv.org/pdf/1704.04861.pdf https://arxiv.org/pdf/1704.04861.pdf .

Liu W, Anguelov D, Erhan D, et al. SSD: single shot multiBox detector[C]//Proceedings of the 14th European Conference on Computer Vision. Amsterdam, The Netherlands: Springer, 2016: 21-37.[ DOI: 10.1007/978-3-319-46448-0_2 http://dx.doi.org/10.1007/978-3-319-46448-0_2 ]

Dai J F, Li Y, He K M, et al. R-FCN: object detection via region-based fully convolutional networks[C]//Proceedings of the 30th International Conference on Neural Information Processing Systems. Barcelona, Spain: ACM, 2016: 379-387.

Lin G H, Zhang Y M, Xu G, et al. Smoke detection on video sequences using 3D convolutional neural networks[J]. Fire Technology, 2019.[DOI:10.1007/s10694-019-00832-w]

Ji S W, Xu W, Yang M, et al. 3D convolutional neural networks for human action recognition[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013, 35(1):221-231.[DOI:10.1109/tpami.2012.59]

Chen J Z, Wang Z J, Chen H H, et al. Dynamic smoke detection using cascaded convolutional neural network for surveillance videos[J]. Journal of University of Electronic Science and Technology of China, 2016, 45(6):992-996.

陈俊周, 汪子杰, 陈洪瀚, 等.基于级联卷积神经网络的视频动态烟雾检测[J].电子科技大学学报, 2016, 45(6):992-996.[DOI:10.3969/j.issn.1001-0548.2016.06.020]

Simonyan K, Zisserman A. Two-stream convolutional networks for action recognition in videos[C]//Proceedings of the 27th International Conference on Neural Information Processing Systems. Montreal, Canada: ACM, 2014: 568-576.

Xu G, Zhang Y M, Zhang Q X, et al. Deep domain adaptation based video smoke detection using synthetic smoke images[J]. Fire Safety Journal, 2017, 93:53-59.[DOI:10.1016/j.firesaf.2017.08.004]

Xu G, Zhang Q X, Liu D C, et al. Adversarial adaptation from synthesis to reality in fast detector for smoke detection[J]. IEEE Access, 2019, 7:29471-29483.[DOI:10.1109/access.2019.2902606]

Hu Y C, Lu X B. Real-time video fire smoke detection by utilizing spatial-temporal ConvNet features[J]. Multimedia Tools and Applications, 2018, 77(22):29283-29301.[DOI:10.1007/s11042-018-5978-5]

Aslan S, Güdükbay U, Töreyin B U, et al. Deep convolutional generative adversarial networks based flame detection in video[EB/OL].[2019-04-20] . https://arxiv.org/pdf/1902.01824.pdf https://arxiv.org/pdf/1902.01824.pdf .

Radford A, Metz L, Chintala S. Unsupervised representation learning with deep convolutional generative adversarial networks[EB/OL].[2019-04-20] . https://arxiv.org/pdf/1511.06434.pdf https://arxiv.org/pdf/1511.06434.pdf .

Yin M X, Lang C Y, Li Z, et al. Recurrent convolutional network for video-based smoke detection[J]. Multimedia Tools and Applications, 2019, 78(1):237-256.[DOI:10.1007/s11042-017-5561-5]

Graves A, Fernández A, Gomez F, et al. Connectionist temporal classification: labelling unsegmented sequence data with recurrent neural networks[C]//Proceedings of the 23rd International Conference on Machine Learning. Pittsburgh, Pennsylvania, USA: ACM, 2006: 369-376.[ DOI: 10.1145/1143844.1143891 http://dx.doi.org/10.1145/1143844.1143891 ]

Xu G, Zhang Y M, Zhang Q X, et al. Video smoke detection based on deep saliency network[J]. Fire Safety Journal, 2019, 105:277-285.[DOI:10.1016/j.firesaf.2019.03.004]

Liang M, Hu X L. Recurrent convolutional neural network for object recognition[C]//Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition. Boston, MA, USA: IEEE, 2015: 3367-3375.[ DOI: 10.1109/CVPR.2015.7298958 http://dx.doi.org/10.1109/CVPR.2015.7298958 ]

Li X Q, Chen Z X, Wu Q M J, et al. 3D parallel fully convolutional networks for real-time video wildfire smoke detection[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2018.[DOI:10.1109/tcsvt.2018.2889193]

Badrinarayanan V, Kendall A, Cipolla R. SegNet:a deep convolutional encoder-decoder architecture for image segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(12):2481-2495.[DOI:10.1109/TPAMI.2016.2644615]

Chen L C, Papandreou G, Kokkinos I, et al. DeepLab:semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018, 40(4):834-848.[DOI:10.1109/TPAMI.2017.2699184]

Yuan F N, Zhang L, Xia X, et al. Deep smoke segmentation[J]. Neurocomputing, 2019, 357:248-260.[DOI:10.1016/j.neucom.2019.05.011]

Toulouse T, Rossi L, Akhloufi M, et al. Benchmarking of wildland fire colour segmentation algorithms[J]. IET Image Processing, 2015, 9(12):1064-1072.[DOI:10.1049/iet-ipr.2014.0935]

Chino D Y T, Avalhais L P S, Rodrigues J F, et al. BoWFire: detection of fire in still images by integrating pixel color and texture analysis[C]//Proceedings of the 28th SIBGRAPI Conference on Graphics, Patterns and Images. Salvador, Brazil: IEEE, 2015: 95-102.[ DOI: 10.1109/sibgrapi.2015.19 http://dx.doi.org/10.1109/sibgrapi.2015.19 ]

Foggia P, Saggese A, Vento M. Real-time fire detection for video-surveillance applications using a combination of experts based on color, shape, and motion[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2015, 25(9):1545-1556.[DOI:10.1109/TCSVT.2015.2392531]

Ko B C, Cheong K H, Nam J Y. Fire detection based on vision sensor and support vector machines[J]. Fire Safety Journal, 2009, 44(3):322-329.[DOI:10.1016/j.firesaf.2008.07.006]

Vezzani R, Cucchiara R. Video surveillance online repository (ViSOR):an integrated framework[J]. Multimedia Tools and Applications, 2010, 50(2):359-380.[DOI:10.1007/s11042-009-0402-9]

Töreyin B U, Cetin A E. Online detection of fire in video[C]//Proceedings of 2007 IEEE Conference on Computer Vision and Pattern Recognition. Minneapolis, MN, USA: IEEE, 2007: 1-5.[ DOI: 10.1109/CVPR.2007.383442 http://dx.doi.org/10.1109/CVPR.2007.383442 ]

Töreyin B U, Dedeoğlu Y, Cetin A E. Flame detection in video using hidden Markov models[C]//Proceedings of 2005 IEEE International Conference on Image Processing. Genova, Italy: IEEE, 2005: Ⅱ-1230.[ DOI: 10.1109/ICIP.2005.1530284 http://dx.doi.org/10.1109/ICIP.2005.1530284 ]

Töreyin B U, Dedeoğlu Y, Güdükbay U, et al. Computer vision based method for real-time fire and flame detection[J]. Pattern Recognition Letters, 2006, 27(1):49-58.[DOI:10.1016/j.patrec.2005.06.015]