夏雪1, 袁非牛1,2, 章琳1,3, 杨龙箴1, 史劲亭4(1.江西财经大学信息管理学院, 南昌 330032;2.上海师范大学信息与机电工程学院, 上海 201418;3.江西科技师范大学数学与计算机科学学院, 南昌 330038;4.江西农业大学职业师范(技术)学院, 南昌 330045)
From traditional methods to deep ones: review of visual smoke recognition, detection, and segmentation
Xia Xue1, Yuan Feiniu1,2, Zhang Lin1,3, Yang Longzhen1, Shi Jinting4(1.School of Information Technology, Jiangxi University of Finance and Economics, Nanchang 330032, China;2.College of Information, Mechanical and Electrical Engineering, Shanghai Normal University, Shanghai 201418, China;3.School of Mathematics and Computer Science, Jiangxi Science and Technology Normal University, Nanchang 330038, China;4.Vocational School of Teachers(Technology), Jiangxi Agricultural University, Nanchang 330045, Jiangxi, China)
Sensor-based smoke detection techniques have been widely used in industrial applications. With the development of artificial intelligence, especially the successful commercial application of deep learning, the number of cases in which computer vision-based techniques are applied to smoke detection for fire alarm has increased. Computer vision techniques have not been used as substitutes of sensors in smoke detection systems because of frequent false and missed alarm. By improving computer capability and storage devices, several shortcomings in traditional video smoke detection have been improved or even solved, but these improvements are accompanied with new challenges. To keep up with the development of and latest research on smoke recognition, detection, and segmentation, this study focuses on related domestic and international literature published from 2017 to 2019. From the perspective of tasks and based on years of studying smoke detection, we divide forest fire alarm relying on smoke into three categories, namely, smoke recognition, detection, and segmentation. The three categories of tasks are of different grains and called smoke surveillance tasks. This study grain-wisely presents the latest methods of achieving the above-mentioned surveillance tasks in different aspects ranging from traditional techniques to deep ones. Concretely, related studies on coarse-grained surveillance tasks based on traditional algorithms are introduced first, followed by those on fine-grained tasks implemented by deep learning frameworks. Among the three surveillance tasks, smoke recognition is adopted as the basis. Hence, regarding smoke recognition, detection, and segmentation as recognition-based tasks in coarse-to-fine grain is reasonable. For instance, smoke recognition is the coarsest-grained task and smoke segmentation is the finest-grained recognition task among the three surveillance tasks. Given that the latest literature focuses more on detection and segmentation than on recognition, this study follows this trend and introduces methods of smoke region rough extraction, which obtains a candidate smoke region, and region refinement, which obtains the final detection or segmentation results. Furthermore, according to research, the most distinguishing characteristics of smoke are dynamic features, such as motion and diffusion, and the most stable and robust characteristics of smoke are static features, such as texture. Therefore, during the introduction to smoke region extraction, the extraction and leveraging of static and motion features are explored in every step to gain discriminative capability and robustness for accurate smoke recognition and location. Meanwhile, because deep learning methods tend to present end-to-end solutions rather than individual steps for surveillance tasks, introducing deep learning-based surveillance tasks step-wisely is difficult. Consequently, deep learning-based methods for surveillance tasks are introduced in another section grain-wisely. The overall frameworks and inner concepts are involved rather than the algorithm steps of deep learning-based smoke surveillance. Lastly, the strengths and weaknesses in smoke surveillance tasks are determined, and widely used evaluation indicators and several available datasets are summarized to allow researchers to search for evaluators and annotated datasets. Future development trends are also predicted. Through a comprehensive literature review of surveillance tasks in coarse-to-fine grain, the key techniques, problems to be solved, and promising research directions are demonstrated. Thus, potential solutions can be provided to surveillance task-based forest fire alarm. Further research based on this review might promote the industrial application of smoke surveillance tasks.