浏览全部资源
扫码关注微信
Received：10 February 2020，
Revised：2020-3-23，
Accepted：30 March 2020，
Published：16 December 2020
移动端阅览
目的
2
在自动化和智能化的现代生产制造过程中，视频异常事件检测技术扮演着越来越重要的角色，但由于实际生产制造中异常事件的复杂性及无关生产背景的干扰，使其成为一项非常具有挑战性的任务。很多传统方法采用手工设计的低级特征对视频的局部区域进行特征提取，然而此特征很难同时表示运动与外观特征。此外，一些基于深度学习的视频异常事件检测方法直接通过自编码器的重构误差大小来判定测试样本是否为正常或异常事件，然而实际情况往往会出现一些原本为异常的测试样本经过自编码得到的重构误差也小于设定阈值，从而将其错误地判定为正常事件，出现异常事件漏检的情形。针对此不足，本文提出一种融合自编码器和one-class支持向量机（support vector machine，SVM）的异常事件检测模型。
方法
2
通过高斯混合模型（Gaussian mixture model，GMM）提取固定大小的时空兴趣块（region of interest，ROI）；通过预训练的3维卷积神经网络（3D convolutional neural network，C3D）对ROI进行高层次的特征提取；利用提取的高维特征训练一个堆叠的降噪自编码器，通过比较重构误差与设定阈值的大小，将测试样本判定为正常、异常和可疑3种情况之一；对自编码器降维后的特征训练一个one-class SVM模型，用于对可疑测试样本进行二次检测，进一步排除异常事件。
结果
2
本文对实际生产制造环境下的机器人工作场景进行实验，采用AUC（area under ROC）和等错误率（equal error rate，EER）两个常用指标进行评估。在设定合适的误差阈值时，结果显示受试者工作特征（receiver operating characteristic，ROC）曲线下AUC达到91.7%，EER为13.8%。同时，在公共数据特征集USCD（University of California，San Diego）Ped1和USCD Ped2上进行了模型评估，并与一些常用方法进行了比较，在USCD Ped1数据集中，相比于性能第2的方法，AUC在帧级别和像素级别分别提高了2.6%和22.3%；在USCD Ped2数据集中，相比于性能第2的方法，AUC在帧级别提高了6.7%，从而验证了所提检测方法的有效性与准确性。
结论
2
本文提出的视频异常事件检测模型，结合了传统模型与深度学习模型，使视频异常事件检测结果更加准确。
Objective
2
With the recent improvements in people's living standards and quality and the rapid development of digital information technology
all sectors of society have paid increasing attention to the application of science and technology in the field of public safety. To maintain a safe public environment
video surveillance equipment has been increasingly installed in streets
schools
communities
subways
and other public places. However
traditional video surveillance systems gradually become unable to process the ever-increasing size of video data. Therefore
the development of intelligent surveillance systems with automatic detection
identification
and alarm functions has broad and far-reaching significance for maintaining public safety and developing artificial intelligence. Anomaly detection is an important part of intelligent monitoring systems that plays a key role in maintaining public safety. As such
anomaly detection has become a hot research topic for both academic and industrial practitioners. In the past
video anomalies are manually detected
which requires much human labor. Therefore
the introduction of an efficient and automated anomaly detection system has significantly reduced the labor costs for such undertaking. Video anomaly detection technologies play an important role in automated and intelligent modern production and manufacturing
video anomaly detection remains a challenging task in complex factory environments given the anomalous events and interference of unrelated contexts in such scenarios. Many methods use hand-designed low-level features to extract features from the local areas of a video. However
these features cannot represent both motion and appearance. To address this problem
we propose a novel detection method based on deep spatial-temporal features.
Method
2
First
given that abnormalities are mainly observed in the motion areas of videos
this article extracts the surveillance video motion area via a Gaussian mixture model (GMM). Specifically
this model is used to extract a fixed-size spatial-temporal region of interest from a video. Second
to facilitate the detection of subsequent abnormal events
high-level features are extracted from the region of interest (ROI) via a 3Dconvolutional neural network. Third
to enhance anomaly detection efficiency
the extracted features are used to train a denoising auto-encoder and to detect anomalous events based on reconstruction errors. Finally
given that the self-encoding reconstruction errors of some tested abnormal samples tend to be very small
a model that uses only self-encoding reconstruction errors for anomaly detection can miss many abnormal events. To further rule out anomalies
a one-class support vector machine (SVM) is trained on low-dimensional features
Result
2
Several experiments are performed in an actual manufacturing environment operated by robots. Two common indicators are used for evaluation
namely
area under ROC (AUC) and equal error rate (EER).The receiver operating characteristic (ROC) curve is drawn by using the results obtained from various classification standards and can be used to evaluate classifier performance. Meanwhile
the AUC represents the coverage area under the ROC curve
whereas the EER can be represented by the point where the ROC curve intersects with a 45° straight line. A smaller EER indicates a better detection effect. When the appropriate error threshold is set(approximately 0.15)
the AUC under the ROC curve reaches 91.7%
whereas the EER is 13.8%.The performance of the proposed model is also evaluated and compared with that of other models on public data feature sets University of California
San Diego (USCD) Ped1 and Ped2. In the USCD Ped1 dataset
the proposed model demonstrates 2.6% and 22.3% improvements in its AUC at the frame and pixel levels
respectively. In the same dataset
compared with the second-best method
the proposed model has a 5.7% higher AUC at the frame level
thereby verifying its effectiveness and accuracy.
Conclusion
2
The proposed video abnormal event detection model combines traditional and deep learning models to increase the accuracy of video abnormal event detection results. A 3D convolutional neural network (C3D) was used to extract the spatiotemporal features. A video anomaly event detection method based on deep spatiotemporal features was also developed by combining the stacked denoising autoencoder with a one-class SVM model. In extracting deep spatiotemporal features through a pre-trained C3D network
those features that were extracted from the last convolutional layer of the network were treated as the features of the spatiotemporal interest block. These features consider both the appearance and motion modes. A denoising auto-encoder was also trained to reduce the dimensions of C3D-extracted features
and the reconstruction error of an auto-encoder was used to facilitate the detection of abnormal events. Experimental results show that the proposed model can still detect anomalies when such events appear in partially occluded situations. Therefore
this model can be used for anomalous event detection in dense scenes. Future studies may consider examining other network architectures
integrating multiple input data (e.g.
RGB or optical flow frames)
and introducing trajectory tracking methods to track obstructed objects and improve abnormality detection accuracy. The proposed framework is suitable for highly complex scenarios.
Benezeth Y, Jodoin P M, Saligrama V and Rosenberger C. 2009. Abnormal events detection based on spatio-temporal co-occurences//Proceedings of 2009 IEEE Conference on Computer Vision and Pattern Recognition. Miami: IEEE: 2458-2465[ DOI: 10.1109/cvpr.2009.5206686 http://dx.doi.org/10.1109/cvpr.2009.5206686 ]
Chen Y Q, Zhou X S and Huang T S. 2001. One-class SVM for learning in image retrieval//Proceedings of 2001 International Conference on Image Processing. Thessaloniki: IEEE: 34-37[ DOI: 10.1109/ICIP.2001.958946 http://dx.doi.org/10.1109/ICIP.2001.958946 ]
Chong Y S and Tay Y H. 2017. Abnormal event detection in videos using spatiotemporal autoencoder//Proceedings of the 14th International Symposium on Advances in Neural Networks. Sapporo: Springer: 189-196[ DOI: 10.1007/978-3-319-59081-3_23 http://dx.doi.org/10.1007/978-3-319-59081-3_23 ]
Cong Y, Yuan J S and Liu J. 2011. Sparse reconstruction cost for abnormal event detection//Proceedings of 2011 IEEE Conforence on Computer Vision and Pattern Recognition. Providence: IEEE: 3449-3456[ DOI: 10.1109/CVPR.2011.5995434 http://dx.doi.org/10.1109/CVPR.2011.5995434 ]
Cong Y, Yuan J S and Liu J. 2013. Abnormal event detection in crowded scenes using sparse representation. Pattern Recognition, 46(7):1851-1864[DOI:10.1016/j.patcog.2012.11.021]
Dalal N and Triggs B. 2005. Histograms of oriented gradients for human detection//Proceedings of 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. San Diego: IEEE: 886-893[ DOI: 10.1109/CVPR.2005.177 http://dx.doi.org/10.1109/CVPR.2005.177 ]
Gu X X, Cui J R and Zhu Q. 2014. Abnormal crowd behavior detection by using the particle entropy. Optik, 125(14):3428-3433[DOI:10.1016/j.ijleo.2014.01.041]
Hasan M, Choi J, Neumann J, Roy-Chowdhury A K and Davis L S. 2016. Learning temporal regularity in video sequences//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE: 733-742[ DOI: 10.1109/cvpr.2016.86 http://dx.doi.org/10.1109/cvpr.2016.86 ]
Helbing D and Molnár P. 1995. Social force model for pedestrian dynamics. Physical Review E, 51(5):#4282[DOI:10.1103/PhysRevE.51.4282]
Horn B K P and Schunck B G. 1981. Determining optical flow. Artificial Intelligence, 17(1/3):185-203[DOI:10.1016/0004-3702(81)90024-2]
Hu W M, Xiao X J, Fu Z Y, Xie D, Tan T N and Maybank S. 2006. A system for learning statistical motion patterns. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28(9):1450-1464[DOI:10.1109/TPAMI.2006.176]
Ji SW, Xu W, Yang M and Yu K. 2013. 3D convolutional neural networks for human action recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(1):221-231[DOI:10.1109/tpami.2012.59]
Jiang F, Yuan J S, Tsaftaris S A and Katsaggelos A K. 2011. Anomalous video event detection using spatiotemporal context. Computer Vision and Image Understanding, 115(3):323-333[DOI:10.1016/j.cviu.2010.10.008]
Johnson N and Hogg D. 1996. Learning the distribution of object trajectories for event recognition. Image and Vision Computing, 14(8):609-615[DOI:10.1016/0262-8856(96)01101-8]
Karpathy A, Toderici G, Shetty S, Leung T, Sukthankar R and Li F F. 2014. Large-scale video classification with convolutional neural networks//Proceedings of 2014 IEEE Conference on Computer Vision and Pattern Recognition. Columbus: IEEE: 1725-1732[ DOI: 10.1109/CVPR.2014.223 http://dx.doi.org/10.1109/CVPR.2014.223 ]
Kratz L and Nishino K. 2009. Anomaly detection in extremely crowded scenes using spatio-temporal motion pattern models//Proceedings of 2009 IEEE Conference on Computer Vision and Pattern Recognition. Miami: IEEE: 1446-1453[ DOI: 10.1109/CVPR.2009.5206771 http://dx.doi.org/10.1109/CVPR.2009.5206771 ]
Krizhevsky A, Sutskever I and Hinton G E. 2012. ImageNet classification with deep convolutional neural networks//Proceedings of the 25th International Conference on Neural Information Processing Systems. Red Hook: ACM: 1097-1105
Li W X, Mahadevan V and Vasconcelos N. 2013. Anomaly detection and localization in crowded scenes. IEEE Transactions on Pattern Analysis and Machine Intelligence, 36(1):18-32[DOI:10.1109/TPAMI.2013.111]
Lin H H, Deng J D, Woodford B J and Shahi A. 2016. Online weighted clustering for real-time abnormal event detection in video surveillance//Proceedings of the 24th ACM International Conference on Multimedia. New York: ACM: 536-540[ DOI: 10.1145/2964284.2967279 http://dx.doi.org/10.1145/2964284.2967279 ]
Liu W, Luo W X, Lian D Z and Gao S H. 2018. Future frame prediction for anomaly detection-a new baseline//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, USA: IEEE: 6536-6545[ DOI: 10.1109/cvpr.2018.00684 http://dx.doi.org/10.1109/cvpr.2018.00684 ]
Lu C W, Shi J P and Jia J Y. 2013a. Abnormal event detection at 150 FPS in MATLAB//Proceedings of 2013 IEEE International Conference on Computer Vision. Sydney: IEEE: 2720-2727[ DOI: 10.1109/iccv.2013.338 http://dx.doi.org/10.1109/iccv.2013.338 ]
Lu C W, Shi J P and Jia J Y. 2013b. Online robust dictionary learning//Proceedings of 2013 IEEE Conference on Computer Vision and Pattern Recognition. Portland: IEEE: 415-422[ DOI: 10.1109/CVPR.2013.60 http://dx.doi.org/10.1109/CVPR.2013.60 ]
Mahadevan V, Li W X, Bhalodia V and Vasconcelos N. 2010. Anomaly detection in crowded scenes//Proceedings of 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. San Francisco: IEEE: 1975-1981[ DOI: 10.1109/CVPR.2010.5539872 http://dx.doi.org/10.1109/CVPR.2010.5539872 ]
Mehran R, Oyama A and Shah M. 2009. Abnormal crowd behavior detection using social force model//Proceedings of 2009 IEEE Conference on Computer Vision and Pattern Recognition. Miami: IEEE: 935-942[ DOI: 10.1109/CVPR.2009.5206641 http://dx.doi.org/10.1109/CVPR.2009.5206641 ]
Ren S Q, He K M, Girshick R and Sun J. 2015. Faster R-CNN: Towards real-time object detection with region proposal networks//Proceedings of the 28th International Conference on Neural Information Processing Systems. Cambridge: ACM: 91-99
Ribeiro M, Lazzaretti A E and Lopes H S. 2018. A study of deep convolutional auto-encoders for anomaly detection in videos. Pattern Recognition Letters, 105:13-22[DOI:10.1016/j.patrec.2017.07.016]
Saligrama V and Chen Z. 2012. Video anomaly detection based on local statistical aggregates//Proceedings of 2012 IEEE Conference on Computer Vision and Pattern Recognition. Providence: IEEE: 2112-2119[ DOI: 10.1109/cvpr.2012.6247917 http://dx.doi.org/10.1109/cvpr.2012.6247917 ]
Schlegl T, Seeböck P, Waldstein S M, Schmidt-Erfurth U and Langs G. 2017. Unsupervised anomaly detection with generative adversarial networks to guide marker discovery//Proceedings of the 25th International Conference on Information Processing in Medical Imaging. Boone: Springer: 146-157[ DOI: 10.1007/978-3-319-59050-9_12 http://dx.doi.org/10.1007/978-3-319-59050-9_12 ]
Sultani W, Chen C and Shah M. 2018. Real-world anomaly detection in surveillance videos//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, USA: IEEE: 6479-6488[ DOI: 10.1109/cvpr.2018.00678 http://dx.doi.org/10.1109/cvpr.2018.00678 ]
Tran D, Bourdev L, Fergus R, Torresani L and Paluri M. 2015. Learning spatiotemporal features with 3D convolutional networks//Proceedings of 2015 IEEE International Conference on Computer Vision. Santiago: IEEE: 4489-4497[ DOI: 10.1109/ICCV.2015.510 http://dx.doi.org/10.1109/ICCV.2015.510 ]
Vincent P, Larochelle H, Bengio Y and Manzagol P A. 2008. Extracting and composing robust features with denoising autoencoders//Proceedings of the 25th International Conference on Machine Learning. New York, USA: ACM: 1096-1103[ DOI: 10.1145/1390156.1390294 http://dx.doi.org/10.1145/1390156.1390294 ]
Wang S Q, Zeng Y J, Liu Q, Zhu C Z, Zhu E and Yin J P. 2018. Detecting abnormality without knowing normality: a two-stage approach for unsupervised video abnormal event detection//Proceedings of the 26th ACM International Conference on Multimedia. New York, USA: ACM: 636-644[ DOI: 10.1145/3240508.3240615 http://dx.doi.org/10.1145/3240508.3240615 ]
Xu D, Yan Y, Ricci E and Sebe N. 2017. Detecting anomalous events in videos by learning deep representations of appearance and motion. Computer Vision and Image Understanding, 156:117-127[DOI:10.1016/j.cviu.2016.10.010]
Zhao B, Li F F and Xing E P. 2011. Online detection of unusual events in videos via dynamic sparse coding//Proceedings of 2011 IEEE Conference on Computer Vision and Pattern Recognition. Providence: IEEE: 3313-3320[ DOI: 10.1109/CVPR.2011.5995524 http://dx.doi.org/10.1109/CVPR.2011.5995524 ]
Zhao Y R, Deng B, Shen C, Liu Y, Lu H T and Hua X S. 2017. Spatio-temporal AutoEncoder for video anomaly detection//Proceedings of the 25th ACM International Conference on Multimedia. New York, USA: ACM: 1933-1941[ DOI: 10.1145/3123266.3123451 http://dx.doi.org/10.1145/3123266.3123451 ]
Zhu X B, Liu J, Wang J Q, Fang Y K and Lu H Q. 2012. Anomaly detection in crowded scene via appearance and dynamics joint modeling//Proceedings of the 19th IEEE International Conference on Image Processing. Orlando: IEEE: 2705-2708[ DOI: 10.1109/ICIP.2012.6467457 http://dx.doi.org/10.1109/ICIP.2012.6467457 ]
Zivkovic Z. 2004. Improved adaptive Gaussian mixture model for background subtraction//Proceedings of the 17th International Conference on Pattern Recognition. Cambridge:IEEE:28-31[ DOI:10.1109/ICPR.2004.1333992 http://dx.doi.org/10.1109/ICPR.2004.1333992 ]]
相关作者
相关机构
京公网安备11010802024621
