浏览全部资源
扫码关注微信
收稿：2021-07-26，
修回：2021-11-18，
录用：2021-11-25，
纸质出版：2022-03-16
移动端阅览
目的
2
旷场实验(open field test，OFT)是行为学与药理实验分析中常用的实验方法。为了对比测试组和参考组被测小鼠的行为特征差异，通常需要耗费大量精力对旷场实验数据进行处理和观测。由于旷场实验数据量大且较依赖观测人员的主观判断，导致对小鼠行为差异观测的精度较低且缺乏量化评价指标。为此，本文提出一种基于卷积神经网络(convolutional neural networks，CNN)的旷场实验视频分类方法，可基于量化特征对两组小鼠的行为差异自动分类。
方法
2
从视频空域和时域中提取22维的小鼠运动行为特征，经过量化后生成特征矩阵，进而以矩阵拼接方式构造可学习的行为特征矩阵样本，利用不同结构卷积神经网络对提取的行为特征矩阵样本进行训练和分类，并分析网络结构对分类结果的影响，在实现两组小鼠分类的基础上，对不同维度小鼠行为特征对分类精度的重要性进行评价。
结果
2
在真实旷场实验数据集上的实验分析表明，本文算法的分类准确率为99.25%。此外，由实验结果分析发现小鼠的大角度转向频次、停留区域与时间对小鼠分类的重要性高于其他维度特征。
结论
2
提出的特征拼接矩阵学习方法能够准确识别两组小鼠旷场实验视频的差异，本文方法的分类准确率明显优于现有人工分析及经典机器学习方法。
Objective
2
open field test (OFT) video classification is a wide spread method to assess locomotors activity and mice habits control in pharmacological experiments. One major goal of traditional OFT is to sort out distinguishable features between a testing group and a reference group. Researchers will analyze those video clips to find out animal behaviors differences for the two groups of mice based on digital OFT videos records. The manual inspection of OFT data is time consumed and high cost involved. A detailed analysis process relies heavily on professional experiences. Our research illustrates a learning-based classification through an OFT-videos-based quantitative feature extraction. We show that a spliced feature matrix learning method has its priority than current independent features. Hence
a convolutional neural network based solution sort out the oriented video clips of two groups of mice. Our analyzed results can be summarized as following: 1) a novel spliced feature matrix of OFT video is demonstrated based on 22-dimension quantitative behavioral features. 2) To analyze the influence of network structure on the classification
we design and test 8 different convolutional neural networks in the experiments.
Method
2
Our research focuses on the varied characteristics between the two groups of mice macroscopic behavior. In the context of the spatial and temporal domain of OFT video data
22 distinct types of features are extracted including average crawling speed and distance
the positional distribution of staying
resting time
turning frequencies and so on. These quantitative vector-based descriptions of the OFT video can be well distinguished based on traditional machine learning classifiers
such as K-means clustering
boosting classification
support vector machine (SVM)
or convolutional neural network (CNN). Some quantitative feature vectors are proposed to separate the testing group from the reference group. Another critical factor is the mice crawling path
which cannot be vector-oriented. A novel regularization and fusion are employed to illustrate both the quantitative features and non-quantitative crawling path. By constructing the targeted self-correlation matrix of weighted feature vectors
a 484 × 484 quantitative feature matrix is spliced with a 484 × 484 crawling path image
and it leads a spliced feature matrix representation with a 2D dimension of 484 × 968. We introduce a CNN-based classification of spliced feature matrices from different network structures. Based on 4-layers CNN to 10-layers CNN
the quantitative feature matrix or the spliced feature matrices are learned via the basic network structures
which are we evaluate the impacts of different network structures and feature dimensions on the precision of OFT video classification further. In the experiments
we test the demonstrated feature extraction and classification on the real OFT dataset. The dataset is composed of OFT video of 32 mice
including 17 mice in the test group and 15 mice in the reference group. The testing animal group is injected with a kind of anti-depressants
and the reference group injected with only placebo. Each mouse is eye-captured independently using a video camera for a 24 hours timescale detection in the OFT experiment. These video data is cropped into targeted short duration video clips
each of which is 10 minutes long. We construct a dataset based on 3 080 testing samples and 1 034 reference samples. In the training analysis
the training dataset is based on 3 000 samples and the testing dataset is based on 1 114 samples.
Result
2
Our demonstration indicates that the proposed algorithm is optimized to manual classification and supported-vector-machine solution. In respect of the experiment datasets
the classification precision of the proposed algorithm is up to 99.25%. The accurate classification of mouse OFT video can be achieved via a simplified network structure
such as a 9-layers CNN. The varied feature dimension to classification accuracy is not balanced. In terms of the ablation results among different quantitative features
the large angle turning
staying time and positional distribution are critical for identifying its grouping than other quantitative feature dimensions. Non-quantitative crawling path image has obvious contribution to classification
which improves 2%—3% precision in classification. The demonstration also verifies can achieve good classification via a sample network structure.
Conclusion
2
Overall
we propose a CNN-based solution to classify the testing group OFT videos from the reference group of OFT videos via the spliced feature matrix learning. Both the quantitative features and qualitative crawling path are melted into a novel regularization and fusion. The identified classification has optimized on manual classification and traditional SVM methods. Our classification precision can reach 99.25% on the experimental dataset. It demonstrates a great potential for the OFT analysis. The further research work can verify the relationship between distinguishable features and detailed mouse behavior. The learning network optimization can be further called to enhance the generalization of the OFT classification.
Brattoli B, Tighe J, Zhdanov F, Perona P and Chalupka K. 2020. Rethinking zero-shot video classification: end-to-end training for realistic applications//Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle, USA: IEEE: 4612-4622 [ DOI: 10.1109/CVPR42600.2020.00467 http://dx.doi.org/10.1109/CVPR42600.2020.00467 ]
Burgos-Artizzu X P, Dollár P, Lin D Y, Anderson D J and Perona P. 2012. Social behavior recognition in continuous video//Proceedings of 2012 IEEE Conference on Computer Vision and Pattern Recognition. Providence, USA: IEEE: 1322-1329 [ DOI: 10.1109/CVPR.2012.6247817 http://dx.doi.org/10.1109/CVPR.2012.6247817 ]
Carreira J and Zisserman A. 2017. Quo vadis, action recognition? A new model and the kinetics dataset//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, USA: IEEE: 4724-4733 [ DOI: 10.1109/CVPR.2017.502 http://dx.doi.org/10.1109/CVPR.2017.502 ]
Choleris E, Thomas A W, Kavaliers M and Prato F S. 2001. A detailed ethological analysis of the mouse open field test: effects of diazepam, chlordiazepoxide and an extremely low frequency pulsed magnetic field. Neuroscience and Biobehavioral Reviews, 25(3): 235-260 [DOI: 10.1016/S0149-7634(01)00011-2]
Cortes C and Vapnik V. 1995. Support-vector networks. Machine Learning, 20(3): 273-297 [DOI: 10.1007/BF00994018]
Decker C M and Hamprecht F A. 2014. Detecting individual body parts improves mouse behavior classification//Proceedings of the Workshop on Visual Observation and Analysis of Vertebrate and Insect Behavior, Stockholm, Sweden: #2089814
Deng L and Yu D. 2014. Deep learning: methods and applications. Foundations and TrendsⓇ in Signal Processing, 7(3/4): 197-387 [DOI: 10.1561/2000000039]
Durand T, Thome N and Cord M. 2015. MANTRA: minimum maximum latent structural SVM for image classification and ranking//Proceedings of 2015 IEEE International Conference on Computer Vision. Santiago, Chile: IEEE: 2713-2721 [ DOI: 10.1109/ICCV.2015.311 http://dx.doi.org/10.1109/ICCV.2015.311 ]
Feng W G, Gao J, Buckles B P and Wu K W. 2011. Wireless capsule endoscopy video classification using an unsupervised learning approach. Journal of Image and Graphics, 16(11): 2041-2046
冯文刚, 高隽, Buckles B P, 吴克伟. 2011. 无监督学习的无线胶囊内诊镜视频分类方法. 中国图象图形学报, 16(11): 2041-2046 [DOI: 10.11834/jig.20111113]
Freund Y and Schapire R E. 1997. A decision-theoretic generalization of on-line learning and an application to boosting. Journal of Computer and System Sciences, 55(1): 119-139 [DOI: 10.1006/jcss.1997.1504]
Giancardo L, Sona D, Huang H P, Sannino S, Managò F, Scheggia D, Papaleo F and Murino V. 2013. Automatic visual tracking and social behaviour analysis with multiple mice. PLoS One, 8(9): e74557 [DOI: 10.1371/journal.pone.0074557]
Gu J X, Ding X Q and Wang S J. 2009. A survey of activity analysis algorithms. Journal of Image and Graphics, 14(3): 377-387
谷军霞, 丁晓青, 王生进. 2009. 行为分析算法综述. 中国图象图形学报, 14(3): 377-387 [DOI: 10.11834/jig.20090301]
Hall C S. 1934. Emotional behavior in the rat. I. Defecation and urination as measures of individual differences in emotionality. Journal of Comparative Psychology, 18(3): 385-403 [DOI: 10.1037/h0071444]
Hong W Z, Kennedy A, Burgos-Artizzu X P, Zelikowsky M, Navonne S G, Perona P and Anderson D J. 2015. Automated measurement of mouse social behaviors using depth sensing, video tracking, and machine learning. Proceedings of the National Academy of Sciences of the United States of America, 112(38): E5351-E5360 [DOI: 10.1073/pnas.1515982112]
Jiang Y G, Wu Z X, Wang J, Xue X Y and Chang S F. 2018. Exploiting feature and class relationships in video categorization with regularized deep neural networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 40(2): 352-364 [DOI: 10.1109/TPAMI.2017.2670560]
Karpathy A, Toderici G, Shetty S, Leung T, Sukthankar R and Li F F. 2014. Large-scale video classification with convolutional neural networks//Proceedings of 2014 IEEE conference on Computer Vision and Pattern Recognition. Columbus, USA: IEEE: 1725-1732 [ DOI: 10.1109/CVPR.2014.223 http://dx.doi.org/10.1109/CVPR.2014.223 ]
Katsageorgiou V M, Zanotto M, Huang H P, Ferretti V, Papaleo F, Sona D and Murino V. 2016. Unsupervised mouse behavior analysis: a data-driven study of mice interactions//Proceedings of the 23rd International Conference on Pattern Recognition (ICPR). Cancun, Mexico: IEEE: 925-930 [ DOI: 10.1109/ICPR.2016.7899754 http://dx.doi.org/10.1109/ICPR.2016.7899754 ]
Lin W H and Hauptmann A. 2002. News video classification using SVM-based multimodal classifiers and combination strategies//Proceedings of the 10th ACM International Conference on Multimedia. Juan-les-Pins, France: ACM: 323-326 [ DOI: 10.1145/641007.641075 http://dx.doi.org/10.1145/641007.641075 ]
Lu B, Wei X K and Bi D Y. 2005. Applications of support vector machines in classification. Journal of Image and Graphics, 10(8): 1029-1035
陆波, 尉询楷, 毕笃彦. 2005. 支持向量机在分类中的应用. 中国图象图形学报, 10(8): 1029-1035 [DOI: 10.11834/jig.200508190]
Monteiro J P, Oliveira H P, Aguiar P and Cardoso J S. 2014. A depth-map approach for automatic mice behavior recognition//Proceedings of 2014 IEEE International Conference on Image Processing (ICIP). Paris, France: IEEE: 2261-2265 [ DOI: 10.1109/ICIP.2014.7025458 http://dx.doi.org/10.1109/ICIP.2014.7025458 ]
Nair A, Srinivasan P, Blackwell S, Alcicek C, Fearon R, De Maria A, Panneershelvam V, Suleyman M, Beattie C, Petersen S, Legg S, Mnih V, Kavukcuoglu K and Silver D. 2015. Massively parallel methods for deep reinforcement learning [EB/OL]. https://arxiv.org/pdf/1507.04296.pdf https://arxiv.org/pdf/1507.04296.pdf
Nam H and Han B. 2016. Learning multi-domain convolutional neural networks for visual tracking//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, USA: IEEE: 4293-4302 [ DOI: 10.1109/CVPR.2016.465 http://dx.doi.org/10.1109/CVPR.2016.465 ]
Ng J Y H, Hausknecht M, Vijayanarasimhan S, Vinyals O, Monga R and Toderici G. 2015. Beyond short snippets: deep networks for video classification//Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition. Boston, USA: IEEE: 4694-4702 [ DOI: 10.1109/CVPR.2015.7299101 http://dx.doi.org/10.1109/CVPR.2015.7299101 ]
Nováková J, Vinklerová J, Kellnerová R and Šulcová A. 1997. Computer analysis of video-taped agonistic behaviour in mice: effects of interferon-alfa, diazepam and naloxone. Scripta Medica, 70(7): 267-274
Schölkopf B, Platt J C, Shawe-Taylor J, Smola A J and Williamson R C. 2001. Estimating the support of a high-dimensional distribution. Neural Computation, 13(7): 1443-1471 [doi: 10.1162/089976601750264965]
Schuldt C, Laptev I and Caputo B. 2004. Recognizing human actions: a local SVM approach//Proceedings of the 17th International Conference on Pattern Recognition. Cambridge, UK: IEEE: 32-36 [ DOI: 10.1109/ICPR.2004.1334462 http://dx.doi.org/10.1109/ICPR.2004.1334462 ]
Stanford S C. 2007. The open field test: reinventing the wheel. Journal of Psychopharmacology, 21(2): 134-135 [DOI: 10.1177/0269881107073199]
Svetnik V, Liaw A, Tong C, Culberson J C, Sheridan R P and Feuston B P. 2003. Random forest: a classification and regression tool for compound classification and QSAR modeling. Journal of Chemical Information and ComputerSciences, 43(6): 1947-1958 [DOI: 10.1021/ci034160g]
Walsh R N and Cummins R A. 1976. The open-field test: a critical review. Psychological Bulletin, 83(3): 482-504 [DOI: 10.1037/0033-2909.83.3.482]
Wang J, Yang Y, Mao J H, Huang Z H, Huang C and Xu W. 2016. CNN-RNN: a unified framework for multi-label image classification//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, USA: IEEE: 2285-2294 [ DOI: 10.1109/CVPR.2016.251 http://dx.doi.org/10.1109/CVPR.2016.251 ]
Xu Z Q, Hu J N and Deng W H. 2016. Recurrent convolutional neural network for video classification// Proceedings of 2016 IEEE International Conference on Multimedia and Expo (ICME). Seattle, USA: IEEE: 1-6 [ DOI: 10.1109/ICME.2016.7552971 http://dx.doi.org/10.1109/ICME.2016.7552971 ]
Zhou W S, Vellaikal A and Kuo C C J. 2000. Rule-based video classification system for basketball video indexing//Proceedings of 2000 ACM Workshops on Multimedia. Los Angeles, California, USA: ACM: 213-216 [ DOI: 10.1145/357744.357941 http://dx.doi.org/10.1145/357744.357941 ]
相关作者
相关机构
京公网安备11010802024621
