浏览全部资源
扫码关注微信
导出
分享
收藏
专辑
收稿：2019-05-24，
修回：2019-8-21，
录用：2019-8-28，
纸质出版：2020-01-16
移动端阅览
目的
2
针对古代壁画图像自身特征提取存在的主观单一性和客观不充分性等问题，以经典AlexNet网络模型为基础，提出了一种结合特征融合思想的卷积神经网络模型，用于古代壁画图像的自动分类。
方法
2
首先，由于大型壁画数据集较为缺乏，通过对壁画样本使用缩放、亮度变换、加噪和翻转等图像增强算法来扩大数据集，并提取壁画图像第1阶段的边缘等底层特征；其次，采用结构不同的双通道网络对提取的第1阶段特征进行第2阶段的深层抽象，得到两个通道的特征；最后，融合两个通道的特征，共同构建损失函数得到分类结果，从而提高模型的鲁棒性和特征表达能力。
结果
2
实验结果表明，在构造的壁画图像数据集上，该模型最终达到了85.39%的准确率。与AlexNet模型以及一些改进的卷积神经网络模型相比，各项评价指标均有大约5%的提高；与未进行预训练的经典模型相比，本文网络结构不易产生过拟合现象；与结合预训练的经典模型相比，准确率大致上有1%~5%的提升，从硬件条件、网络结构和内存消耗上来说代价更小。由此验证了本文模型对于壁画图像自动分类的合理性和有效性。
结论
2
本文提出的壁画分类模型，综合考虑网络宽度和深度的影响，能从多局部的角度提取壁画图像丰富的细节特征，具有一定的优势和使用价值，可进一步结合到与壁画图像分类的相关模型中。
Objective
2
Chinese ancient murals
as a type of painting on the wall
have a long history of 4000 years and are an indispensable part of Chinese ancient paintings. With the increasing abundance of digital mural images
classifying mural resources is becoming increasingly urgent. The core of mural image classification is how to construct the feature description of an object. In addition to expressing the object adequately
this description should be able to distinguish among different types of objects. However
ancient mural images have certain pluralism and subjectivity due to artificial drawing. Considering the subjective singularity and objective insufficiency of traditional mural image feature extraction
we propose a convolutional neural network based on classical AlexNet network model and feature fusion idea for the automatic classification of ancient mural images.
Method
2
First
we define the optimizer as Adam with a learning rate of 0.001 through experiments. We then extract each convolution layer feature of AlexNet for classification. Through the comparison of running time and accuracy
we select the convolution layer to express mural features further. Second
we combine the idea of feature fusion and exchange the two convolution kernels to form channels 1 and 2. The convolution kernels of channel 1 are 11
5
and 3
and those of channel 2 are 11
3
and 5. The combination of this method constitutes a two-channel convolution feature extraction layer
which enables the model to utilize multilocal features fully. The overfitting phenomenon caused by numerous full-connection layers is considered. On the basis of the two-channel convolution feature extraction layer
we continue to compare the features of different full-connection layers and select further appropriate full-connection layer features to express mural images. Finally
a mural image classification model with a two-channel convolution layer and optimal full connection layer is presented. The proposed mural image classification model can be divided into three processes. 1) Mural image preprocessing. Given the lack of large mural datasets
we use image enhancement operations
such as zooming
brightness transformation
noise addition
and flipping
to enlarge the mural samples. An ancient mural image dataset
including Buddha
Bodhisattva
Buddhist disciples
secular figures
animals
plants
buildings
and auspicious cloud
is constructed. 2) Training stage of mural image classification model. The module has three stages. In the first stage
the model extracts the low-level features
such as the edge information of the trainset images. In the second stage
the two-channel network with different structures is used to abstract the features of the first stage. The features of the two channels are then obtained. In the last stage
the loss function training network model is constructed by fusing the features of the two channels. Feature fusion improves the robustness of the model and the capability of feature expression. 3) Training stage of mural image classification model. We use the network model with trained parameters to predict the classification results of test set samples. The classification accuracy
recall
and f1-score are obtained.
Result
2
Through the comparison of running time and accuracy
the comparative experimental results of different convolution layers show that in the AlexNet model
the third convolution layer is the most suitable network layer for this dataset. In addition
the accuracy rate will decrease if the number of layers is higher or lower than the number of layers in the paper. Similarly
the comparative experimental results of different full-connection layers show that the features of the three-layer full-connection layer are further stable and sufficient based on the two-channel convolution extraction layer. Therefore
a six-layer convolution neural network model
including a three-layer dual channel and three-layer full connection layer
is presented with five convolution layers in the two-channel model. The model achieves 85.39% accuracy on the constructed mural image dataset. Experimental results show that the accuracy of the model in most classes is the highest
and each evaluation index of the model is improved by approximately 5%
compared with the AlexNet model and several improved convolution neural network models. Compared with the classical model without pretraining
this model encounters increased difficulty in producing overfitting. Compared with the model with pretraining
the accuracy of the model is improved by approximately 1%~5%
and the cost is reduced in terms of hardware conditions
network structure
and memory consumption. These experimental data verify the validity of the model for the automatic classification of mural images.
Conclusion
2
Considering the influence of network width and depth
ancient mural classification model with AlexNet model using feature fusion can fully express the rich details of mural images. This model has certain advantages and application value and can be further integrated into the mural classification-related model. However
this method is a shallow convolution neural network based on AlexNet
which fails to mine the high-level features of mural images fully. As a result
some images with similar low-level features
such as color and texture
cannot be classified correctly. Moreover
the running time of mural classification in this model is measured by hour
which consumes considerable resources and is inefficient. Therefore
we will combine deep models to express the high-level features of mural images in future work. We will also improve the efficiency of model training to make mural classification further effective and fast.
Almisreb A A, Jamil N and Din N M. 2018. Utilizing Alexnet deep transfer learning for ear recognition//Proceedings of the 2018 4th International Conference on Information Retrieval and Knowledge Management. Kota Kinabalu, Malaysia: IEEE: 1-5[ DOI:10.1109/infr km .2018.8464769 http://dx.doi.org/10.1109/infrkm.2018.8464769 ]
Alom M Z, Taha T M, Yakopcic C, Westberg S, Sidike P, Nasrin M S, van Esesn B C, Awwal A A S and Asari V K. 2018. The history began from AlexNet: a comprehensive survey on deep learning approaches[EB/OL]. 2018-03-03[2018-09-12] . https://arxiv.org/abs/1803.01164 https://arxiv.org/abs/1803.01164
Caldelli R, Becarelli R and Amerini I. 2017. Image origin classification based on social network provenance. IEEE Transactions on Information Forensics and Security, 12(6):1299-1308[DOI:10.1109/tifs.2017.2656842]
Chen C and Qi F. 2019. Review on development of convolutional neural network and its application in computer vision. Computer Science, 46(3):63-73
陈超, 齐峰. 2019.卷积神经网络的发展及其在计算机视觉领域中的应用综述.计算机科学, 46(3):63-73)[DOI:10.11896/j.issn.1002-137X.2019.03.008]
Hao Y B. 2017. Research and Implementation on Classification Algorthm with People of Ancient Chinese Murals Based on Style Characteristics. Tianjin:Tianjin University
郝亚博. 2017.基于风格特征的中国古代人物壁画分类算法研究与实现.天津:天津大学)
He K M, Zhang X Y, Ren S Q and Sun J. 2016. Deep residual learning for image recognition//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, USA: IEEE: 770-778[ DOI:10.1109/cvpr.2016.90 http://dx.doi.org/10.1109/cvpr.2016.90 ]
Krizhevsky A, Sutskever I and Hinton G E. 2012. ImageNet classification with deep convolutional neural networks//Proceedings of the 25th International Conference on Neural Information Processing Systems. Lake Tahoe, Nevada: Curran Associates Inc.: 1097-1105
Liu L M. 2016. Research of Rapid Three-dimensional Reconstruction Based Digital of Ultra-high Resolution Mural. Hangzhou:Zhejiang University
刘礼铭. 2016.基于3维重建的超高分辨率壁画快速数字化技术研究.杭州:浙江大学)
Liu X J. 2014. Research on Feature Extraction and Evolution Patterns of Auspicious Cloud in the Dunhuang Grotto murals. Wuhan:Wuhan University of Technology
刘喜姣. 2014.敦煌壁画中祥云文化元素的特征提取及演化规律研究.武汉:武汉理工大学)
Shin H C, Roth H R, Gao M C, Lu L, Xu Z Y, Nogues I, Yao J H, Mollura D and Summers R M. 2016. Deep convolutional neural networks for computer-aided detection:CNN architectures, dataset characteristics and transfer learning. IEEE Transactions on Medical Imaging, 35(5):1285-1298[DOI:10.1109/TMI.2016.2528162]
Simonyan K and Zisserman A. 2014. Very deep convolutional networks for large-scale image recognition[EB/OL]. 2014-09-24[2015-04-10] . https://arxiv.org/pdf/1409.1556.pdf https://arxiv.org/pdf/1409.1556.pdf
Sun T. 2018. Undertaking the Responsibility for the Revival of Chinese Mural Art. Art Research, (5):17-26
孙韬. 2018.中央美术学院托起中国壁画复兴之重任.美术研究, (5):17-26)[DOI:10.13318/j.cnki.msyj.2018.05.005]
Szegedy C, Vanhoucke V, Ioffe S, Shlens J and Wojna Z. 2016. Rethinking the inception architecture for computer vision//Proceeding of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, USA: IEEE, 2818-2826[ DOI:10.1109/cvpr.2016.308 http://dx.doi.org/10.1109/cvpr.2016.308 ]
Tang D W, Lu D M, Xu D Q and Yang B. 2014. Clustered multiple instance learning for mural image classification. Journal of Image and Graphics, 19(5):708-715
唐大伟, 鲁东明, 许端清, 杨冰. 2014.壁画图像分类中的分组多实例学习方法.中国图象图形学报, 19(5):708-715)[DOI:10.11834/jig.20140508]
Tang D W, Lu D M, Yang B and Xu D Q. 2013. Similarity metrics between mural images with constraints of the overall structure of contours. Journal of Image and Graphics, 18(8):968-975
唐大伟, 鲁东明, 杨冰, 许端清. 2013.轮廓整体结构约束的壁画图像相似性度量.中国图象图形学报, 18(8):968-975)[DOI:10.11843/jig.20130811]
Wang Q and Lu D M. 2012. Composition analysis-based relevance ranking for ancient mural. Journal of Zhejiang University (Engineering Science), 46(3):392-401
王琦, 鲁东明. 2012.基于构图分析的古代壁画相关度评价方法.浙江大学学报(工学版), 46(3):392-401)[DOI:10.3785/j.issn.1008-973x.2012.03.003]
Wei B G, Pan Y H and Hua Z. 1999. An analogy-based virtual approach for color restoration of wall painting. Journal of Computer Research and Development, 36(11):1364-1368
魏宝刚, 潘云鹤, 华忠. 1999.基于类比的壁画色彩虚拟复原.计算机研究与发展, 36(11):1364-1368)
Wei W, Qian W. 2013. Review on the application of 3D laser scanning in conservation and archaeology. Sciences of Conservation and Archaeology, 25(1):96-107
魏薇, 潜伟. 2013. 3维激光扫描在文物考古中应用述评.文物保护与考古科学, 25(1):96-107)
Xiao L S, Yan Q and Deng S Y. 2017. Scene classification with improved AlexNet model//Proceedings of the 2017 12th International Conference on Intelligent Systems and Knowledge Engineering. Nanjing, China: IEEE: 1-6[ DOI:10.1109/iske.2017.8258820 http://dx.doi.org/10.1109/iske.2017.8258820 ]
Zeng P P and Li L S. 2019. Classification and recognition of common fruit images based on convolutional neural network. Machine Design and Research, 35(1):23-26, 34
曾平平, 李林升. 2019.基于卷积神经网络的水果图像分类识别研究.机械设计与研究, 35(1):23-26, 34)[DOI:10.13952/j.cnki.jofmdr.2019.0093]
Zhang J W and Wang Q W. 2018. Digital representation and reconstruction of architecture in Dunhuang frescoes. Journal of Dunhuang Studies, 3(3):187-196
张加万, 王巧雯. 2018.敦煌壁画中建筑的数字化再现与重构.敦煌学辑刊, 3(3):187-196)[DOI:10.3969/j.issn.1001-6252.2018.03.015]
Zhang L C, Ma R and Zhang Y X. 2018. Application of improved lenet-5 model in apple image recognition. Computer Engineering and Design, 39(11):3570-3575
张力超, 马蓉, 张垚鑫. 2018.改进的LeNet-5模型在苹果图像识别中的应用.计算机工程与设计, 39(11):3570-3575)[DOI:10.16208/j.issn1000-7024.2018.11.048]
Zhong Z Q, Yuan J and Tang X Y. 2018. Left-vs-right eye discrimination based on convolutional neural network. Journal of Computer Research and Development, 55(8):1667-1673
钟志权, 袁进, 唐晓颖. 2018.基于卷积神经网络的左右眼识别.计算机研究与发展, 55(8):1667-1673)[DOI:10.7544/issn1000-1239.2018.20180215]
相关作者
相关机构
京公网安备11010802024621
