浏览全部资源
扫码关注微信
收稿：2018-09-07，
修回：2019-3-28，
纸质出版：2019-09-16
移动端阅览
目的
2
目前深度神经网络已成功应用于众多机器学习任务，并展现出惊人的性能提升效果。然而传统的深度网络和机器学习算法都假定训练数据和测试数据服从的是同一分布，而这种假设在实际应用中往往是不成立的。如果训练数据和测试数据的分布差异很大，那么由传统机器学习算法训练出来的分类器的性能将会大大降低。为了解决此类问题，提出了一种基于多层校正的无监督领域自适应方法。
方法
2
首先利用多层校正来调整现有的深度网络，利用加法叠加来完美对齐源域和目标域的数据表示；然后采用多层权值最大均值差异来适应目标域，增加网络的表示能力；最后提取学习获得的域不变特征来进行分类，得到目标图像的识别效果。
结果
2
本文算法在Office-31图像数据集等4个数字数据集上分别进行了测试实验，以对比不同算法在图像识别和分类方面的性能差异，并进行准确度测量。测试结果显示，与同领域算法相比，本文算法在准确率上至少提高了5%，在应对照明变化、复杂背景和图像质量不佳等干扰情况时，亦能获得较好的分类效果，体现出更强的鲁棒性。
结论
2
在领域自适应相关数据集上的实验结果表明，本文方法具备一定的泛化能力，可以实现较高的分类性能，并且优于其他现有的无监督领域自适应方法。
Objective
2
With the continuous development of computer technology in recent years
image recognition technology has become one of the most active research topics in the field of computer vision. The image recognition method generates a robust classification model by using existing training samples. Compared with traditional methods
deep learning algorithms have shown superior learning and classification performance by forming abstract and high-level representation through the combination of low-level features. Traditional deep networks assume that training data and test data follow the same distribution
but the same is not true in real-world applications. At the same time
collecting and annotating data sets for each new task or new domain is an expensive and time-consuming process
and the data available for training are limited. If the distributions of training data and test data greatly differ
then the performance of the classifiers trained by traditional machine learning algorithms diminishes considerably
and the advantages of deep network architectures relying on large tag data sets could no longer be maximized. Solving these problems requires cross-domain learning. Cross-domain learning refers to the transfer and learning of knowledge under different data distribution conditions by using the link between existing knowledge and experience to promote the learning of new tasks and ultimately reduce the impact of differences in the sample distributions between domains. The domain adaptive algorithm is used to solve the problem in which the performance of a classifier is degraded due to differences in the distributions of training samples and test samples. Such algorithm has become an effective approach to deal with domain migration problems in image recognition due to the relaxation of the same distribution requirements between fields in traditional machine learning. Its goal is to overcome the differences between training sample and test sample distributions
improve the performance of the training model by using the commonality between fields
achieve classifier migration
and ultimately improve classification accuracy.
Method
2
An unsupervised domain adaptive method based on multi-layer correction is proposed. A five-layer neural network structure is established by modifying and optimizing the ResNet-18 network. The correction is used to amend the internal representation of the target data to simulate the source data. For the residual layer of the target data
the data classifier of the source domain must adapt to the target domain by additive correction
and additive superposition is employed to perfectly align the data representations of the source and target domains. If the prior distribution of the class is not considered
then the weight deviation of the class can be easily ignored
thereby decreasing domain adaptation performance. Hence
we introduce the auxiliary weight of a specific class to override the set source sample. In this way
the re-weighted source data share the same category weight as the target data. On the basis of the results
we use multiple weight MMD (maximum mean discrepancy) to modify the fully connected layer and increase the representation capability of the network. Finally
the domain invariant features obtained by the learning process are extracted and classified to obtain the final recognition effect of the target image.
Result
2
Test experiments on digital datasets
such as Office-31 image dataset and MNIST
are carried out to confirm the validity of the proposed method. We compare the performances of different algorithms by measuring their classification accuracies in image and digital recognition. For the image dataset
we test two traditional transfer learning methods
three depth domain adaptive methods
and two mainstream deep neural network models. Test results show that our method has higher recognition ability than existing methods and is superior to other methods by 9.5%
on the average
in terms of classification accuracy. For the digital dataset
we test the convolutional neural network classic model LeNet
the subspace alignment method for domain adaptation (SA)
and the deep adaptive network model (DAN). Test results show that the proposed method has an average increase of 9.5% in classification accuracy and the best classification performance. In response to disturbances
such as illumination changes
complex backgrounds
and poor image quality
the proposed method can obtain better classification results and show stronger robustness than other methods.
Conclusion
2
Deep neural networks have good application prospects in image recognition. The domain adaptive method is also an effective method to deal with domain migration problems in image recognition. With the continuous deepening of network structure and expansion of data samples
the cost and time loss of daily training are also increasing. The unsupervised domain adaptive method has become an important processing method in image recognition because it does not require target sample tags. The application of the domain adaptive method to image recognition and other tasks carries theoretical significance. A multi-layer corrected network structure
which is a new method for unsupervised domain adaptation
is proposed. The additional layer increases the capacity of the neural network because of its excellent generalization performance. The experimental results on the domain adaptive correlation datasets show that the proposed method can learn the complete domain invariant representation for domain adaptive problems
achieves high classification performance
and is superior to other existing unsupervised domain adaptive methods.
Pan S J, Yang Q. A survey on transfer learning[J]. IEEE Transactions on Knowledge and Data Engineering, 2010, 22(10):1345-1359.[DOI:10.1109/TKDE.2009.191]
Long M S, Wang J M, Cao Y, et al. Deep learning of transferable representation for scalable domain adaptation[J]. IEEE Transactions on Knowledge and Data Engineering, 2016, 28(8):2027-2040.[DOI:10.1109/TKDE.2016.2554549]
Bruzzone L, Marconcini M. Domain adaptation problems:a DASVM classification technique and a circular validation strategy[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2010, 32(5):770-787.[DOI:10.1109/TPAMI.2009.57]
Tang S, Ye M, Li X D. Domain adaptation object recognition[J]. ZTE Technology Journal, 2017, 23(4):25-31.
唐宋, 叶茂, 李旭冬.领域自适应目标识别综述[J].中兴通讯技术, 2017, 23(4):25-31.[DOI:10.3969/j.issn.1009-6868.2017.04.005]
Lunga D, Yang H L, Reith A, et al. Domain-adapted convolutional networks for satellite image classification:a large-scale interactive learning workflow[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2018, 11(3):962-977.[DOI:10.1109/JSTARS.2018.2795753]
Sohn K, Liu S F, Zhong G Y, et al. Unsupervised domain adaptation for face recognition in unlabeled videos[C]//Proceedings of 2017 IEEE International Conference on Computer Vision. Venice, Italy: IEEE, 2017: 5917-5925.[ DOI: 10.1109/ICCV.2017.630 http://dx.doi.org/10.1109/ICCV.2017.630 ]
Chen Y H, Li W, Sakaridis C, et al. Domain adaptive faster R-CNN for object detection in the wild[EB/OL]. 2018-03-08[2018-08-12] . https://arxiv.org/pdf/1803.03243.pdf https://arxiv.org/pdf/1803.03243.pdf .
Tzeng E, Hoffman J, Zhang N, et al. Deep domain confusion: maximizing for domain invariance[EB/OL]. 2014-12-10[2018-08-12] . https://arxiv.org/pdf/1412.3474.pdf https://arxiv.org/pdf/1412.3474.pdf .
Long M S, Cao Y, Wang J M, et al. Learning transferable features with deep adaptation networks[EB/OL]. 2015-05-27[2018-08-12] . https://arxiv.org/pdf/1502.02791.pdf https://arxiv.org/pdf/1502.02791.pdf .
Long M S, Zhu H, Wang J M, et al. Deep transfer learning with joint adaptation networks[EB/OL]. 2016-05-21[2018-08-12] . https://arxiv.org/pdf/1605.06636.pdf https://arxiv.org/pdf/1605.06636.pdf .
Li S, Song S J, Wu C. Layer-wise domain correction for unsupervised domain adaptation[J]. Frontiers of Information Technology&Electronic Engineering, 2018, 19(1):91-103.[DOI:10.1631/FITEE.1700774]
Yosinski J, Clune J, Bengio Y, et al. How transferable are features in deep neural networks?[EB/OL]. 2014-11-06[2018-08-12] . https://arxiv.org/pdf/1411.1792.pdf https://arxiv.org/pdf/1411.1792.pdf .
He K M, Sun J. Convolutional neural networks at constrained time cost[C]//Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition. Boston, MA, USA: IEEE, 2015: 5353-5360.[ DOI: 10.1109/CVPR.2015.7299173 http://dx.doi.org/10.1109/CVPR.2015.7299173 ]
He K M, Zhang X Y, Ren S Q, et al. Deep residual learning for image recognition[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, NV, USA: IEEE, 2016: 770-778.[ DOI: 10.1109/CVPR.2016.90 http://dx.doi.org/10.1109/CVPR.2016.90 ]
Long M S, Zhu H, Wang J M, et al. Unsupervised domain adaptation with residual transfer networks[EB/OL]. 2016-02-14[2018-08-12] . https://arxiv.org/pdf/1602.04433.pdf https://arxiv.org/pdf/1602.04433.pdf .
Borgwardt K M, Gretton A, Rasch M J, et al. Integrating structured biological data by kernel maximum mean discrepancy[J]. Bioinformatics, 2006, 22(14):e49-e57.[DOI:10.1093/bioinformatics/btl242]
Gretton A, Borgwardt K M, Rasch M J, et al. A kernel two-sample test[J]. Journal of Machine Learning Research, 2012, 13:723-773.
Li Y J, Swersky K, Zemel R. Generative moment matching networks[EB/OL]. 2015-02-10[2018-08-12] . https://arxiv.org/pdf/1502.02761.pdf https://arxiv.org/pdf/1502.02761.pdf .
Gardner J R, Upchurch P, Kusner M J, et al. Deep manifold traversal: changing labels with convolutional features[EB/OL]. 2015-11-19[2018-08-12] . https://arxiv.org/pdf/1511.06421.pdf https://arxiv.org/pdf/1511.06421.pdf .
Wang Y N, Qin P L, Li C P, et al. Improved algorithm of image super resolution based on residual neural network[J]. Journal of Computer Application, 2018, 38(1):246-254.
王一宁, 秦品乐, 李传朋, 等.基于残差神经网络的图像超分辨率改进算法[J].计算机应用, 2018, 38(1):246-254.[DOI:10.11772/j.issn.1001-9081.2017061461]
Saenko K, Kulis B, Fritz M, et al. Adapting visual category models to new domains[C]//Proceedings of the 11th European Conference on Computer Vision. Heraklion, Crete, Greece: Springer, 2010: 213-226.[ DOI: 10.1007/978-3-642-15561-1_16 http://dx.doi.org/10.1007/978-3-642-15561-1_16 ]
Lécun Y, Bottou L, Bengio Y, et al. Gradient-based learning applied to document recognition[J]. Proceedings of the IEEE, 1998, 86(11):2278-2324.[DOI:10.1109/5.726791]
Hull J J. A database for handwritten text recognition research[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1994, 16(5):550-554.[DOI:10.1109/34.291440]
Netzer Y, Wang T, Coates A, et al. Reading digits in natural images with unsupervised feature learning[C]//Proceedings of the NIPS Workshop on Deep Learning and Unsupervised Feature Learning. Granada, Spain: NIPS, 2011: 1-9.
Pan S J, Tsang I W, Kwok J T, et al. Domain adaptation via transfer component analysis[J]. IEEE Transactions on Neural Networks, 2011, 22(2):199-210.[DOI:10.1109/TNN.2010.2091281]
Gong B Q, Shi Y, Sha F, et al. Geodesic flow kernel for unsupervised domain adaptation[C]//Proceedings of 2012 IEEE Conference on Computer Vision and Pattern Recognition. Providence, RI, USA: IEEE, 2012: 2066-2073.[ DOI: 10.1109/CVPR.2012.6247911 http://dx.doi.org/10.1109/CVPR.2012.6247911 ]
He K M, Zhang X Y, Ren S Q, et al.Delving deep into rectifiers: surpassing human-level performance on ImageNet classification[C]//Proceedings of 2015 IEEE International Conference on Computer Vision. Santiago, Chile: IEEE, 2015: 1026-1034.[ DOI: 10.1109/ICCV.2015.123 http://dx.doi.org/10.1109/ICCV.2015.123 ]
Mikolov T, Sutskever I, Chen K, et al. Distributed representations of words and phrases and their compositionality[C]//Proceedings of the 26th International Conference on Neural Information Processing Systems. Lake Tahoe, Nevada, USA: Curran Associates Inc., 2013: 3111-3119.
Kingma D P, Ba J. Adam: a method for stochastic optimization[EB/OL]. 2014-12-22[2018-08-12] . https://arxiv.org/pdf/1412.6980.pdf https://arxiv.org/pdf/1412.6980.pdf .
Fernando B, Habrard A, Sebban M, et al. Unsupervised visual domain adaptation using subspace alignment[C]//Proceedings of 2013 IEEE International Conference on Computer Vision. Sydney, NSW, Australia: IEEE, 2013: 2960-2967.
相关作者
相关机构
京公网安备11010802024621
