浏览全部资源
扫码关注微信
收稿：2019-12-19，
修回：2020-4-14，
录用：2020-4-21，
纸质出版：2020-09-16
移动端阅览
目的
2
度量学习是机器学习与图像处理中依赖于任务的基础研究问题。由于实际应用背景复杂，在大量不可避免的噪声环境下，度量学习方法的性能受到一定影响。为了降低噪声影响，现有方法常用L1距离取代L2距离，这种方式可以同时减小相似样本和不相似样本的损失尺度，却忽略了噪声对类内和类间样本的不同影响。为此，本文提出了一种非贪婪的鲁棒性度量学习算法——基于L2/L1损失的边缘费歇尔分析（marginal Fisher analysis based on L2/L1 loss，MFA-L2/L1），采用更具判别性的损失，可提升噪声环境下的识别性能。
方法
2
在边缘费歇尔分析（marginal Fisher analysis，MFA）方法的基础上，所提模型采用L2距离刻画相似样本损失、L1距离刻画不相似样本损失，同时加大对两类样本的惩罚程度以提升方法的判别性。首先，针对模型非凸带来的求解困难，将目标函数转为迭代两个凸函数之差便于求解；然后，受DCA（difference of convex functions algorithm）思想启发，推导出非贪婪的迭代求解算法，求得最终度量矩阵；最后，算法的理论证明保证了迭代算法的收敛性。
结果
2
在5个UCI（University of California Irrine）数据集和7个人脸数据集上进行对比实验：1）在不同程度噪声的5个UCI数据集上，MFA-L2/L1算法最优，且具有较好的抗噪性，尤其在30%噪声程度的Seeds和Wine数据集上，与次优方法LDA-NgL1（non-greedy L1-norm linear discriminant analysis））相比，MFA-L2/L1的准确率高出9%；2）在不同维度的AR和FEI人脸数据集上的实验，验证了模型采用L1损失、采用L2损失提升了模型的判别性；3）在Senthil、Yale、ORL、Caltech和UMIST人脸数据集的仿真实验中，MFA-L2/L1算法呈现出较强鲁棒性，性能排名第1。
结论
2
本文提出了一种基于L2/L1损失的鲁棒性度量学习模型，并推导了一种便捷有效的非贪婪式求解算法，进行了算法收敛性的理论分析。在不同数据集的不同噪声情况下的实验结果表明，所提算法具有较好的识别率和鲁棒性。
Objective
2
Distance metric learning
which is task dependent
is an essential research issue in machine learning and image processing. As a usual preprocessing step for classification and recognition
distance metric learning method enhances the performance of machine learning methods
such as clustering and k-nearest neighbor. This kind of method aims to learn the underlying structure of distribution given the semantic similarity or labels of samples. The optimization goal of distance metric learning is to shrink the distance between similar pairs and expand the distance between dissimilar pairs. With the development of machine learning
many distance metric learning methods have been proposed and applied successfully. These methods use L2 distance to measure loss. However
the L2 distance amplifies the influence of noise with the square operation
because of the unavoidable noise resulting from the complex application background. This situation may make the observed dissimilar and similar pairs respectively appear to be further apart and closer than in fact
which misleads the learning. Thus
the existence of noise leads to the poor performance of distance metric learning in machine learning and pattern recognition. Existing robust distance metric learning methods usually adopt L1 distance rather than L2 distance
aiming to diminish the distance scale of pairs simultaneously and thus reduce the influence of noise. However
by adopting a kind of loss for similar and dissimilar pairs
these methods neglect the different optimization directions between intra- and interclasses. Therefore
a robust distance metric learning algorithm based on L2/L1 loss with nongreedy solution is proposed. This method adopts more discriminative loss and improves the recognition performance in the presence of feature noise.
Method
2
Based on marginal Fisher analysis (MFA) method
the proposed model adopts L2 and L1 distance to measure the loss of similar pairs and dissimilar pairs
respectively. In details
it considers the influence of noise on intraclass and interclass scatters separately. Affected by noise
the observed similar pairs may be closer than the truth
and the observed dissimilar pairs may be further apart than the truth
which may leads that the punishments on pairs is not enough. Adopting L2/L1 loss
L2 distance gives more punishments on similar samples than L1 distance does
and L1 distance gives more punishments on dissimilar samples than L2 distance does. However
the model is nonconvex
which is involved with both minimization of L2 norm and maximization of the L1 norm. Hence
the existing solution for trace ratio problems are not suitable for the objective function. This makes the object difficult to solve
thus a nongreedy solution is derived. First
the objective function
which is a ratio
is transformed into a difference of two convex functions during each iteration. This process makes the object easy to solve. Then
inspired by the idea of the difference of convex function algorithms (DCA)
an iterative algorithm with nongreedy solution is derived to solve the object
and a projection matrix is learned. Analyzing the algorithm
in fact
an auxiliary function of the object is generated during the iteration
which makes sure that the value of objective function decreases. Lastly
a theoretical analysis of the iterative algorithm is performed to ensure the convergence of the proposal.
Result
2
Synthetic experiments are conducted on five UCI(University of California Irrine) datasets and seven face datasets with noise. The nearest neighbor classifier based on the learned matrix is used to compare the performance of related methods. The proposed method outperforms other methods in terms of accuracy. First
when 5%
15%
25%
and 30% feature noise is added on five UCI datasets
MFA-L2/L1 achieves the best performance and exhibits robustness against noise. In addition
the accuracy of MFA-L2/L1 is 9% higher than the second-best method. Second
during the accuracy comparison with varying dimension of projection based on AR dataset
the proposal beats MFA
which validates that using the L1 loss for dissimilar pairs makes the proposal more discriminative. During the accuracy comparison with varying dimensions of projection based on FEI dataset
the proposal beats LDA-NgL1(non-greedy L1-norm liner discriminant analysis)
which validates that using the L2 loss for similar pairs makes the proposal more discriminative. Lastly
based on five face datasets with noise
MFA-L2/L1 behaves better than other methods in terms of robustness.
Conclusion
2
This study proposes a robust distance metric learning method based on L2/L1 loss
which adopts different losses for similar pairs and dissimilar pairs respectively. To solve a nonconvex object
the object is transformed into a difference of two convex functions during each iteration. By virtue of the idea of DCA
an iterative algorithm is derived
which generates an auxiliary function decreasing the object. The convergence of the algorithm is guaranteed by a theoretical proof. Based on the public datasets
several series of experiments with different levels and types of synthetic noise are executed. Results on different datasets show that the proposal performs well and is robust to noise based on the experiment datasets. Future research will focus on local distance metric learning in the presence of label noise
because the label noise makes the observed distribution deviate from the true distribution more than the feature noise does.
An L T H and Tao P D. 2005. The DC (difference of convex functions) programming and DCA revisited with DC models of real world nonconvex optimization problems. Annals of Operations Research, 133(1/4):23-46[DOI:10.1007/s10479-004-5022-1]
Belhumeur P N, Hespanha J P and Kriegman D J. 1997. Eigenfaces vs. fisherfaces:recognition using class specific linear projection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 19(7):711-720[DOI:10.1109/34.598228]
Bellet A, Habrard A and Sebban M. 2014. A survey on metric learning for feature vectors and structured data[EB/OL].[2019-12-13] . https://arxiv.org/pdf/1306.6709.pdf https://arxiv.org/pdf/1306.6709.pdf
Chen W J, Li C N, Shao Y H, Zhang J and Deng N Y. 2018. Robust L 1 -norm multi-weight vector projection support vector machine with efficient algorithm. Neurocomputing, 315:345-361[DOI:10.1016/j.neucom.2018.04.083].
Dai G X, Xie J and Fang Y. 2018. Deep correlated holistic metric learning for sketch-based 3D shape retrieval. IEEE Transactions on Image Processing, 27(7):3374-3386[DOI:10.1109/TIP.2018.2817042]
Dua D and Graff C. 2019. UCI machine learning repository[DB/OL].[2019-12-13] . http://archive.ics.uci.edu/ml http://archive.ics.uci.edu/ml
Graham D B and Allinson N M. 1998. Characterising virtual eigensignatures for general purpose face recognition//Wechsler H, Phillips P J, Bruce V, SouliéF F and Huang T S, eds. Face Recognition. Berlin Heidelberg: Springer: 446-456[ DOI:10.1007/978-3-642-72201-1_25 http://dx.doi.org/10.1007/978-3-642-72201-1_25 ]
Green B F. 1952. The orthogonal approximation of an oblique structure in factor analysis. Psychometrika, 17(4):429-440[DOI:10.1007/BF02288918]
Guillaumin M, Verbeek J and Schmid C. 2009. Is that you? Metric learning approaches for face identification//Proceedings of the 12th IEEE International Conference on Computer Vision. Kyoto: IEEE: 498-505[ DOI:10.1109/ICCV.2009.5459197 http://dx.doi.org/10.1109/ICCV.2009.5459197 ]
Hoi S C H, Liu W, Lyu M R and Ma W Y. 2006. Learning distance metrics with contextualconstraints for image retrieval//Proceedings of 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. New York: IEEE: 2072-2078[ DOI:10.1109/CVPR.2006.167 http://dx.doi.org/10.1109/CVPR.2006.167 ]
Hu J L, Lu J W and Tan Y P. 2014. Discriminative deep metric learning for face verification in the wild//Proceedings of 2014 IEEE Conference on Computer Vision and Pattern Recognition. Columbus: IEEE: 1875-1882[ DOI:10.1109/CVPR.2014.242 http://dx.doi.org/10.1109/CVPR.2014.242 ]
Hu J L, Lu J W and Tan Y P. 2016. Deep metric learning for visual tracking. IEEE Transactions on Circuits and Systems for Video Technology, 26(11):2056-2068[DOI:10.1109/TCSVT.2015.2477936]
Huo Q R, Lu Y, Liu Y and Chao J B. 2017. Metric learning for tracking utilizing phase congruency. Journal of Image and Graphics, 22(3):297-304
霍其润, 陆耀, 刘羽, 巢进波. 2017.结合相位一致性的度量学习跟踪.中国图象图形学报, 22(3):297-304)[DOI:10.11834/jig.20170303]
Ke Q F and Kanade T. 2005. Robust L1 norm factorization in the presence of outliers and missing data by alternative convex programming//Proceedings of 2005 IEEE Computer Society Conference on Computer vision and Pattern Recognition. San Diego: IEEE: 739-746[ DOI:10.1109/CVPR.2005.309 http://dx.doi.org/10.1109/CVPR.2005.309 ]
Köstinger M, Hirzer M, Wohlhart P, Roth P M and Bischof H. 2012. Large scale metric learning from equivalence constraints//Proceedings of 2012 IEEE Conference on Computer vision and Pattern Recognition. Providence: IEEE: 2288-2295[ DOI:10.1109/CVPR.2012.6247939 http://dx.doi.org/10.1109/CVPR.2012.6247939 ]
Kwak N. 2008. Principal component analysis based on L1-norm maximization. IEEE Transactions on Pattern Analysis and Machine Intelligence, 30(9):1672-1680[DOI:10.1109/TPAMI.2008.114]
Liao S C, Hu Y, Zhu X Y and Li S Z. 2015. Person re-identification by local maximal occurrence representation and metric learning//Proceedings of 2015 IEEE Conference on Computer vision and Pattern Recognition. Boston: IEEE: 2197-2206[ DOI:10.1109/CVPR.2015.7298832 http://dx.doi.org/10.1109/CVPR.2015.7298832 ]
Liong V E, Lu J W, Tan Y P and Zhou J. 2017. Deep coupled metric learning for cross-modal matching. IEEE Transactions on Multimedia, 19(6):1234-1244[DOI:10.1109/TMM.2016.2646180]
Little M A, McSharry P E, Roberts S J, Costello D A and Moroz I M. 2007. Exploiting nonlinear recurrence and fractal scaling properties for voice disorder detection. Biomedical Engineering Online, 6: #23[ DOI:10.1186/1475-925X-6-23] http://dx.doi.org/10.1186/1475-925X-6-23] .
Liu Y, Gao Q X, Miao S, Gao X B, Nie F P and Li Y S. 2017. A non-greedy algorithm for L1-norm LDA. IEEE Transactions on Image Processing, 26(2):684-695[DOI:10.1109/TIP.2016.2621667]
Lu J W, Hu J L and Zhou J. 2017. Deep metric learning for visual understanding:an overview of recent advances. IEEE Signal Processing Magazine, 34(6):76-84[DOI:10.1109/MSP.2017.2732900]
Martinez A M and Kak A C. 2001. PCA versus LDA. IEEE Transactions on Pattern Analysis and Machine Intelligence, 23(2):228-233[DOI:10.1109/34.908974]
Moutafis P, Leng M J and Kakadiaris I A. 2017. An overview and empirical comparison of distance metric learning methods. IEEE Transactions on Cybernetics, 47(3):612-625[DOI:10.1109/TCYB.2016.2521767]
Nie F P, Huang H, Ding C, Luo D J and Wang H. 2011. Robust principal component analysis with non-greedy L1-norm maximization//Proceedings of the 22nd International Joint Conference on Artificial Intelligence. Barcelona, Spain: AAAI: 1433-1438
Samaria F S and Harter A C. 1994. Parameterisation of a stochastic model for human face identification//Proceedings of 1994 IEEE Workshop on Applications of Computer Vision. Sarasota: IEEE: 138-142[ DOI:10.1109/ACV.1994.341300 http://dx.doi.org/10.1109/ACV.1994.341300 ]
Shi Y, Bellet A and Sha F. 2014. Sparse compositional metric learning//Proceedings of the 28th AAAI Conference on Artificial Intelligence. Québec: AAAI: 2078-2084
Thomaz C E and Giraldi G A. 2010. A new ranking method for principal components analysis and its application to face image analysis. Image and Vision Computing, 28(6):902-913[DOI:10.1016/j.imavis.2009.11.005]
Wang H, Nie F P and Huang H. 2014. Robust distance metric learning via simultaneous L1-norm minimization and maximization//Proceedings of the 31st International Conference on Machine Learning. Beijing: JMLR: 1836-1844
Weber M. 1999. Frontal face dataset[DB/OL].[2019-12-13] . https://wenku.baidu.com/view/f1fc60ec81c758f5f61f672a.html https://wenku.baidu.com/view/f1fc60ec81c758f5f61f672a.html
Weinberger K Q and Saul L K. 2009. Distance metric learning for large margin nearest neighbor classification. Journal of Machine Learning Research, 10:207-244
Xiang S M, Nie F P and Zhang C S. 2008. Learning a Mahalanobis distance metric for data clustering and classification. Pattern Recognition, 41(12):3600-3612[DOI:10.1016/j.patcog.2008.05.018]
Xing E P, Ng A Y, Jordan M I and Russell S. 2002. Distance metric learning with application to clustering with side-information//Proceedings of Advances in Neural Information Processing Systems. Vancouver: MIT Press: 521-528
Xiong F, Gou M R, Camps O and Sznaier M. 2014. Person re-identification using kernel-based metric learning methods//Proceedings of the 13th European Conference on Computer Vision. Zurich: Springer: 1-16[ DOI:10.1007/978-3-319-10584-0_1 http://dx.doi.org/10.1007/978-3-319-10584-0_1 ]
Xu D, Yan S C, Tao D C, Lin S and Zhang H J. 2007. Marginal fisher analysis and its variants for human gait recognition and content-based image retrieval. IEEE Transactions on Image Processing, 16(11):2811-2821[DOI:10.1109/TIP.2007.906769]
Ye Q L, Fu L Y, Zhang Z, Zhao H H and Naiem M. 2018. Lp-and Ls-norm distance based robust linear discriminant analysis. Neural Networks, 105:393-404[DOI:10.1016/j.neunet.2018.05.020]
Zhang J and Zhao X. 2017. Global-local metric learning for person re-identification. Journal of Image and Graphics, 22(4):472-481
张晶, 赵旭. 2017.整合全局-局部度量学习的人体目标再识别.中国图象图形学报, 22(4):472-481)[DOI:10.11834/jig.20170407]
Zhang L, Wei W, Tian C N, Li F and Zhang Y N. 2016a. Exploring structured sparsity by a reweighted Laplace prior for hyperspectral compressive sensing. IEEE Transactions on Image Processing, 25(10):4974-4988[DOI:10.1109/TIP.2016.2598652]
Zhang L, Xiang T and Gong S G. 2016b. Learning a discriminative null space for person re-identification//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern recognition. Las Vegas: IEEE: 1239-1248[ DOI:10.1109/CVPR.2016.139 http://dx.doi.org/10.1109/CVPR.2016.139 ]
Zhang S P, Qi Y K, Jiang F, Lan X Y, Yuen P C and Zhou H Y. 2018. Point-to-set distance metric learning on deep representations for visual tracking. IEEE Transactions on Intelligent Transportation Systems, 19(1):187-198[DOI:10.1109/TITS.2017.2766093]
Zhong F J and Zhang J S. 2013. Linear discriminant analysis based on L1-norm maximization. IEEE Transactions on Image Processing, 22(8):3018-3027[DOI:10.1109/TIP.2013.2253476]
相关作者
相关机构
京公网安备11010802024621
