Objective

2

In image set classification

symmetric positive definite (SPD) matrices are usually utilized to model image sets.The resulting Riemannian manifold yields a high discriminative power in many visual recognition tasks.However

existing classic classification algorithms are mostly applied in the Euclidean space and cannot work directly on SPD matrices.To apply the classification algorithm of Euclidean space to image set classification

this work comprehensively reviews the unique Log-Euclidean metric (LEM) of the SPD manifold and the properties of the existing classical classification algorithm

and the classification task based on the image sets is achieved.

Method

2

Given that the SPD matrices lie on Riemannian space

we map the samples on the SPD manifold to the tangent space through logarithm mapping

and each sample in the tangent space corresponds to an image set.The form of the samples in the tangent space is a symmetrical matrix

and its dimensionality conforms with the samples on the SPD manifold.The symmetric matrix in the tangent space contains redundant information and has a large dimension.To improve the performance and efficiency of the algorithm

we need to reduce the dimensionality of the data in the tangent space.In our technique

we use the Nyström method and (2D)

2

PCA to obtain low-dimensional data that contain the main information of the image sets.1) The Nyström method can approximate the infinite-dimensional samples in the reproducing kernel Hilbert space (RKHS).The dimensionality of the samples mapped into the RKHS by kernel mapping is infinite

and the Riemannian kernel is obtained by the inner product of the samples in the tangent space using the LEM of the SPD manifold.For a set of

$$M$$

training samples

the Riemannian kernel matrix

$$\mathit{\boldsymbol{K}} = {[k({\mathit{\boldsymbol{x}}_i}

{\mathit{\boldsymbol{x}}_j})]_{M \times M}}$$

can be written as

$$\mathit{\boldsymbol{K}} \cong {\mathit{\boldsymbol{Z}}^{\bf{T}}}\mathit{\boldsymbol{Z}} = \mathit{\boldsymbol{V}}{\mathit{\boldsymbol{ \boldsymbol{\varSigma} }}^{1/2}}{\mathit{\boldsymbol{ \boldsymbol{\varSigma} }}^{1/2}}{\mathit{\boldsymbol{V}}^{\bf{T}}}$$

.Here

$${\mathit{\boldsymbol{Z}}_{d \times M}} = {\mathit{\boldsymbol{ \boldsymbol{\varSigma} }}^{1/2}}{\mathit{\boldsymbol{V}}^{\bf{T}}}$$

$$\mathit{\boldsymbol{ \boldsymbol{\varSigma} }}$$

and

$$\mathit{\boldsymbol{V}}$$

are the top

$$d$$

eigenvalues and the eigenvectors of

$$\mathit{\boldsymbol{K}}$$

where

$$d$$

is the rank of the kernel matrix

$$\mathit{\boldsymbol{K}}$$

.The projection matrix can be denoted as

$${\mathit{\boldsymbol{ \boldsymbol{\varSigma} }}^{-1/2}}{\mathit{\boldsymbol{V}}^{\bf{T}}}$$

and the

$$d$$

-dimensional vector approximation of the random sample

$$\mathit{\boldsymbol{y}}$$

in the RKHS can be written as

$${\mathit{\boldsymbol{ \boldsymbol{\varSigma} }}^{-1/2}}{\mathit{\boldsymbol{V}}^{\bf{T}}}{(k(\mathit{\boldsymbol{y}}

{\mathit{\boldsymbol{x}}_1})

\ldots

k(\mathit{\boldsymbol{y}}

{\mathit{\boldsymbol{x}}_M}))^{\rm{T}}}$$

.2) (2D)

2

PCA (two-directional two-dimensional PCA) is a well-known dimensionality reduction (DR) technique for two-dimensional data in machine learning and pattern recognition.(2D)

2

PCA overcomes the limitation of PCA of working only on one-dimensional data and of 2DPCA being used for the row and column DR of two-dimensional data to obtain two direction projection matrices.In our experiments

the row direction projection matrix

$${\mathit{\boldsymbol{W}}_{\rm{R}}}$$

is consistent with column direction projection matrix

$${\mathit{\boldsymbol{W}}_{\rm{C}}}:\mathit{\boldsymbol{W}} = {\mathit{\boldsymbol{W}}_{\rm{R}}} = {\mathit{\boldsymbol{W}}_{\rm{C}}}$$

in which

$${\mathit{\boldsymbol{W}}_{D \times d}}$$

is a projection matrix for the row and column directions

because the form of the samples in the tangent space is a symmetrical matrix.The sample

$$\mathit{\boldsymbol{x}} \in {{\bf{R}}^{D \times D}}$$

can be reduced as

$$\mathit{\boldsymbol{x}}\prime = {\mathit{\boldsymbol{W}}^{\bf{T}}}\mathit{\boldsymbol{xW}}$$

where

$$\mathit{\boldsymbol{x}}\prime \in {{\bf{R}}^{d \times d}}$$

and an efficient low-dimensional representation of the sample in the tangent space is achieved.To this end

the SPD matrices are transformed into low-dimensional descriptors with respect to the corresponding image sets.The classical sparse representation classification algorithm

Fisher discrimination dictionary learning in Euclidean space

which has good recognition rates for single images

can be utilized to classify the points of low-dimensional descriptors.

Result

2

Our approach is used for several tasks

including face identification

object classification

and virus cell recognition

and we experiment on the YouTube celebrities (YTC)

ETH-80

and Virus datasets.Results show that our algorithm has not only a higher recognition rate but also a relatively smaller standard deviation than several classical algorithms for image set classification

such as covariance discriminant learning (CDL) and projection metric learning.In the experiment on the ETH-80 dataset

our approach achieves a recognition rate of 96.25%

which indicates a considerable improvement over those of discriminative canonical correlations (DCC)

CDL

and other classic methods.The standard deviation is only 2.12

which is smaller than that of the other methods; this finding indicates that our method has the best robustness on the ETH-80 dataset.For the YTC dataset

the recognition rate of our method is 78.26%

which is 10% higher than that of the other methods.This result shows the evident advantage of our method for the YTC dataset.The standard deviation is smallest

which shows that our method also has the best robustness for the YTC dataset.However

for the Virus dataset

our method has the highest recognition rate 58.67% but a large standard deviation.This finding indicates that the robustness of our method for this dataset is insufficient.

Conclusion

2

In this work

we consider the geometrical properties of the SPD manifolds and the related Riemannian metric and combine them with the classical classification algorithm of the Euclidean space to implement image set classification and achieve good results.Experimental results show that our proposed method achieves high accuracies and generally small standard deviation.Therefore

our method can be widely applied in image set classification.Our future work will focus on constructing a lower-dimensional and more discriminative SPD manifold than that in this research.