Choosing the right dictionary used for sparse coding has an important effect on image reconstruction and pattern classification. Therefore

a new sparse representation algorithm based on Gabor Feature Set Discriminative Dictionary Learning is proposed for image recognition. Considering that Gabor feature is robust to variations of illumination

expression

and pose

the proposed method first extracts the image Gabor features with multi-scale and multi-orientation.Then it uses the augmented Gabor local feature matrix whose dimension has been reduced to construct the initial feature dictionary. This reduction is based on the Fisher discrimination criterion. A structural dictionary

whose atoms correspond to the class labels

is learned so that each sub-dictionary of the learned new dictionary is a good representation of the samples from the corresponding class. Meanwhile

the Fisher discrimination criterion is imposed on the coding coefficients so that they have small within-class scatter but big between-class scatter. Consequently

a new classification scheme associated with the proposed method is then presented by using

the discriminative information and sparse coding coefficients. Experiments on three types of databases show that the proposed method is valid and efficient.