A study has been made on the algorithm of solving optimal set of discriminant vectors in this paper. A concise representation method of between-class scatter matrix and population scatter matrix is proposed theoretically based on theories of blocking matrix and optimization under certain conditions. A new algebraic method of feature extraction is presented. The most obvious advantage of the proposed algorithm is that the computation time decreases drastically. The statement is supported by the numerical simulation experiments on facial database of ORL. The experimental results indicate that high recognition rate can be obtained through the appropriate selection of the dimension of block matrix although there exists nonlinear relationship between recognition rate and dimension of block matrix. The proposed concise representation method of scatter matrix suits for all the applications of pattern recognition using Fisher criteria.