Gauss Markov random field model takes advantage of both image intensity and spatial information imposed by Gibbs smoothness prior to the pixel labels and thus can be used to effectively in segmenting the noisy imagesHowever it is always difficult to confirm the Gibbs penalty factor βAs usual，to get good segmentation result for every segmenting to be image，various values of β will be tested by handSo to solve this problem，this article defines a new and simple class adaptive penalty factor βIt is automatically calculated from the posterior probability and is anisotropic for each classFurthermore the model iteratively obtains their parameters estimation in the EM MAP algorithmFinally，by application of this algorithm to brain MR Image segmentation，the proposed segmentation scheme is proved effective for noisy image and at the same time it distinguishes itself by higher correct classification ratio and correct classification ratio for each class