A conditional random field (CRF) model is used to incorporate different feature potentials of objects for multi-class object recognition and segmentation in natural images.By using an over-segmentation algorithm，we propose a new region based CRF model called R-CRF model.We train our model on annotated samples by using Joint-boost algorithm and investigate the performance of the theme based R-CRF model for class based pixel-wise segmentation of images.We compare our results with recent published results on the MSRC 21-class database.The result shows that our theme based R-CRF model significantly outperforms the current state-of-the-art.Especially，by introducing theme and regions，our model obtains greatly improved accuracy of structured classes with high visual variability and fewer training examples.