According to the problem of vision based localization for the outdoor patrol robot，an approach of road crossing scene recognition based on scale invariant feature transform(SIFT) and color features is proposed in this paper.Firstly，the SIFT features are extracted and the color histogram in HSI space is calculated.Secondly，the K-D trees algorithm is used to match SIFT features of images in road crossing images database，and the Bhattacharyya distance match result is calculated using color histogram.Finally，the SIFT features match result and the Bhattacharyya distance match result are combined together to confirm the suitable image in database.The image pre-classified idea is also adopted to accelerate the SIFT features matching.The experiment results demonstrate that the algorithm is robust to the various illumination，dynamic disturbance and self-circumrotating，and can be used for robot location.