Unlike a mobile robot in an indoor-structured environment

an outdoor robot should recognize non-geometric terrain characteristics within a reasonable time and adjust the appropriate path

gait

and motion planning strategies to cope with the terrain. Visual terrain classification has become a hot topic in outdoor mobile robot research. The bag-of-visual-words (BOVW) framework

which can aggregate low-level visual descriptors and establish contact with semantic features

has become the most common approach and an effective paradigm for visual terrain classification. In this paper

we provide a comprehensive study of each step in the BOVW framework for visual terrain classification. Diverse methods in each step are introduced and summarized

and their characteristics and relations are explored. The BOVW framework includes four main steps: 1) feature extraction

2) codebook generation

3) feature coding

and 4) pooling and normalization. Feature extraction acquires low-level feature information from the terrain images to develop local descriptors. In the codebook generation step

a codebook is formed through clustering. The coding step uses the codebook to map the descriptors in the terrain image to the coding space. Then

coding results are aggregated into a single vector

that is

the mid-level feature

of the fixed length by pooling and normalization. Finally

the mid-level feature is fed into a linear or nonlinear classifier

such as SVM

for terrain classification. The diverse methods in each step are summarized and compared systematically. The performances of the method are preliminarily tested on a terrain dataset. The BOVW framework for visual terrain classification is reviewed in the paper. We also present a preliminary comparison of different BOVW frameworks for visual terrain classification on the terrain dataset. On the basis of the result

we find that every step is crucial in contributing to the final classification performance

and an improper choice in one step will markedly weaken the effectiveness and efficiency of the visual classification system as a whole. New handcrafted descriptors that are specific to the visual terrain

modified BOVW framework

and feature fusion are three potential research directions. Visual terrain classification is an important technology for recognizing non-geometric terrain characteristics for outdoor mobile robots. Compared with other sensors

visual information most closely resembles the manner by which humans perceive the environment and provides richer terrain information

and visual terrain classification has become a hotspot issue in outdoor mobile robot technology. However

visual appearances of the same terrain type may exhibit vast differences

and various types of terrain may appear highly similar. Therefore

these issues engender numerous challenges to visual terrain classification. Both effectiveness and efficiency are necessary factors that should be taken into account in the design of the visual terrain classification system. Therefore

studies on the BOVW for visual terrain classification are of considerable significance.