Objective

2

Path tracking is not a new topic

being part of the various components of an autonomous vehicle that aim to steer the vehicle to track a defined path. The traditional steering controller

which uses location information and path information to steer autonomous vehicles

cannot achieve human-like driving behaviors according to real-life driving scenes or environments. When human-like steering behavior is considered a feature in the steering model

the steering problem of autonomous vehicles becomes challenging. The traditional steering controller tracks the defined path by predicting the steering angle of the front wheel according to the current location of the vehicles and the path information

but it is only a purely mechanical driving behavior rather than a human-like driving behavior. Thus

researchers employ a neural network as a steering model

training the neural network by using the images captured from the front-facing camera mounted on the vehicle along with the associated steering angles either from the perspective of human beings or simulators; this network is also known as end-to-end neural network. Nevertheless

most of the existing neural networks consider only the visual camera frames as input

ignoring other available information such as location

motion

and model of vehicle. The training dataset of the end-to-end neural network is supposed to cover all kinds of driving weather or scenes

such as rainy day

snow day

overexposure

and underexposure

so that the network can learn as much as possible the relationship between the image frames and driving behaviors

and enhance the universality of the neural network. The end-to-end neural network also relies on large-scale training datasets to enhance the robustness of the network. Overdependence on cameras results in the steering performance being greatly affected by the environment. Therefore

the combination of the traditional steering controller and end-to-end neural network can complement each other's advantages. With the use of only small-scale datasets that cover fewer driving scenes for training

the control behaviors of the new network can be human-like

robust

and able to cover multiple driving scenes. In this paper

we proposed a fusion neural network framework called deep pure pursuit (deep PP) to incorporate a convolutional neural network (CNN) with a traditional steering controller to build a robust steering model.

Method

2

In this study

a human-like steering model that fuses visual geometry group network (VGG)-type CNN and a traditional steering controller is built. The VGG-type CNN consists of 8 layers

including three convolutional layers

three pooling layers

and two fully connected layers. It uses 3×3 non-stride convolutions with 32

64

and 128 kernels are used. Following each convolutional layer

a 2×2 max-pooling layer with stride 2 is configured to decrease the used parameters. The fully connected layers are designed to function as a controller for steering. While CNN extracts visual features from video frames

PP is employed to utilize the location information and motion model information. Fifty target points of the defined path ahead of the vehicle are selected to calculate the predict front-wheel steering angle by PP. The minimum and maximum look-ahead distance of PP are separately set to 1.5 m and 20 m

respectively

ahead of the vehicle. After visual features from the CNN model and 50 steering angles from PP are extracted

a combinational feature vector is proposed to integrate visual features with 50 steering angles. The features are concatenated with the fully connected layers to build the mapping relationship. In our augmentation

the images are flipped and rotated to improve the self-recovery capacities from a poor location or orientation. In each image

the bottom 30 pixels and the top 40 pixels are cropped to remove the front of the car and most of the sky above the horizon

and then the processed images are resized to a lower resolution to accelerate the training and testing. Our model is implemented in Google's TensorFlow. The experiments are conducted on a Titan X GPU. The max number of epochs is set to 10. Each epoch contains 10 000 frames to train the model. The batch size is set to 32. Adam optimizer with learning rate 1E-4 is deployed to train our model. The activation function of our model is ReLU. Root mean square error(RMSE) was used to evaluate the performance of different models.

Result

2

To train and validate our proposed solution

we collect datasets by using CARLA(Center for Advanced Research on Language Acquisition) simulator and a real-life autonomous vehicle. In the simulation dataset

we trained the models under the ClearNoon weather parameter and evaluated on 14 instances of poor driving weather. In the real-life dataset

13 080 frames are collected for training

and 2 770 frames are collected for testing. We compared our model with a CNN model of the Udacity challenge and a traditional steering controller

PP

in verifying the effectiveness of deep PP. Experiment results show that our steering model can track the steering commands from the autopilot in CARLA more closely than CNN and PP can under 14 instances of poor driving conditions and improve the RMSE by 50.28% and 35.39%

separately. In real-life experiments

the proposed model is tested on a real-life dataset to prove its applicability. The discussion of different look-ahead distance demonstrates that PP controller is sensitive to the look-head distance. The maximal deviation from the human driver's steering commands reaches 0.245 2 rad. The discussion of location noise on the PP controller and deep PP proves that deep PP can better maintain robustness to location drift.

Conclusion

2

In this study

we proposed a fusion neural network framework that incorporates visual features from the camera with additional location information and motion model information. Experiment results show that our model can track the steering commands of autopilot or human driver more closely than the CNN model of the Udacity challenge and PP and maintained high robustness under 14 poor driving conditions.