Objective

2

In a real-time driving process

the vehicle must be positioned to complete the basic tasks of horizontal and vertical control. The premise of vehicle positioning problems is to understand road information. Road information includes all kinds of traffic signs

among which lane line is an important pavement information in the road scene. Such information is crucial for lane maintenancing

departure warning

and path planning; it is also important in research on advanced driving assistance systems. Therefore

lane line detection has become an important topic in real-time vehicle driving. Road scene images can be obtained using a vehicle camera

lidar

and other equipments

thus making it easy to obtain lane line images. However

lane line detection suffers from some difficulties. Traditional lane line detection methods usually design features manually. Starting from the bottom features

such as color

brightness

shape

and gray level

the method involves image processing via denoising

binarization

and graying. Then

the lane line features are extracted by combining edge detection

Hough transform

color threshold setting

perspective transform

and other methods. Afterward

the lane lines are fitted by straight or curve line models. These methods are simple and easy to implement

but the accuracy of lane line detection is poor under the influence of multiscene environment conditions

such as object occlusion

light change

and shadow interference; moreover

the manual design of features is time consuming and thus cannot meet the real-time requirements of vehicle driving. To solve these problems

this study proposes a lane detection model named efficient residual factorized network-auxiliary loss(ERFNet-AL)

which embeds an auxiliary loss.

Method

2

The model improves the semantic segmentation network of ERFNet. After the encoder of ERFNet

a lane prediction branch and an auxiliary training branch are added to make the decoding phase parallel with the lane prediction and auxiliary training branches. After the convolution layer of the auxiliary training branch

bilinear interpolation is used to match the resolution of input images to classify four lane lines and the background of images. The training set images in the dataset are sent to the lane line detection model after preprocessing

such as clipping

rotating

scaling

and normalization. The features are extracted through the semantic segmentation network ERFNet

thereby obtaining the probability distribution of each lane line. The auxiliary training branch uses a convolution operation to extract features

and bilinear interpolation is used to replace the deconvolution layer after the convolution layer to match the resolution of the input images and classify the four lane lines and background. After using convolution

batch normalization

dropout layers

and other operations

the lane line prediction branch predicts the existence of lane lines or virtual lane lines and outputs the probability value of lane line classification. An output probability value greater than 0.5 indicates the existence of lane lines. If at least one lane line exists

then a probability distribution map of the corresponding lane lines must be determined. On the probability distribution map of each lane line

the coordinates of the largest point with a probability greater than a specific threshold is identified by row

and the corresponding coordinate points is selected in accordance with the rules of selecting points in the SCNN (spatial convolutional neural network) model. If the number of points found are greater than 2

then these points are connected to form a fitted lane line. Then

the cross-entropy loss between the predicted value of the auxiliary training branch output and the real label is calculated and used as the auxiliary loss. The weight of all four lane lines is 1

and the weight of the background is 0.4. The auxiliary

semantic segmentation

and lane prediction losses are weighted and summed in accordance with a certain weight

and the network parameters are adjusted via backpropagation. Among them

the total loss includes the main

auxiliary

and lane prediction losses. During training

the weights of ERFNet on the Cityscapes dataset are used as pretraining weights. During training

the model with the largest mean intersection over union is taken as the best model.

Result

2

After testing in nine scenarios of the CULane public dataset

the F1 index of the model in the normal scenario is found to be 91.85%

which is a 1.25% increase compared with that of the SCNN model (90.6%). Moreover

the F1 index in seven scenes

including crowded

night

no line

shadow

arrow

dazzle light

and curve scenarios

is increased by 1%~7%; the total average F1 value in nine scenarios is 73.76%

which is 1.96% higher than the best ResNet-101-self-attention distillation (SAD) model; the average run time of each image is 11.1 ms

which is 11 times shorter than the average running time of the spatial CNN model when tested on a single GPU of GeForce GTX 1 080; the parameter quantity of the model is only 2.49 MB

which is 7.3 times less than that of the SCNN model. On the CULane dataset

ENet with SAD is the lane line detection model with the shortest average run time of a single-image test. The average run time of this model is 13.4 ms

whereas that of our model is 11.1 ms. Compared with ENet with SAD

the average running time is reduced by 2.3 ms. When detecting lane lines at a crossroad scenario

the number of false positive is large

which may be due to the large number of lane lines at crossroads

whereas only four lane lines are detected in our experiment.

Conclusion

2

In various complex scenarios

such as object occlusion

lighting changes

and shadow interference

the model is minimally affected by the environment for real-time driving vehicles

and its accuracy and real-time performance are improved. The next work will aim to increase the number of lane lines

optimize the model

and improve the model's detection performance at crossroads.