Objective

2

Intelligent mobile robots are widely used in industry

logistics

home service

and other fields. From complex industrial robots to simple sweeping robots

the ability of simultaneous localization and mapping is essential. Taking the common low-cost sweeping robot as an example

the scheme adopted is to obtain the distance information between the robot and an object in a plane through 2D lidar

and establish occupation grid map by using laser SLAM(simultaneous localization and mapping) to support robot navigation

path planning

and other functions. With the increasing demand of intelligent service

the geometric map with simple information cannot meet the needs of people. Tasks such as "cleaning the chair" and "going to the refrigerator" require the robot to perceive the environment from the geometric level to the content level. In addition to describing the geometric contour of the environment through grid map

the intelligent robot should have the functions of target recognition

semantic segmentation

or scene classification to obtain semantic information. SLAM is a key technology for a mobile robot to explore

perceive

and navigate in an unknown environment

which can be divided into laser SLAM and visual SLAM. Laser SLAM is accurate and convenient for robot navigation and path planning

but it lacks semantic information. The image of visual SLAM can provide rich semantic information with a higher feature discrimination

but the map constructed by visual SLAM cannot be directly used for path planning and navigation. A semantic grid mapping method based on laser camera system is proposed in this paper to realize the construction of semantic map for mobile robot and path planning.

Method

2

To construct a semantic map that can be used for path planning

this paper uses a monocular camera assisted lidar to extract the object level features and the bounding box segmentation matching algorithm to obtain the semantic laser segmentation data as well as participate in the construction of the map. When the robot constructs the occupation grid map

the grid that stores the semantic information is called the semantic grid. The semantic map updates the occupation probability of the grid corresponding to each object and the semantic information in the grid. Then

through the steps of global optimization

semantic grid clustering

and semantic grid annotation

the semantic grid map with object category and contour is obtained. In addition

semantic tasks are published on the semantic grid map

the semantic weighting algorithm is used to identify the easily moving objects in the environment

and the path planning is improved. This system is mainly divided into three parts: semantic laser data extraction

semantic grid mapping and path planning. The input of the system includes the scanning data of 2D lidar and the pictures of monocular camera

and the output is the semantic grid map that can be used for path planning. Semantic laser segmentation data extraction is based on the laser camera system. The laser radar provides the scanning data in a certain height plane in the space. The module projects the laser segmentation in the camera field of view to the image and matches with the detection frame to obtain the semantic laser segmentation data. When the laser segmentation data are acquired

the semantic grid mapping should be carried out simultaneously. When the laser data are used to construct the grid map

the corresponding objects in the grid are marked with semantics and contour according to the semantic information. Moreover

density-based spatial clustering of applications with noise(DBSCAN) clustering algorithm is used to cluster the grids of the same object category and obtain the grid set representing each independent object. Finally

according to the semantic information of objects in the grid set

the corresponding object positions occupying the grid map are marked with different colors and words

and the semantic grid map is obtained. The semantic grid map contains geometric information and content information of environment

which can provide specific semantic objects for robot path planning and assist robot navigation. Adaptive Monte Carlo localization (AMCL) is used to locate a mobile robot in a 2D environment

and the pose information of a robot is determined. The global path planning uses A

*

algorithm to plan the global path from the starting point to the target given any target in the map and optimizes it according to the semantic weighting algorithm.

Result

2

The hardware of the system consists of a mobile robot and a control machine. The robot platform is composed of a kobuki chassis

an Robo Sense(RS) lidar

and a monocular camera. The laptop of the control computer is configured with 2.5 GHz main frequency

Intel Core i5-7300 HQ processor

GTX 1050ti GPU

and 8 G memory. Semantic mapping and path planning system are deployed as software facilities and run in the robot operating system(ROS) environment. The test system is Ubuntu 16.04. First

the performance of semantic laser data extractions is evaluated

and the detection accuracy of different object detection

laser segmentation

and bounding box segmentation matching is measured. Second

the semantic grid mapping experiment realizes the semantic grid mapping of mobile robot in the corridor and office. Results show that the semantic map in this paper integrates the object-level semantics well on the basis of occupying the grid map

enriches the map content

and improves the readability of the map. Finally

in the path planning experiment

the semantic grid map can perceive the environment content

provide semantic information for robot navigation and path planning

and support the intelligent service of robot. Compared with the route in the original map

the path after semantic discrimination is more flexible and more suitable for the situation of mobile objects in map building.

Conclusion

2

Experiments in various environments prove that the method proposed can obtain a semantic grid map with a high consistency with the real environment and labeled target information

and the experimental hardware structure is simple

low cost with good performance

and suitable for the navigation and intelligent robotic path plan.