Current Issue Cover
基于视觉感知的智能扫地机器人的垃圾检测与分类

宁凯1, 张东波1,2, 印峰1,2, 肖慧辉1(1.湘潭大学信息工程学院, 湘潭 411105;2.机器人视觉感知与控制技术国家工程实验室, 长沙 410012)

摘 要
目的 为了提高扫地机器人的自主性和智能化程度,为扫地机器人配备视觉传感器,使其获得视觉感知能力,通过研究有效的垃圾检测分类模型与算法,实现对垃圾的定位与识别,引导扫地机器人对垃圾进行自动识别与按类处理,提高工作的目的性和效率,避免盲动和减少能耗。方法 选择检测速度较快的YOLOv2作为主网络模型,结合密集连接卷积网络,嵌入深层密集模块,对YOLOv2进行改进,提出一种YOLOv2-dense网络,该网络可以充分利用图像的高分辨率特征,实现图像浅层和深层特征的复用与融合。结果 测试结果表明,智能扫地机器人使用本文方法可以有效识别不同形态的常见垃圾类别,在真实场景中,测试识别准确率为84.98%,目标检测速度达到26帧/s。结论 实验结果表明,本文构建的YOLOv2-dense网络模型具有实时检测的速度,并且在处理具有不同背景、光照、视角与分辨率的图片时,表现出较强的适应和识别性能。在机器人移动过程中,可以保证以较高的准确率识别出垃圾的种类,整体性能优于原YOLOv2模型。
关键词
Garbage detection and classification of intelligent sweeping robot based on visual perception

Ning Kai1, Zhang Dongbo1,2, Yin Feng1,2, Xiao Huihui1(1.College of Information Engineering, Xiangtan University, Xiangtan 411105, China;2.Robot Visual Perception & Control Technology National Engineering Laboratory, Changsha 410012, China)

Abstract
Objective Home service robots have attracted widespread attention in recent years due to their close relationship with the daily lives of humans. Sweeping robots are the first home service robots that have entered the consumer market and are available extensively. At present, the intelligent sweeping robots on the market have only basic functions such as automatic path planning, automatic charging, and automatic obstacle avoidance, thereby greatly reducing the workload of housework, which is an important reason it is widely accepted by the market. Despite this situation, the current level of intelligence of sweeping robots remains low, and the main shortcomings are reflected in two aspects:First, high-level perception and discriminating ability toward the environment are lacking. For example, the behavior pattern adopted in the cleaning process is usually a random walk mode, and some higher intelligence sweeping robots may support simple path planning functions, such as taking a "Z" path. However, this function is generally classified under the "blind" mode, because the robot will perform cleaning activities whether or not garbage is present in its working path. Therefore, the work efficiency is low, and the energy consumption is greatly increased. Second, the current sweeping robots generally do not have the ability to distinguish the category of garbage. If garbage can be handled according to their correct category, then it will not only facilitate the sorting of garbage but also meet environmental protection requirements. Sweeping robots are equipped with visual sensors to achieve visual perception to improve their autonomy and intelligentization. A study of the effective classification model and algorithm of garbage detection can develop the process of location and garbage recognition, and a sweeping robot can be guided to automatically recognize and deal with different types of garbage, thereby improving the purpose and efficiency of its work. Moreover, the sweeping robot can avoid being in the blind state and reduce unnecessary energy consumption. Method The proposed garbage detection and classification method based on machine vision technology aims to improve the autonomous ability of the sweeping robot. This method selects the YOLOv2 network as the main network, which has a fast detection speed in the regression method. The YOLOv2 network is combined with the dense convolutional network (DenseNet) to make full use of the high-resolution features of detection objects. The shallow and deep features of object detection can be reused and fused by embedding deep dense modules to reduce the loss of feature information. Finally, a garbage detection and classification model is built by using a multiscale strategy to train the detection network. The self-built sample dataset is used in the training and testing process. To expand the sample size, the experiment uses the image data enhancement tool ImageDataGenerator provided by Keras to perform horizontal mirror flip, random rotation, cutting, zooming, and other types of processing. The training process of the YOLOv2-dense network is as follows:First, the original image is obtained through data enhancement, and the dataset obtained by the original image and the data enhancement is manually marked. Then, YOLOv2-dense training is performed with the dataset, and the training model is obtained. To adapt large dynamic changes in the scale of the object during the movement of the mobile robot, a multiscale training strategy is adopted for network training, and the detection model is obtained. The mobile robot experiment platform uses the omnidirectional motion of the Anycbot four-wheel-drive robot chassis, the AVR motion controller, and the NVIDIA Jetson TX2 of the 256-core GPU as the visual processor. Result The detection network model is trained and tested with our self-built dataset. Test results show that the improved network structure can retain more original picture information and enhance the extraction ability of target features, and this method can effectively identify the common types of garbage, including different forms of liquid and solid, and quickly and accurately mark out the location of the garbage. Real-time detection with different angles and distances presents good results. The accuracy of YOLOv2 detection is only 82.3%, and the speed is 27 frames per second. By contrast, the accuracy of the proposed improved network YOLOv2-dense reaches 84.98%, which is 2.62% higher than that of YOLOv2, and a speed of 26 frames per second can be achieved, which satisfies real-time detection. Conclusion The experimental results show that the YOLOv2-dense network model built in this study has the speed of real-time detection. It shows strong adaptability and recognition performance when dealing with pictures with different backgrounds, illuminations, visual angles, and resolutions. For the same object, the detection effect is greatly changed at different angles and scales, and the detection performance of long-distance objects is poor. Moreover, for 3D objects, different viewing angles have a greater impact on detection. The result of the object detection is dynamically changed during the movement of the robot. The proposed method can ensure that the type of garbage is accurately identified most of the time or within a certain distance and angle of view. Moreover, the proposed method can guarantee the identification of garbage types with higher accuracy, and its overall performance is better than that of the original YOLOv2 model.
Keywords

订阅号|日报