浏览全部资源
扫码关注微信
收稿：2018-09-21，
修回：2018-12-7，
纸质出版：2019-07-16
移动端阅览
目的
2
对城市发展过程中产生的建筑固废进行处理，并将之转换为资源和能源，是极佳的保护环境的经济发展模式。然而人工分拣处理存在效率低、污染严重、对人身危害大等问题。目前工业界在探索一种有效的基于机械臂自动抓取的建筑固废自动分拣系统，其中图像分割技术是非常必要的一个环节。但是工业现场的环境因素造成固废对象的颜色严重退化，会影响最终的固废对象分割。本文针对建筑固废图像分割难度大的现状，提出一种基于多模态深度神经网络的方法来解决固废对象分割问题。
方法
2
首先，在颜色退化严重的场景下，把RGB图像和深度图一起作为深度卷积神经网络的输入，利用深度卷积神经网络进行高维特征学习，通过softmax分类器获得每个像素的标签分配概率。其次，基于新的能量函数建立全连接条件随机场，通过最小化能量函数寻找全局最优解来分割图像，从而为每一类固废对象产生一个独立的分割块。最后，利用局部轮廓信息计算深度梯度，实现同一类别的不同实例的固废对象精确分割。
结果
2
在固废图像测试集上，该方法取得了90.02%均像素精度和89.03%均交并比（MIOU）。此外，与目前一些优秀的语义分割算法相比，也表现出了优越性。
结论
2
本文方法能够对每一个固废对象同时进行有效的分割和分类，为建筑垃圾自动分拣系统提供准确的固废对象轮廓和类别信息，从而方便实现机械臂的自动抓取。
Objective
2
Construction waste is no longer useless nowadays. It has become an excellent economic development mode to protect the environment by recycling waste generated during construction and converting it into resources and energy. The current situation of construction waste in China has become increasingly severe. With the development of urbanization
old buildings have been demolished
rebuilt
and or replaced by skyscrapers. Moreover
once-inhabited areas have been gradually transformed into cities with ever expanding sizes. These cities develop quickly but entail serious hidden dangers. Construction waste generated by numerous construction work sites have become difficult to ignore. Urban construction waste refers to all kinds of construction waste generated during the construction
transformation
decoration
demolition
and laying of various buildings and structures and their auxiliary facilities. It mainly includes muck
waste concrete
waste brick
waste pipe
and waste wood. According to statistics
the amount of construction building waste in China now accounts for 30% to 40% of the total amount of municipal waste. In the next 10 years
China will produce more than 1.5 billion tons of construction waste per year on the average. By 2020
construction waste will reach 2.6 billion tons; by 2030
it will reach 7.3 billion tons. Resource utilization and recycling are inevitable options for dealing with construction waste in buildings. To deal with construction waste in buildings effectively
one can start from its characteristics. Construction waste is a mixture of various building material wastes
which are actually unutilized resources. In the 1990s
several communities in California first launched a single-stream recycling project
which referred to the mixture of all paper products
plastics
glass
metal
and other wastes. The mixture was separated into single items by a sorting system. In the sorting system
waste was mainly processed by a combination of hardware equipment and manpower. The system was not fully automated and relied mainly on human recycling; thus
it was inefficient. This attempt was meaningful because it let people understand the feasibility of recycling waste. Many construction wastes
such as waste bricks
waste rock
and scrap steel
can be recycled after being sorted
rejected
or crushed. However
systems
such as a single-stream recycling project
cannot handle substantial construction waste. Owing to the development of artificial intelligence technology
the use of intelligent robotic equipment in the field of construction waste recycling can greatly improve the capability
efficiency
and safety of recycling. Among these equipment
the robotic arm is the most widely used automated mechanical device in the industrial field. It can quickly grasp objects and work continuously. The emergence of robotic arms provides a new and efficient solution for the automatic sorting of construction waste in buildings. The use of robotic arms to sort construction waste is a revolutionary innovation for the construction waste treatment industry. The position and contour information of the object are indispensable to the robotic arm grabbing task. The application of computer image segmentation algorithms in this scene is undoubtedly suitable. Through image segmentation algorithms
a construction waste image can be accurately segmented to obtain the position and contour of each object. Combining robotic arms and image segmentation algorithms to achieve efficient construction waste recovery is worth exploring. However
segmentation of construction waste objects from construction waste images obtained via segmentation algorithms is difficult due to the characteristics of industrial sites and construction waste objects. With regard to the difficulty of object segmentation in construction waste images
this study proposes a construction waste object segmentation method based on a multimodal information deep neural network to solve the image segmentation problem and provides accurate construction waste object contour and category information for the construction waste automatic sorting system. Therefore
the system can realize automatic grabbing using a robot arm.
Method
2
First
in scenes with severe color degradation
feature learning with RGB images alone does not meet actual needs. Therefore
training the salient features with depth information is necessary. Second
we treat the RGB image and the corresponding depth image as the input of the deep convolutional neural network. The deep convolutional neural network is used to perform high-dimensional feature learning
and the feature maps obtained from the convolutional layers of the last layer are weighted and summed then fed as input data of the SoftMax classifier. Finally
we obtain the label allocation probability of each pixel. On the basis of the probability that each pixel belongs to a category
we construct a multi-label
fully connected
conditional random field. The unary energy term treats each pixel as an independent item without considering the relationship between pixels. The binary energy term represents the relationship among pixels. Thus
similar pixels are divided into the same category
and pixels with large differences between each other are assigned to different categories
which makes the segmented edges smooth. We are thus able to obtain accurate segmentation results. Therefore
according to the actual situation
we propose an energy function suitable for construction waste objects. The global optimal solution is obtained by minimizing the energy function to segment the object in the image
thereby generating an independent segmentation block for each type of construction waste object. Finally
fine segmentation of local ambiguous regions is performed according to the depth gradient information. Ambiguous areas refer to the adhesion areas between construction waste objects that are difficult to distinguish due to the degradation of visual characteristics. The depth gradient information is used to obtain the local depth edge map
from which the local ambiguity area is extracted. For the local ambiguity area
the algorithm extracts the effective internal edge to segment the adhesion objects belonging to the same class.
Result
2
In the construction waste image test set
our method achieves 90.02% mean pixel accuracy and 89.03% mean intersection over union. Compared with several excellent semantic segmentation algorithms
the experimental results show that the proposed method performs better and improves segmentation accuracy.
Conclusion
2
The algorithm proposed in this study can effectively segment and classify most construction waste objects simultaneously and can provide accurate contour and classification information of the construction waste object to a construction waste automatic sorting system so as to facilitate the automatic grasping of construction waste by robotic arms.
Hao Z G, Su X M. Research on the reuse of construction waste[J]. Architecture&Culture, 2017, (2):110-111.[
郝占国, 苏晓明.建筑垃圾再利用研究[J].建筑与文化, 2017, (2):110-111.]
Wang W X. Recycling of construction waste[J]. China Building Materials, 2005, 3(8):67-71.[
王武祥.建筑垃圾的循环利用[J].中国建材, 2005, 3(8):67-71.][DOI:10.16291/j.cnki.zgjc.2005.08.005]
Krähenbühl P, Koltun V. Efficient inference in fully connected crfs with gaussian edge potentials[C]//Advances in Neural Information Processing Systems. Granada, Spain: [s.n.], 2011: 109-117.
Reddi S S, Rudin S F, Keshavan H R. An optimal multiple threshold scheme for image segmentation[J]. IEEE Transactions on Systems, Man, and Cybernetics, 1984, SMC-14(4):661-665.[DOI:10.1109/TSMC.1984.6313341]
Ma W Y, Manjunath B S. EdgeFlow:a technique for boundary detection and image segmentation[J]. IEEE Transactions on Image Processing, 2000, 9(8):1375-1388.[DOI:10.1109/83.855433]
Leung T K, Malik J. Contour continuity in region based image segmentation[C]//Proceedings of the 5th European Conference on Computer Vision. Germany: Springer-Verlag, 1998: 544-559.
Achanta R, Shaji A, Smith K, et al. SLIC superpixels compared to state-of-the-art superpixel methods[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2012, 34(11):2274-2282.[DOI:10.1109/TPAMI.2012.120]
Felzenszwalb P F, Huttenlocher D P. Efficient graph-based image segmentation[J]. International Journal of Computer Vision, 2004, 59(2):167-181.[DOI:10.1023/B:VISI.0000022288.19776.77]
Hruschka H, Natter M. Comparing performance of feedforward neural nets K-means for cluster-based market segmentation[J]. European Journal of Operational Research, 1999, 114(2):346-353.[DOI:10.1016/s0377-2217(98)00170-2]
Song M J S, Civco D. Road extraction using SVM and image segmentation[J]. Photogrammetric Engineering&Remote Sensing, 2004, 70(12):1365-1371.[DOI:10.14358/PERS.70.12.1365]
Li J, Bioucas-Dias J M, Plaza A. Spectral-spatial hyperspectral image segmentation using subspace multinomial logistic regression and Markov random fields[J]. IEEE Transactions on Geoscience and Remote Sensing, 2012, 50(3):809-823.[DOI:10.1109/TGRS.2011.2162649]
Shelhamer E, Long J, Darrell T. Fully convolutional networks for semantic segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(4):640-651.[DOI:10.1109/TPAMI.2016.2572683]
Zhao H S, Shi J P, Qi X J, et al. Pyramid scene parsing network[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Hawaii, USA: IEEE, 2017: 2881-2890.
Qiu Y Q, Chen J W, Guo J S, et al. Three dimensional object segmentation based on spatial adaptive projection for solid waste[C]//CCF Chinese Conference on Computer Vision. Springer, Singapore, 2017: 453-464.[ DOI: 10.1007/978-981-10-7305-2_39 http://dx.doi.org/10.1007/978-981-10-7305-2_39 ]
Holz D, Holzer S, Rusu R B, et al. Real-time plane segmentation using RGB-D cameras[M]//Röfer T, Mayer N M, Savage J, et al. RoboCup 2011: Robot Soccer World Cup XV. Berlin, Heidelberg: Springer, 2011: 306-317.[ DOI:10.1007/978-3-642-32060-6_26 http://dx.doi.org/10.1007/978-3-642-32060-6_26 ]
Richtsfeld A, Mörwald T, Prankl J, et al. Learning ofperceptual grouping for object segmentation on RGB-D data[J]. Journal of Visual Communication and Image Representation, 2014, 25(1):64-73.[DOI:10.1016/j.jvcir.2013.04.006]
Simonyan K, Zisserman A. Very Deep Convolutional Networks for Large-Scale Image Recognition[EB/OL].2004-09-04[2019-04-22] . https://arxiv.org/pdf/1409.1556.pdf https://arxiv.org/pdf/1409.1556.pdf .
Chen L C, Papandreou G, Kokkinos I, et al. DeepLab:semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2016, 40(4):834-848.[DOI:10.1109/TPAMI.2017.2699184]
Badrinarayanan V, Kendall A, Cipolla R. SegNet:a deep convolutional encoder-decoder architecture for image segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(12):2481-2495.[DOI:10.1109/TPAMI.2016.2644615]
Chen L C, Papandreou G, Kokkinos I, et al. Semantic image segmentation with deep convolutional nets and fully connected crfs[EB/OL]. 2014-12-22[2019-04-22] . https://arxiv.org/pdf/1412.7062.pdf https://arxiv.org/pdf/1412.7062.pdf .
Hazirbas C, Ma L, Domokos C, et al. FuseNet: incorporating depth into semantic segmentation via fusion-based CNN architecture[C]//Proceedings of the 13th Asian Conference on Computer Vision. Taipei, China: Springer, Cham, 2016: 213-228.[ DOI:10.1007/978-3-319-54181-5_14 http://dx.doi.org/10.1007/978-3-319-54181-5_14 ]
相关作者
相关机构
京公网安备11010802024621
