浏览全部资源
扫码关注微信
收稿日期：2020-10-10，
修回日期：2020-12-29，
录用日期：2021-1-5，
纸质出版日期：2021-12-16
移动端阅览
目的
2
遥感图像道路提取在城市规划、交通管理、车辆导航和地图更新等领域中发挥了重要作用，但遥感图像受光照、噪声和遮挡等因素以及识别过程中大量相似的非道路目标干扰，导致提取高质量的遥感图像道路有很大难度。为此，提出一种结合上下文信息和注意力机制的U-Net型道路分割网络。
方法
2
使用Resnet-34预训练网络作为编码器实现特征提取，通过上下文信息提取模块对图像的上下文信息进行整合，确保对道路的几何拓扑结构特征的提取；使用注意力机制对跳跃连接传递的特征进行权重调整，提升网络对于道路边缘区域的分割效果。
结果
2
在公共数据集Deep Globe道路提取数据集上对模型进行测试，召回率和交并比指标分别达到0.847 2和0.691 5。与主流方法U-Net和CE-Net（context encoder network）等进行比较，实验结果表明本文方法在性能上表现良好，能有效提高道路分割的精确度。
结论
2
本文针对遥感图像道路提取中道路结构不完整和道路边缘区域不清晰问题，提出一种结合上下文信息和注意力机制的遥感道路提取模型。实验结果表明该网络在遥感图像道路提取上达到良好效果，具有较高的研究和应用价值。
Objective
2
Road extraction from remote sensing images has played an important role in city planning
traffic management
vehicle navigation
map updating and other fields nowadays
the characteristics of the road area in the remote sensing image have been affected by many factors such as lighting
noise and occlusion in the image acquisition process. A huge number of similar non-road objects
such as building areas and water areas have interfered with the road area recognition process simultaneously. The above two factors have increased the difficulty of road extraction from remote sensing images. Supervised learning-based road extraction algorithms such as support vector machines and traditional artificial neural networks have to artificially design features so as to train classification models. The recognition rate of these traditional methods has been significantly decreased when facing with the interference of the similar non-road targets and the rich information of the background in the images. Recently
a variety of deep learning techniques of convolutional neural networks have been widely used in the field of remote sensing image processing based on its efficient feature learning ability. Deep learning network has made a great progress in road extraction. It can not only obtain the overall network structure of the road
but also the clear boundaries of the road. A U-Net road segmentation network has been proposed improve the road extraction quality based on the context information and attention mechanism.
Method
2
A novel deep neural network for road extraction from remote sensing images has been proposed based on the symmetrical structure of U-Net network and attention mechanism. In the network structure
the introduction of the pre-trained Resnet-34 network can effectively extract image features at different granularities. Resnet-34 residual network has been used as the backbone network of the novel U-Net network. Residual learning can greatly reduce the training time of the deep network
avoid the phenomenon of gradient disappearance
and improve the training accuracy. Meanwhile
the context information has contained the interaction information between different objects
the interaction information between the object and the scene
which can be used as features to combine the various parts between the roads and distinguish the road and the background. The context information extraction module can integrate the context information to ensure that the geometric topology of the road is extracted in the image. To adjust the feature weights
the attention mechanism module can be transmitted by the skip connection
strengthened by the feature information of the road area
suppressed by the feature information of the non-road area improved by the segmentation effect of the road edge thereby effectively improving the accuracy of road segmentation. The improved model has resolved the incomplete and disconnected road structure to a certain extent by adding the context information extraction module. Furthermore
the decoder combined attention mechanism has been used to adjust the feature weights of skip connections to improve the segmentation effect of the road edge area. Combining attention mechanism and context information extraction module can effectively use global and local remote sensing image information to improve the road extraction performance.
Result
2
The model on the Deep Globe 2018 road extraction challenge dataset has been tested to evaluate the performance of the proposed model quantitatively. The Deep Global satellite road extraction dataset has contained 6 226 pairs of RGB satellite remote sensing images and labeled with dimension of 1 024×1 024 pixels. The dataset has been divided into 5 500 training set and 726 test set in the experiments. In order to evaluate the performance of the road segmentation model
two semantic segmentation performance indices have been commonly used in remote sensing image road segmentation: recall rate (recall) and intersection over union (IOU). The comprehensive experiments have shown that the recall rate and intersection over union of the proposed algorithm for the Deep Globe 2018 road extraction challenge dataset reached 0.847 2 and 0.691 5
respectively. The proposed model can segment a continuous road network. At the same time
the missing location information has been effectively restored to make the edges of the road clearer. The proposed algorithm can improve the use of remote sensing image information by adding context information module and attention mechanism. Compared with U-Net
context encoder network(CE-Net) and other models
it has higher accuracy and robustness.
Conclusion
2
A road extraction model for remote sensing images combined context information and attention mechanism has been proposed. The novel model has benefited from its pre-trained Resnet-34 backbone network and utilization of context information. Utilization of context information has solved incomplete and disconnected road structure to a certain extent. The decoder of the attention mechanism has improved the segmentation effect of the road edge area. The experimental results have demonstrated that the network achieved good results of road extraction from remote sensing images. The proposed method has improved the road segmentation accuracy and displayed the potential in remote sensing image processing.
An R, Feng X Z and Wang H L. 2003. Road feature extraction form remote sensing classified imagery based on mathematical morphology and analysis of road networks. Journal of Image and Graphics, 8(7): 798-804
安如, 冯学智, 王慧麟. 2003. 基于数学形态学的道路遥感影像特征提取及网络分析. 中国图象图形学报, 8(7): 798-804 [DOI:10.3969/j.issn.1006-8961.2003.07.016]
Anil P N and Natarajan S. 2010. A novel approach using active contour model for semi-automatic road extraction from high resolution satellite imagery//Proceedings of the 2nd International Conference on Machine Learning and Computing. Bangalore, India: IEEE: 263-266[ DOI: 10.1109/icmlc.2010.36 http://dx.doi.org/10.1109/icmlc.2010.36 ]
Buslaev A, Seferbekov S, Iglovikov V and Shvets A. 2018. Fully convolutional network for automatic road extraction from satellite imagery//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). Salt Lake City, USA: IEEE: 197-1973[ DOI: 10.1109/cvprw.2018.00035 http://dx.doi.org/10.1109/cvprw.2018.00035 ]
Demir I, Koperski K, Lindenbaum D, Pang G, Huang J, Basu S, Hughes F, Tuia D and Raska R. 2018. DeepGlobe 2018: a challenge to parse the earth through satellite images//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). Salt Lake City, USA: IEEE: 172-181[ DOI: 10.1109/cvprw.2018.00031 http://dx.doi.org/10.1109/cvprw.2018.00031 ]
Deng J, Dong W, Socher R, Li L J, Li K and Li F F. 2009. Imagenet: a large-scale hierarchical image database//Proceedings of 2009 IEEE Conference on Computer Vision and Pattern Recognition. Miami, USA: IEEE: 248-255[ DOI: 10.1109/CVPR.2009.5206848 http://dx.doi.org/10.1109/CVPR.2009.5206848 ]
Gu Z W, Cheng J, Fu H Z, Zhou K, Hao H Y, Zhao Y T, Zhang T Y, Gao S H and Liu J. 2019. CE-Net: context encoder network for 2D medical image segmentation. IEEE Transactions on Medical Imaging, 38(10): 2281-2292[DOI:10.1109/tmi.2019.2903562]
He K M, Zhang X Y, Ren S Q and Sun J. 2016. Deep residual learning for image recognition//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas, USA: IEEE: 770-778[ DOI: 10.1109/cvpr.2016.90 http://dx.doi.org/10.1109/cvpr.2016.90 ]
Iglovikov V and Shvets A. 2018. Ternausnet: U-net with VGG11 encoder pre-trained on imagenet for image segmentation[EB/OL]. [2020-10-10] . https://arxiv.org/pdf/1801.05746.pdf https://arxiv.org/pdf/1801.05746.pdf
Kirthika A and Mookambiga A. 2011. Automated road network extraction using artificial neural network//Proceedings of 2011 International Conference on Recent Trends in Information Technology (ICRTIT). Chennai, India: IEEE: 1061-1065[ DOI: 10.1109/icrtit.2011.5972323 http://dx.doi.org/10.1109/icrtit.2011.5972323 ]
Ma R G, Wang WX and Liu S. 2012. Extracting roads based on Retinex and improved Canny operator with shape criteria in vague and unevenly illuminated aerial images. Journal of Applied Remote Sensing, 6(1): #063610[DOI:10.1117/1.jrs.6.063610]
Mnih V and Hinton G E. 2010. Learning to detect roads in high-resolution aerial images//Proceedings of Computer Vision-ECCV 2010. Berlin, Germany: Springer: 210-223[ DOI: 10.1007/978-3-642-15567-3_16 http://dx.doi.org/10.1007/978-3-642-15567-3_16 ]
Munteanu A, Selea T and Neagul M. 2019. Deep learning techniques applied for road segmentation//The 21st International Symposium on Symbolic and Numeric Algorithms for Scientific Computing (SYNASC). Timisoara, Romania: IEEE: 297-303[ DOI: 10.1109/SYNASC49474.2019.00049 http://dx.doi.org/10.1109/SYNASC49474.2019.00049 ]
Oktay O, Schlemper J, Folgoc L L, Lee M, Lee M, Heinrich M, Misawa K, Mori K, McDonagh S, Hammerla N Y, Kainz B, Glocker B and Rueckert D. 2018. Attention U-Net: learning where to look for the pancreas[EB/OL]. [2020-09-10] . https://arxiv.org/pdf/1804.03999.pdf https://arxiv.org/pdf/1804.03999.pdf
Oquab M, Bottou L, Laptev I and Sivic J. 2014. Learning and transferring mid-level image representations using convolutional neural networks//Proceedings of 2014 IEEE Conference on Computer Vision and Pattern Recognition. Columbus, USA: IEEE: 1717-1724[ DOI: 10.1109/cvpr.2014.222 http://dx.doi.org/10.1109/cvpr.2014.222 ]
Ronneberger O, Fischer P and Brox T. 2015. U-Net: convolutional networks for biomedical image segmentation//Proceedings of the 18th International Conference on Medical Image Computing and Computer-Assisted Intervention. Munich, Germany: Springer: 234-241[ DOI: 10.1007/978-3-319-24574-4_28 http://dx.doi.org/10.1007/978-3-319-24574-4_28 ]
Saito S, Yamashita T and Aoki Y. 2016. Multiple object extraction from aerial imagery with convolutional neural networks. Journal of Imaging Science and Technology, 60(1): #10402[DOI:10.2352/J.ImagingSci.Technol.2016.60.1.010402]
Simler C. 2011. An improved road and building detector on VHR images//Proceedings of 2011 IEEE International Geoscience and Remote Sensing Symposium. Vancouver, Canada: IEEE: 507-510[ DOI: 10.1109/igarss.2011.6049176 http://dx.doi.org/10.1109/igarss.2011.6049176 ]
Szegedy C, Ioffe S, Vanhoucke V and Alemi A. 2016. Inception-v4, inception-resnet and the impact of residual connections on learning[EB/OL]. [2020-10-10] . https://arxiv.org/pdf/1602.07261.pdf https://arxiv.org/pdf/1602.07261.pdf
Wang J H, Qin Q M, Gao Z L, Ye X and Meng J J. 2016. Road extraction from high-resolution remote sensing imagery by including spatial texture feature. Journal of Hunan University (Natural Sciences), 43(4): 153-156
王建华, 秦其明, 高中灵, 叶昕, 孟晋杰. 2016. 加入空间纹理信息的遥感图像道路提取. 湖南大学学报(自然科学版), 43(4): 153-156 [DOI:10.3969/j.issn.1674-2974.2016.04.021]
Wang W X, Yang N, Zhang Y, Wang F P, Cao T and Eklund P. 2016. A review of road extraction from remote sensing images. Journal of Traffic and Transportation Engineering (English Edition), 3(3): 271-282[DOI:10.1016/j.jtte.2016.05.005]
Wu L and Hu Y A. 2010. A survey of automatic road extraction from remote sensing images. Acta Automatica Sinica, 36(7): 912-922
吴亮, 胡云安. 2010. 遥感图像自动道路提取方法综述. 自动化学报, 36(7): 912-922 [DOI:10.3724/SP.J.1004.2010.00912]
Xu Y Y, Feng Y X, Xie Z, Hu A N and Zhang X M. 2018. A research on extracting road network from high resolution remote sensing imagery//Proceedings of the 26th International Conference on Geoinformatics. Kunming, China: IEEE: 1-4[ DOI: 10.1109/geoinformatics.2018.8557042 http://dx.doi.org/10.1109/geoinformatics.2018.8557042 ]
Yang X F, Li X T, Ye Y M, Zhang X F, Zhang H J, Huang X H and Zhang B. 2019. Road detection via deep residual dense U-Net//Proceedings of 2019 International Joint Conference on Neural Networks (IJCNN). Budapest, Hungary: IEEE: 1-7[ DOI: 10.1109/ijcnn.2019.8851728 http://dx.doi.org/10.1109/ijcnn.2019.8851728 ]
Yu F and Koltun V. 2016. Multi-scale context aggregation by dilated convolutions[EB/OL]. [2020-09-10] . https://arxiv.org/pdf/1511.07122.pdf https://arxiv.org/pdf/1511.07122.pdf
Zhang Y H, He J, Kan X, Xia G H, Zhu L L and Ge T T. 2018. Summary of road extraction methods for remote sensing images. Computer Engineering and Applications, 54(13): 1-10, 51
张永宏, 何静, 阚希, 夏广浩, 朱灵龙, 葛涛涛. 2018. 遥感图像道路提取方法综述. 计算机工程与应用, 54(13): 1-10, 51 [DOI:10.3778/j.issn.1002-8331.1804-0271]
Zhang Z X, Liu Q J and Wang Y H. 2018. Road extraction by deep residual U-Net. IEEE Geoscience and Remote Sensing Letters, 15(5): 749-753[DOI:10.1109/lgrs.2018.2802944]
相关作者
相关机构
京公网安备11010802024621