浏览全部资源
扫码关注微信
Received：15 December 2020，
Revised：2021-3-1，
Accepted：08 March 2021，
Published：16 November 2021
移动端阅览
目的
2
螺栓是确保输电线路安全牢靠的基石，螺栓缺销、松动和锈蚀等缺陷是造成输电线路损坏甚至重大事故的重要原因之一。本文针对螺栓缺陷存在视觉不可分的问题，提出了一种改进NTS-Net（navigator-teacher-scrutinizer network）的螺栓属性多标签分类方法。
方法
2
为了增强模型对不规则轮廓的处理能力，采用可变形卷积的ResNet-50网络作为特征提取网络从原图中提取全局特征。利用NTS-Net学习得到图像信息量最大的判别性局部区域。为了考虑不同局部特征对不同属性标签的不同影响，在局部特征与全局特征融合时引入通道注意力机制，提取特征的通道权重，获取关键通道特征以改善多标签分类效果。
结果
2
实验结果表明，本文在螺栓多属性分类数据集上的平均分类精确率为84.5%，比采用传统的多标签分类精确率提升了10%~20%。
结论
2
本文通过可变形卷积提升网络的特征提取能力以及引入通道注意力机制实现了对NTS-Net提供的局部特征的高效利用，为解决螺栓多属性分类中存在的问题提供了一种新的思路。
Objective
2
Unmanned aerial vehicle(UAV)-based transmission line inspection technology has achieved long-term progress and development. The use of computer vision technology to automatically and accurately locate line equipment
such as wires
insulators
and bolts
from aerial inspection images under complex natural backgrounds and accurately detect their defects has become animportant technical issue. The defects inspected by transmission lines mainly include tower
wire
insulator
and metal fitting defects. Given the large size of metal fittings and insulators
their defects are obvious and easy to identify. By contrast
numerous bolts are present in poles
insulators
and metal fittings. Bolts change easily from the normal state to the defect state due to the large number of bolts and complex stress conditions. The use of deep learning has achieved good results in visual detection
identification
and classification of tower
wire
insulator
and metal fitting defects
but only a few studies have been conducted on bolt defects. In addition
bolt defects are not completely visually separable problems; they are visually inseparable
and they can not be solved by object detection algorithms alone. Thus
we believe that the bolt defect detection problem is not only an object detection problem
but also an image classification problem. Multi-label classification of bolts must be implemented efficiently and quickly to provide a basis for defect detection. The convolutional neural network (CNN) is inherently limited by model geometry transformation due to its fixed geometric structure. An offset variable must be added to the position of each sampling point in the convolution kernel to weaken this limitation and improve the feature extraction capability of bolts. By adding these variables
the convolution kernel is given random sampling near the current position and is no longer limited to the previous regular grid points. The convolution operation after expansion is called deformable convolution. Deformable convolution changes the sampling position of the standard convolution kernel by adding an additional offset to the sampling point. The compensation obtained can be learned through training without additional supervision.
Method
2
The object to be inspected in the transmission line bolt multi-label classification task has similar overall characteristics as those of the object in the general image multi-label classification task. The classification model needs to capture the key local features that can distinguish the attributes of different categories. The idea of using local regions to assist in classification belongs to fine-grained classification. Several studies on fine-grained classification algorithms used detailed local area labels to train the model so that the model can accurately locate the regions containing detailed semantic information. However
this approach requires a huge amount of work in the production of labels. In other studies
unsupervised learning was used to locate key areas. Although this strategy eliminates tedious label-making work
the accuracy of the model in locating key details can not be ensured. The multi-label classification method proposed in this study is mainly divided into three steps.First
navigator-teacher-scrutinizer network(NTS-Net) is used as the basic network
and the feature extraction network is improved into a deformable ResNet-50 network in accordance with the various properties of the bolt target shape. Second
the navigator network in NTS-Net continuously learns and provides
k
regions with the most information under the guidance of the teacher network to obtain the discriminative region of the bolt target. Lastly
to make the model use discriminant features effectively
the input features of the
k
regions receiving the most information from the navigator network are extracted
and corresponding feature vectors are generated and connected to the feature vectors of the entire input image. Afterward
the features need to be passed through the channel attention module
which can enhance the feature with a large weight and suppress the feature with a small weight.
Result
2
This study uses the bolt multi-attribute classification dataset to evaluate the model. The bolt defect images are from samples obtained by UAV line inspection. The data sample has a total of 2 000 pictures
of which 1 500 are used as training samples and 500 are used as test samples. The bolt defect attributes are divided into six categories based the idea of visual separability. Each bolt defect image contains one or more defect attributes
which can be divided into the following six categories: a pin hole is present
shim is present
a nut is present
rust is present
the nut is loose
and the pin is loose; they are labeled 0-5 respectively. In the multi-label classification task in this study
a 1×6 matrix is constructed for each picture as the label of the picture. If the corresponding attribute category exists
the value is set to 1 and vice versa. Experimental results show that the mean average precision of the proposed method in the bolt multi-attribute classification dataset is 84.5%
which is 10%~20% higher than the accuracy of multi-label classification using traditional networks.
Conclusion
2
The feature extraction capability of the network is improved through deformable convolution
and the channel attention mechanism is introduced to realize the efficient utilization of the local features provided by NTS-Net. Experimental results show that the proposed method performs better than the traditional method in the bolt multi-attribute classification dataset. The proposed method provides a new idea for applying multi-attribute information to bolt defect reasoning and realizing bolt defect detection.
Chen J W, Liu Z G, Wang H R, Núñez A and Han Z W. 2018. Automatic defect detection of fasteners on the catenary support device using deep convolutional neural network. IEEE Transactions on Instrumentation and Measurement, 67(2): 257-269[DOI: 10.1109/tim.2017.2775345]
Cheng M M, Zhang Z M, Lin W Y and Torr P. 2014. BING: binarized normed gradients for objectness estimation at 300 fps//Proceedings of 2014 IEEE Conference on Computer Vision and Pattern Recognition. Columbus, USA: IEEE: 3286-3293[ DOI: 10.1109/CVPR.2014.414 http://dx.doi.org/10.1109/CVPR.2014.414 ]
Dai J F, Qi H Z, Xiong Y W, Li Y, Zhang G D, Hu H and Wei Y C. 2017. Deformable convolutional networks//Proceedings of 2017 IEEE International Conference on Computer Vision (ICCV). Venice, Italy: IEEE: 764-773[ DOI: 10.1109/ICCV.2017.89 http://dx.doi.org/10.1109/ICCV.2017.89 ]
Deng J, Dong W, Socher R, Li L J, Li K and Li F F. 2009. ImageNet: a large-scale hierarchical image database//Proceedings of 2009 IEEE Conference on Computer Vision and Pattern Recognition. Miami, USA: IEEE: 248-255[ DOI: 10.1109/CVPR.2009.5206848 http://dx.doi.org/10.1109/CVPR.2009.5206848 ]
Fu J L, Zheng H L and Mei T. 2017. Look closer to see better: recurrent attention convolutional neural network for fine-grained image recognition//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, USA: IEEE: 4476-4484[ DOI: 10.1109/CVPR.2017.476 http://dx.doi.org/10.1109/CVPR.2017.476 ]
He K M, Zhang X Y, Ren S Q and Sun J. 2016. Deep residual learning for image recognition//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas, USA: IEEE: 770-778[ DOI: 10.1109/CVPR.2016.90 http://dx.doi.org/10.1109/CVPR.2016.90 ]
Hu J, Shen L, Albanie S, Sun G and Wu E H. 2017. Squeeze-and-Excitation networks[EB/OL]. [2020-10-11] . https://arxiv.org/pdf/1709.01507.pdf https://arxiv.org/pdf/1709.01507.pdf
Krizhevsky A, Sutskever I and Hinton G E. 2012. ImageNet classification with deep convolutional neural networks//Proceedings of the 25th International Conference on Neural Information Processing Systems. Lake Tahoe, USA: Curran Associates Inc. : 1097-1105[ DOI: 10.5555/2999134.2999257 http://dx.doi.org/10.5555/2999134.2999257 ]
Park J, Woo S, Lee J Y and Kweon I S. 2018. BAM: Bottleneck attention module[EB/OL]. [2020-10-11] . https://arxiv.org/pdf/1807.06514.pdf https://arxiv.org/pdf/1807.06514.pdf
Razavian A S, Azizpour H, Sullivan J and Carlsson S. 2014. CNN features off-the-shelf: an astounding baseline for recognition//Proceedings of 2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops. Columbus, USA: IEEE: 512-519[ DOI: 10.1109/cvprw.2014.131 http://dx.doi.org/10.1109/cvprw.2014.131 ]
Selvaraju R R, Cogswell M, Das A, Vedantam R, Parikh D and Batra D. 2017. Grad-CAM: visual explanationsfrom deep networks via gradient-based localization//Proceedings of 2017 IEEE International Conference on Computer Vision (ICCV). Venice, Italy: IEEE: 618-626[ DOI: 10.1109/ICCV.2017.74 http://dx.doi.org/10.1109/ICCV.2017.74 ]
Shih K J, Mallya A, Singh S and Hoiem D. 2015. Part localization using multi-proposal consensus for fine-grained categorization//Proceedings of the British Machine Vision Conference (BMVC). Guildford, UK: BMVA Press: #128[ DOI: 10.5244/c.29.128 http://dx.doi.org/10.5244/c.29.128 ]
Wang J, Yang Y, Mao J H, Huang Z H, Huang C and Xu W. 2016. CNN-RNN: a unified framework for multi-label image classification//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, USA: IEEE: 2285-2294[ DOI: 10.1109/CVPR.2016.251 http://dx.doi.org/10.1109/CVPR.2016.251 ]
Wei Y C, Xia W, Lin M, Huang J S, Ni B B, Dong J, Zhao Y and Yan S C. 2016. HCP: a flexible CNN framework for multi-label image classification. IEEE Transactions on Pattern Analysis and Machine Intelligence, 38(9): 1901-1907[DOI: 10.1109/TPAMI.2015.2491929]
Woo S, Park J, Lee J Y and Kweon I S. 2018. CBAM: Convolutional block attention module//Proceedings of the 15th European Conference on Computer Vision. Munich, Germany: Springer: 3-19[ DOI: 10.1007/978-3-030-01234-2_1 http://dx.doi.org/10.1007/978-3-030-01234-2_1 ]
Xie S N, Girshick R, Dollár P, Tu Z W and He K M. 2017. Aggregated residual transformations for deep neural networks//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, USA: IEEE: 5987-5995[ DOI: 10.1109/CVPR.2017.634 http://dx.doi.org/10.1109/CVPR.2017.634 ]
Xue Y, Wu H D, Zhang N, Yu Z C, Ye X K and Hua X. 2020. Detection of insulation piercing connectors and bolts on the transmission line using improved faster R-CNN. Laser and Optoelectronics Progress, 57(8): #081008
薛阳, 吴海东, 张宁, 俞志程, 叶晓康, 华茜. 2020. 基于改进Faster R-CNN输电线穿刺线夹及螺栓的检测. 激光与光电子学进展, 57(8): #081008 [DOI: 10.3788/LOP57.081008]
Yang Z, Luo T G, Wang D, Hu Z Q, GaoJ and Wang L W. 2018. Learning to navigate for fine-grained classification[EB/OL]. [2020-10-11] . https://arxiv.org/pdf/1809.00287v1.pdf https://arxiv.org/pdf/1809.00287v1.pdf
Zagoruyko S and Komodakis N. 2016. Wide residual networks//Proceedings of British Machine Vision Conference 2016. Guildford, UK: BMVA Press: 87. [ DOI: 10.5244/C.30.87 http://dx.doi.org/10.5244/C.30.87 ]
Zhang K, Guo L R, Gao C and Zhao Z B. 2019. Pyramidal RoR for image classification. Cluster Computing, 22(S2): 5115-5125[DOI: 10.1007/s10586-017-1443-x]
Zhang K, Sun M, Han T X, Yuan X F, Guo L R and Liu T. 2018. Residual networks of residual networks: multilevel residual networks. IEEE Transactions on Circuits and Systems for Video Technology, 28(6): 1303-1314[DOI: 10.1109/TCSVT.2017.2654543]
Zhao Z B, Qi H Y and Nie L Q. 2019. Research overview on visual detection of transmission lines based on deep learning. Guangdong Electric Power, 32(9): 11-23
赵振兵, 齐鸿雨, 聂礼强. 2019. 基于深度学习的输电线路视觉检测研究综述. 广东电力, 32(9): 11-23 [DOI: 10.3969/j.issn.1007-290X.2019.009.002]
Zhao Z B, Qi H Y, Qi Y C, Zhang K, Zhai Y J and Zhao W Q. 2020. Detection method based on automatic visual shape clustering for pin-missing defect in transmission lines. IEEE Transactions on Instrumentation and Measurement, 69(9): 6080-6091[DOI: 10.1109/tim.2020.2969057]
Zhong J P, Liu Z G, Han Z W, Han Y and Zhang W X. 2019. A CNN-based defect inspection method for catenary split pins in high-speed railway. IEEE Transactions on Instrumentation and Measurement, 68(8): 2849-2860[DOI: 10.1109/tim.2018.2871353]
相关作者
相关机构
京公网安备11010802024621
