Image classification method of transmission line bolt defects using the optimal knowledge transfer wide residual network

Yincheng Qi; Chaoxiong Jin; Zhenbing Zhao; Jietao Ding; Bin Lyu

doi:10.11834/jig.200839

Frontier Technology of Power Computer Vision | Views : 0 下载量: 336 CSCD: 0

PDF
Export
Share
Collection
Album

Image classification method of transmission line bolt defects using the optimal knowledge transfer wide residual network
Vol. 26, Issue 11, Pages: 2571-2581(2021)
Received：29 December 2020，

Revised：2021-4-13，

Accepted：20 April 2021，

Published：16 November 2021
DOI： 10.11834/jig.200839
稿件说明：

移动端阅览

DOI：

Yincheng Qi, Chaoxiong Jin, Zhenbing Zhao, Jietao Ding, Bin Lyu. Image classification method of transmission line bolt defects using the optimal knowledge transfer wide residual network[J]. Journal of Image and Graphics, 2021, 26(11): 2571-2581. DOI： 10.11834/jig.200839.

摘要

目的

输电线路螺栓图像具有分辨率低和视觉信息较差的特点，针对螺栓缺陷图像分类时教师网络（大模型）参数量大、学生网络（小模型）分类精度低的问题，提出了一种最优知识传递宽残差网络输电线路螺栓缺陷图像分类方法，来弥补使用大小模型进行分类任务的局限性。

方法

首先改变大模型宽度，即拓宽网络特征表达维度来增加向小模型传递的螺栓缺陷知识和简化小模型结构至3个残差块；然后为了选出传递螺栓缺陷知识性能最优的大模型，提出知识偏差的概念来可视化大模型向小模型螺栓缺陷知识传递的程度，综合分析不同宽度大模型利用知识蒸馏算法与注意力转移算法分别指导小模型训练后的分类准确率，并用知识偏差来确定最优知识传递模型；最后将最优知识传递模型利用知识蒸馏算法与注意力转移算法相结合指导小模型训练，尽可能提升小模型的螺栓缺陷分类性能。

结果

在自建螺栓缺陷图像分类数据集上进行了验证，结果表明宽度为5的大模型向小模型传递螺栓缺陷知识性能最优，使小模型的螺栓缺陷分类准确率提高了5.56%，小模型与最优大模型的分类准确率只差2.17%，知识偏差为0.28，且小模型的参数量仅为大模型参数量的0.56%。

结论

本文提出的最优知识传递宽残差网络输电线路螺栓缺陷图像分类方法，弥补了大小模型螺栓缺陷图像分类的局限性，实现了精度与资源消耗的平衡。

Abstract

Objective

Bolts play a key role in fixing and connecting various metal parts in transmission lines. Defects seriously affect the power transmission of transmission lines. The imaging background of an inspection image is complicated

the imaging distance and angle are variable

and the bolts occupy a small proportion in the inspection image. Thus

bolt defect images of transmission lines have low resolution and scarce visual information

and they usually require a large model with high complexity and excellent performance to classify bolt defects and ensure accuracy. A large model has a complex structure and numerous parameters

and deploying it on a large scale is difficult due to a large amount of computing resources needed in data analysis. A small model has a simple structure and few parameters

but it cannot completely guarantee the accuracy of bolt defect classification. This study proposes an image classification method of transmission line bolt defects based on the optimal knowledge transfer network to compensate for the limitations of bolt defect classification using large and small models.

Method

The width of the large model is changed

that is

the dimension of network feature expression is broadened

to fully mine the target information in the bolt image

thereby increasing the bolt defect knowledge of the transferability of the large model to the small model. To reduce the parameters of the small model considerably and improve its operation and maintenance capabilities

the structure of the small model is simplified to a 10-layer residual network with three residual blocks. The number of convolution kernels of each residual block is 16

and 64. Therefore

the small model still has obvious features that focus on the low gradient of the bolt image in the low layer

the high difference area in the middle layer

and the overall characteristics of the bolt image in the high layer. Then

the large models of different widths use the attention transfer algorithm and the knowledge distillation algorithm to guide the training of the small models

and the accuracy of the small models under different widths after the training is calculated. Afterward

the concept of knowledge deviation is proposed to measure the degree of bolt defect knowledge transfer of large models and select the large model with the best performance in transferring bolt defect knowledge. The performance of the large and small models is mapped on a number line in the form of accuracy. The calculation process of knowledge deviation proceeds as follows. First

the difference in bolt defect classification accuracy between the large model with a known width and the small model being instructed is calculated. Second

the difference in bolt defect classification accuracy between the large model and the small model that is not being instructed is computed. Lastly

the ratio of the two differences before and after the calculation is adopted as the knowledge deviation. The smaller the knowledge deviation is

the greater the degree of bolt defect knowledge transfer is from the large model to the small model. The optimal knowledge transfer model is determined according to the knowledge deviation of different widths and the bolt defect classification accuracy of the small models under different guidance methods. The optimal knowledge transfer model combines the attention algorithm and the knowledge distillation algorithm to guide in small model training and maximize the bolt defect classification performance of the small model.

Result

The self-built bolt defect image classification data set verifies the effectiveness of this method in improving the accuracy of bolt classification of the simplified small model. The data set is constructed by clipping and optimizing the transmission line inspection image. The bolt defect image classification data set contains a total of 6 420 images

including 3 136 normal bolts

2 820 bolts with missing pins

and 464 bolts with missing nuts

which belong to three categories. Experimental results show that the large model with a width of 5 has the best performance in transferring bolt defect knowledge to the small model

and it increases the bolt defect classification accuracy of the small model by 5.56%. The difference in the accuracy of bolt defect classification between the small model and the optimal knowledge transfer model is only 2.17%. The knowledge deviation is 0.28

and the parameter of the small model is only 0.56% of the parameter of the large model.

Conclusion

The proposed bolt defect classification method based on the optimal knowledge transfer network greatly alleviates the problems of large model parameters and low classification accuracy of small models caused by bolt image characteristics. Balance is achieved between the classification accuracy of the bolt defect image and resource consumption. The method meets the requirements of actual field operation and maintenance

such as the work requirements of embedded equipment (e.g.

online monitoring in transmission lines) and reduces the resource consumption of transmission line patrol data analysis.

关键词

Keywords

references

Cho J H and Hariharan B. 2019. On the efficacy of knowledge distillation//Proceedings of 2019 IEEE/CVF International Conference on Computer Vision (ICCV). Seoul, Korea (South): IEEE: 4794-4802[ DOI: 10.1109/ICCV.2019.00489 http://dx.doi.org/10.1109/ICCV.2019.00489 ]

Chung I, Park S, Kim J and Kwak N. 2020. Feature-map-level online adversarial knowledge distillation[EB/OL]. [2021-02-01] . https://arxiv.org/pdf/2002.01775.pdf https://arxiv.org/pdf/2002.01775.pdf

Dai B, Hou Z Q, Yu W S, Hu D and Fan S Y. 2016. Robust visual tracking viafast deep learning. Journal of Image and Graphics, 21(12): 1662-1670

戴铂, 侯志强, 余旺盛, 胡丹, 范舜奕. 2016. 快速深度学习的鲁棒视觉跟踪. 中国图象图形学报, 21(12): 1662-1670 [DOI: 10.11834/jig.20161211]

Fukuda T, Suzuki M, Kurata G, Thomas S, Cui J and Ramabhadran B. 2017. Efficient knowledge distillation from an ensemble of teachers//Proceedings of Interspeech 2017. Stockholm, Sweden: Interspeech: 3697-3701[ DOI: 10.21437/Interspeech.2017-614 http://dx.doi.org/10.21437/Interspeech.2017-614 ]

Geng Z M, Yu M Q, Liu X B and Lyu C. 2020. Combining attention mechanism and knowledge distillation for Siamese network compression. Journal of Image and Graphics, 25(12): 2563-2577

耿增民, 余梦巧, 刘峡壁, 吕超. 2020. 融合注意力机制与知识蒸馏的孪生网络压缩. 中国图象图形学报, 25(12): 2563-2577 [DOI: 10.11834/jig.200051]

Gu C Y, Li Z, Shi J T, Zhao H H, Jiang Y and Jiang X C. 2020. Detection for pin defects of overhead lines by UAV patrol image based on improved Faster-RCNN. High Voltage Engineering, 46(9): 3089-3096

顾超越, 李喆, 史晋涛, 赵航航, 江一, 江秀臣. 2020. 基于改进Faster-RCNN的无人机巡检架空线路销钉缺陷检测. 高电压技术, 46(9): 3089-3096 [DOI: 10.13336/j.1003-6520.hve.20190748]

Heo B, Lee M, Yun S and Choi J Y. 2019. Knowledge distillation with adversarial samples supporting decision boundary[EB/OL] . [2021-02-01]. https://arxiv.org/pdf/1805.05532.pdf https://arxiv.org/pdf/1805.05532.pdf

Hershey J R and Olsen P A. 2007. Approximating the Kullback Leibler divergence between Gaussian mixture models//Proceedings of 2007 IEEE International Conference on Acoustics, Speech, and Signal Processing-ICASSP'07. Honolulu, USA: IEEE: IV-317-IV-320[ DOI: 10.1109/ICASSP.2007.366913 http://dx.doi.org/10.1109/ICASSP.2007.366913 ]

Hinton G, Vinyals O and Dean J. 2015. Distilling the knowledge in a neural network[EB/OL]. [2021-02-01] . https://arxiv.org/pdf/1503.02531.pdf https://arxiv.org/pdf/1503.02531.pdf

Jiang X C, Liu Y D, Fu X F, Xu P, Wang S J and Sheng G H. 2019. Construction ideas and development trends of transmission and distribution equipment of the ubiquitous power internet of things. High Voltage Engineering, 45(5): 1345-1351

江秀臣, 刘亚东, 傅晓飞, 徐鹏, 王劭菁, 盛戈皞. 2019. 输配电设备泛在电力物联网建设思路与发展趋势. 高电压技术, 45(5): 1345-1351 [DOI: 10.13336/j.1003-6520.hve.20190505001]

Li C H and Lee C K. 1993. Minimum cross entropy thresholding. Pattern, recognition, 26(4): 617-625[DOI: 10.1016/0031-3203(93)90115-D]

Li X F, Liu H Y, Liu G H and Su H S. 2020. Transmission line pin defect detection based on deep learning[J/OL]. Power System Technology, [2020-11-04] . https://doi.org/10.13335/j.1000-3673.pst.2020.1028 https://doi.org/10.13335/j.1000-3673.pst.2020.1028

李雪峰, 刘海莹, 刘高华, 苏寒松. 2020. 基于深度学习的输电线路销钉缺陷检测[J/OL]. 电网技术, [2020-11-04] . https://doi.org/10.13335/j.1000-3673.pst.2020.1028 https://doi.org/10.13335/j.1000-3673.pst.2020.1028 [ DOI: 10.13335/j.1000-3673.pst.2020.1028 http://dx.doi.org/10.13335/j.1000-3673.pst.2020.1028 ].

Ma D A, Tang P, Zhao L J and Zhang Z. 2021. Review of data augmentation for image in deep learning. Journal of Image and Graphics, 26(3): 487-502

马岽奡, 唐娉, 赵理君, 张正. 2021. 深度学习图像数据增广方法研究综述. 中国图象图形学报, 26(3): 487-502 [DOI: 10.11834/jig.200089]

Park W, Kim D, Lu Y and Cho M. 2019. Relational knowledge distillation//Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Long Beach, USA: IEEE: 3962-3971[ DOI: 10.1109/CVPR.2019.00409 http://dx.doi.org/10.1109/CVPR.2019.00409 ]

Peng X Y, Chen C, Rao Z Q, Yang B S, Mai X M and Wang K. 2015. Safety inspection and intelligent diagnosis of transmission line based on unmanned helicopter of multi sensor data acquisition. High Voltage Engineering, 41(1): 159-166

彭向阳, 陈驰, 饶章权, 杨必胜, 麦晓明, 王珂. 2015. 基于无人机多传感器数据采集的电力线路安全巡检及智能诊断. 高电压技术, 41(1): 159-166 [DOI: 10.13336/j.1003-6520.hve.2015.01.022]

Romero A, Ballas N, Kahou S E, Chassang A, Gatta C and Bengio Y. 2015. Fitnets: hints for thin deep nets[EB/OL]. [2021-02-01] . https://arxiv.org/pdf/1412.6550.pdf https://arxiv.org/pdf/1412.6550.pdf

Tong W G, Yuan J S and Li B S. 2010. Application of image processing in patrol inspection of overhead transmission line by helicopter. Power System Technology, 34(12): 204-208

仝卫国, 苑津莎, 李宝树. 2010. 图像处理技术在直升机巡检输电线路中的应用综述. 电网技术, 34(12): 204-208 [DOI: 10.13335/j.1000-3673.pst.2010.12.032]

Wang K, Wang J, Liu G, Zhou W Q and Chen J. 2019. Defect detection of pins based on RetinaNet and class balanced sampling methods. Jiangsu Electrical Engineering, 38(4): 80-85

王凯, 王健, 刘刚, 周文青, 陈佳. 2019. 基于RetinaNet和类别平衡采样方法的销钉缺陷检测. 电力工程技术, 38(4): 80-85 [DOI: 10.12158/j.2096-3203.2019.04.012]

Wang W H, Wei F R, Dong L, Bao H B, Yang N and Zhou M. 2020. MiniLM: deep self-attention distillation for task-agnostic compression of pre-trained transformers[EB/OL]. [2021-02-01] . https://arxiv.org/pdf/2002.10957v2.pdf https://arxiv.org/pdf/2002.10957v2.pdf

Xue Y, Wu H D, Zhang N, Yu Z C, Ye X K and Hua X. 2020. Detection of insulation piercing connectors and bolts on the transmission line using improved faster R-CNN. Laser and Optoelectronics Progress, 57(8): #081008

薛阳, 吴海东, 张宁, 俞志程, 叶晓康, 华茜. 2020. 基于改进Faster R-CNN输电线穿刺线夹及螺栓的检测. 激光与光电子学进展, 57(8): #081008 [DOI: 10.3788/LOP57.081008]

Ye H J, Lu S and Zhan D C. 2020. Distilling cross-task knowledge via relationship matching//Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Seattle, USA: IEEE: 12393-12402[ DOI: 10.1109/CVPR42600.2020.01241 http://dx.doi.org/10.1109/CVPR42600.2020.01241 ]

Zagoruyko S and Komodakis N. 2017. Paying more attention to attention: improving the performance of convolutional neural networks via attention transfer[EB/OL]. [2021-02-01] https://arxiv.org/pdf/1612.03928.pdf https://arxiv.org/pdf/1612.03928.pdf

Zagoruyko S and Komodakis N. 2016. Wide residual networks//Proceedings of the British Machine Vision Conference (BMVC). Bristol, UK: BMVA Press: 87.1-87.12[ DOI: 10.5244/C.30.87 http://dx.doi.org/10.5244/C.30.87 ]

Zhao Y Q, Rao Y, Dong S P and Zhang J Y. 2020. Survey on deep learning object detection. Journal of Image and Graphics, 25(4): 629-654

赵永强, 饶元, 董世鹏, 张君毅. 2020. 深度学习目标检测方法综述. 中国图象图形学报, 25(4): 629-654 [DOI: 10.11834/jig.190307]

Zhao Z B, Qi H Y and Nie L Q. 2019. Research overview on visual detection of transmission lines based on deep learning. Guangdong Electric Power, 32(9): 11-23

赵振兵, 齐鸿雨, 聂礼强. 2019. 基于深度学习的输电线路视觉检测研究综述. 广东电力, 32(9): 11-23 [DOI: 10.3969/j.issn.1007-290X.2019.009.002]