浏览全部资源
扫码关注微信
Published： 16 November 2023 ，
移动端阅览
阎光伟， 周香君， 焦润海， 何慧. 2023. 融合先验信息和特征约束的杆塔螺栓缺陷检测. 中国图象图形学报， 28(11):3497-3508
Yan Guangwei， Zhou Xiangjun， Jiao Runhai， He Hui. 2023. Defect detection of tower bolts by fusion of priori information and feature constraints. Journal of Image and Graphics， 28(11):3497-3508
目的
2
螺栓是输电线路中数量最多的紧固件，一旦出现缺陷就会影响电力系统的稳定运行。针对螺栓缺陷自动检测中存在的类内多样性和类间相似性挑战，提出了一种融合先验信息和特征约束的Faster R-CNN（faster regions with convolutional neural network）模型训练方法。
方法
2
在航拍巡检图像预处理阶段，设计了基于先验信息的感兴趣区域提取算法，能够提取被识别目标的上下文区域，从而减少模型训练阶段的数据量，帮助模型在训练阶段关注重点区域，提高其特征提取能力。在模型训练阶段，首先通过费舍尔损失约束Faster R-CNN模型的输出特征生成，使样本特征具有较小的类内距离和较大的类间间隔；然后采用
K
近邻算法处理样本特征得到
K
近邻概率，将其作为难易样本的指示以引导模型后续更加关注难样本。
结果
2
在真实航拍巡检图像构建的螺栓数据集上进行测试，与基线模型相比，本文模型使螺栓识别的平均精度均值（mean average precision，mAP）提高了6.4%，其中正常螺栓识别的平均精度（average precision，AP）提高了0.9%，缺陷螺栓识别的平均精度提高了12%。
结论
2
提出的融合先验信息和特征约束的输电杆塔螺栓缺陷检测方法在缺陷螺栓识别上获得了良好的效果，为实现输电线路螺栓缺陷的自动检测奠定了良好的基础。
Objective
2
Bolts， as fasteners， play a role in connecting and fixing key components in power towers. Adverse environments， mechanical vibrations， and material aging may cause the bolt cotter pins to fall off or come out， thus impacting the normal operations of other parts on the transmission line. Monitoring， testing， and maintenance of bolt conditions play crucial roles in ensuring the uninterrupted operation of a power system. However， the timely and efficient automatic detection of transmission line bolt defects in a power system poses a challenge. Aiming at the challenges of intra-class diversity and inter-class similarity in automatic bolt defect detection， a faster regions with convolutional neural network （Faster R-CNN） model training method based on prior information and feature constraints is proposed in this paper.
Method
2
In the pre-processing of an aerial inspection image， this paper designs a region of interest extraction algorithm based on prior information that can extract the context region of the identified object so as to reduce the amount of data in the training stage， help the model focus on the key areas in the training stage， and improve its feature extraction ability. In the model training stage， the output features of Faster R-CNN model are initially constrained by the Fisher constraint to reduce the intra-class distances of the sample features while increasing their inter-class intervals， which improves the differentiation of sample features. Afterward， the
K
-nearest neighbor algorithm is used to process the sample features to obtain the k-nearest neighbor probability， which is used as an indication of difficult samples to make the model pay attention to difficult samples in the future.
Result
2
All data used in this paper are real aerial inspection images from a power company in Central China. Bolts on the power tower can be divided into bolts with and without pins. Bolts with pins connect and fix the key components and need to bear a large force. Compared with bolts without pins， those with pins are more prone to defects. Therefore， this paper treats bolts with pins as the research object. According to the state of the cotter pin on the bolt， bolts with pins are divided into normal and defective bolts， of which defective bolts have fallen or loose cotter pins. The dataset contains 28 887 images， with each image having a resolution of 5 000 × 3 000 pixels. The training set， validation set， and test set are divided according to the ratio of 8∶1∶1. The proposed model is then tested on this dataset. Compared with the baseline model， the proposed model improves the mean average precision of bolt recognition by 6.4%， increases the average precision of normal bolt identification by 0.9%， and increases the average accuracy of defective bolt identification by 12%.
Conclusion
2
In this paper， the problems of intra-class diversity and inter-class similarity in bolt defect detection are explored ， and a method of transmission tower bolt defect detection based on the fusion of prior information and feature constraints is proposed.This improves the recognition effect of the model on bolt defects and lays a good foundation for the automatic detection of bolt defects in transmission lines.
电力巡检螺栓缺陷检测类内多样性类间相似性先验信息特征约束Faster R-CNN
power inspectionbolt defect detectionintra-class diversityinter-class similarityprior informationfeature constraintsfaster regions with convolutional neural network（Faster R-CNN）
Bochkovskiy A， Wang C Y and Liao H Y M. 2020. YOLOv4： optimal speed and accuracy of object detection ［EB/OL］. ［2022-11-01］. https://arxiv.org/pdf/2004.10934.pdfhttps://arxiv.org/pdf/2004.10934.pdf
Cai Z W and Vasconcelos N. 2018. Cascade R-CNN： delving into high quality object detection//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City， USA： IEEE： 6154-6162 ［DOI： 10.1109/CVPR.2018.00644http://dx.doi.org/10.1109/CVPR.2018.00644］
Cheng G， Zhou P C and Han J W. 2016. RIFD-CNN： rotation-invariant and fisher discriminative convolutional neural networks for object detection//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas， USA： IEEE： 2884-2893 ［DOI： 10.1109/CVPR.2016.315http://dx.doi.org/10.1109/CVPR.2016.315］
Deng J， Dong W， Socher R， Li L J， Li K and Li F F. 2009. Imagenet： a large-scale hierarchical image database//Proceedings of 2009 IEEE Conference on Computer Vision and Pattern Recognition. Miami， USA： IEEE： 248-255 ［DOI： 10.1109/CVPR.2009.5206848http://dx.doi.org/10.1109/CVPR.2009.5206848］
Ester M， Kriegel H P， Sander J and Xu X W. 1996. A density-based algorithm for discovering clusters in large spatial databases with noise.//Proceedings of the 2nd International Conference on Knowledge Discovery and Data Mining. Portland， USA： AAAI Press： 226-231. ［DOI：10.5555/3001460.3001507http://dx.doi.org/10.5555/3001460.3001507］
Everingham M， Van Gool L， Williams C， Winn J， Zisserman A. 2010. The Pascal Visual Object Classes （VOC） challenge. International Journal of Computer Vision， 88（2）： 303-338 ［DOI： 10.1007/S11263-009-0275-4http://dx.doi.org/10.1007/S11263-009-0275-4］
Feng M， Luo W， Yu L， Zhang P， Hao X L， Fan Q， Peng Q W， Zhang T B and Cao L L. 2018. A bolt detection method for pictures captured from an unmanned aerial vehicle in power transmission line inspection. Journal of Electric Power Science and Technology， 33（4）： 135-140
冯敏， 罗旺， 余磊， 张佩， 郝小龙， 樊强， 彭启伟， 张天兵， 曹玲玲. 2018. 适用于无人机巡检图像的输电线路螺栓检测方法. 电力科学与技术学报， 33（4）： 135-140 ［DOI： 10.3969/j.issn.1673-9140.2018.04.019http://dx.doi.org/10.3969/j.issn.1673-9140.2018.04.019］
Girshick R. 2015. Fast R-CNN//Proceedings of 2015 IEEE International Conference on Computer Vision. Santiago， Chile： IEEE： 1440-1448 ［DOI： 10.1109/ICCV.2015.169http://dx.doi.org/10.1109/ICCV.2015.169］
Girshick R， Donahue J， Darrell T and Malik J. 2014. Rich feature hierarchies for accurate object detection and semantic segmentation//Proceedings of 2014 IEEE Conference on Computer Vision and Pattern Recognition. Columbus， USA： 580-587 ［DOI： 10.1109/CVPR.2014.81http://dx.doi.org/10.1109/CVPR.2014.81］
He K M， Zhang X Y， Ren S Q and Sun J. 2016. Deep residual learning for image recognition//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas， USA： IEEE： 770-778 ［DOI： 10.1109/CVPR.2016.90http://dx.doi.org/10.1109/CVPR.2016.90］
Jia M L， Chen B C， Wu Z X， Cardie C， Belongie S and Lim S N. 2021. Rethinking nearest neighbors for visual classification ［EB/OL］. ［2022-11-01］. https://arxiv.org/pdf/2112.08459.pdfhttps://arxiv.org/pdf/2112.08459.pdf
Jiao R H， Liu Y Z， He H， Ma X H and Li Z Y. 2022. A deep learning model for small-size defective components detection in power transmission tower. IEEE Transactions on Power Delivery， 37（4）： 2551-2561 ［DOI： 10.1109/TPWRD.2021.3112285http://dx.doi.org/10.1109/TPWRD.2021.3112285］
Jocher G， Stoken A and Borovec J. 2021. ultralytics/yolov5： v6.0 YOLOv5-x6 1280 models ［EB/OL］. ［2022-11-21］. https://github.com/ultralytics/yolov5https://github.com/ultralytics/yolov5
Li X F， Liu H Y， Liu G H and Su H S. 2021. Transmission line pin defect detection based on deep learning. Power System Technology， 45（8）： 2988-2995
李雪峰， 刘海莹， 刘高华， 苏寒松. 2021. 基于深度学习的输电线路销钉缺陷检测. 电网技术， 45（8）： 2988-2995 ［DOI： 10.13335/j.1000-3673.pst.2020.1028http://dx.doi.org/10.13335/j.1000-3673.pst.2020.1028］
Lin T Y， Dollr P， Girshick R， He K M， Hariharan B and Belongie S. 2017a. Feature pyramid networks for object detection//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu， USA： IEEE： 936-944 ［DOI： 10.1109/CVPR.2017.106http://dx.doi.org/10.1109/CVPR.2017.106］
Lin T Y， Goyal P， Girshick R， He K M and Dollar P. 2017b. Focal loss for dense object detection//Proceedings of 2017 IEEE International Conference on Computer Vision. Venice， Italy： IEEE： 2999-3007［DOI：10.1109/ICCV.2017.324http://dx.doi.org/10.1109/ICCV.2017.324］
Liu W， Anguelov D， Erhan D， Szegedy C， Reed S， Fu C Y and Berg AC. 2016. SSD： single shot multibox detector//Proceedings of the 14th European Conference on Computer Vision. Amsterdam， the Netherlands： Springer： 21-37 ［DOI： 10.1007/978-3-319-46448-0_2http://dx.doi.org/10.1007/978-3-319-46448-0_2］
Liu Z， Lin Y T， Cao Y， Hu H， Wei Y X， Zhang Z， Lin S and Guo B N. 2021. Swin transformer： Hierarchical vision transformer using shifted windows//Proceedings of 2021 IEEE/CVF International Conference on Computer vision. Montreal， Canada： IEEE： 9992-10002 ［DOI： 10.1109/ICCV48922.2021.00986http://dx.doi.org/10.1109/ICCV48922.2021.00986］
Qi Y C， Wu X L， Zhao Z B， Shi B Q and Nie L Q. 2021. Bolt defect detection for aerial transmission lines using Faster R-CNN with an embedded dual attention mechanism. Journal of Image and Graphics， 26（11）： 2594-2604
戚银城， 武学良， 赵振兵， 史博强， 聂礼强. 2021. 嵌入双注意力机制的Faster R-CNN航拍输电线路螺栓缺陷检测. 中国图象图形学报， 26（11）： 2594-2604 ［DOI： 10.11834/jig.200793http://dx.doi.org/10.11834/jig.200793］
Redmon J， Divvala S， Girshick R and Farhadi A. 2016. You only look once： unified， real-time object detection//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas， USA： IEEE： 779-788 ［DOI： 10.1109/CVPR.2016.91http://dx.doi.org/10.1109/CVPR.2016.91］
Redmon J and Farhadi A. 2017. YOLO9000： better， faster， stronger//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu， USA： IEEE： 6517-6525 ［DOI： 10.1109/CVPR.2017.690http://dx.doi.org/10.1109/CVPR.2017.690］
Redmon J and Farhadi A. 2018. YOLOv3： an incremental improvement ［EB/OL］. ［2022-11-01］. https://arxiv.org/pdf/1804.02767.pdfhttps://arxiv.org/pdf/1804.02767.pdf
Ren S Q， He K M， Girshick R and Sun J. 2017. Faster R-CNN： towards real-Time object detection with region proposal networks. IEEE Transactions on Pattern Analysis and Machine Intelligence， 39（6）： 1137-1149 ［DOI： 10.1109/TPAMI.2016.2577031http://dx.doi.org/10.1109/TPAMI.2016.2577031］
Toth J and Gilpin-Jackson A. 2010. Smart view for a smart grid — unmanned aerial vehicles for transmission lines//Proceedings of the 1st International Conference on Applied Robotics for the Power Industry. Montreal， Canada： IEEE： 1-6 ［DOI： 10.1109/CARPI.2010.5624465http://dx.doi.org/10.1109/CARPI.2010.5624465］
Van Der Maaten L and Hinton G. 2008. Visualizing data using t-SNE. Journal of Machine Learning Research， 9（11）： 2579-2605
Wang Y， Chao W L， Weinberger K Q and Van Der Maaten L. 2019. Simpleshot： revisiting nearest-neighbor classification for few-shot learning ［EB/OL］. ［2022-11-01］. https://arxiv.org/pdf/1911.04623.pdfhttps://arxiv.org/pdf/1911.04623.pdf
Wu Y， Chen Y P， Yuan L， Liu Z C， Wang L J， Li H Z and Fu Y. 2020. Rethinking classification and localization for object detection//Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle， USA： IEEE： 10186-10195 ［DOI： 10.1109/CVPR42600.2020.01020http://dx.doi.org/10.1109/CVPR42600.2020.01020］
Xu D， Yu L， Chen T L and Wang J Q. 2017. Detection of catenary rotation binaural based on LBP-HOG feature. Journal of Railway Science and Engineering， 14（2）： 370-378
徐丹， 于龙， 陈唐龙， 王佳祺. 2017. 基于LBP-HOG特征的接触网旋转双耳区域识别. 铁道科学与工程学报， 14（2）： 370-378 ［DOI： 10.19713/j.cnki.43-1423/u.2017.02.024http://dx.doi.org/10.19713/j.cnki.43-1423/u.2017.02.024］
Zeng Y B， Wang X H， Peng X G， Huang J L， Jian S C and Lu D. 2020. Research on risk modeling and forecasting method of transmission line defects. Power System Protection and Control， 48（10）： 91-98
曾勇斌， 王星华， 彭显刚， 黄景林， 简胜超， 鲁迪. 2020. 输电线路缺陷风险建模及其预测方法研究. 电力系统保护与控制， 48（10）： 91-98 ［DOI： 10.19783/j.cnki.pspc.190935http://dx.doi.org/10.19783/j.cnki.pspc.190935］
Zhang H， Wu C R， Zhang Z Y， Zhu Y， Lin H B， Zhang Z， Sun Y， He T， Mueller J， Manmatha R， Li M and Smola A. 2022. ResNeSt： split-attention networks//Proceedings of 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. New Orleans， USA： IEEE： 2735-2745 ［DOI： 10.1109/CVPRW56347.2022.00309http://dx.doi.org/10.1109/CVPRW56347.2022.00309］
Zhao W Q， Jia M Y， Zhai Y J and Zhao Z B. 2022a. Detection method for pin-losing-bolts in transmission lines based on improved YOLOv5s ［J/OL］. ［2022-11-01］. Journal of North China Electric Power University， 1-10
赵文清， 贾梦颖， 翟永杰， 赵振兵. 2022. 基于改进YOLOv5s的输电线路螺栓缺销检测方法［J/OL］. ［2022-11-01］. 华北电力大学学报， 1-10 http://kns.cnki.net/kcms/detail/13.1212.TM.20220920.0949.002.html
Zhao W Q and Xu M F. 2022b. Bolt missing pins detection of transmission lines based on inter-level cross feature fusion. Journal of Image and Graphics， 27（11）： 3222-3231
赵文清， 徐敏夫. 2022. 基于隔级交叉特征融合的输电线螺栓缺销检测. 中国图象图形学报， 27（11）： 3222-3231 ［DOI： 10.11834/jig.210581http://dx.doi.org/10.11834/jig.210581］
Zhao Z B， Jiang Z G， Li Y X， Qi Y C， Zhai Y J， Zhao W Q and Zhang K. 2021. Overview of visual defect detection of transmission line components. Journal of Image and Graphics， 26（11）： 2545-2560
赵振兵， 蒋志钢， 李延旭， 戚银城， 翟永杰， 赵文清， 张珂. 2021. 输电线路部件视觉缺陷检测综述. 中国图象图形学报， 26（11）： 2545-2560 ［DOI： 10.11834/jig.200689http://dx.doi.org/10.11834/jig.200689］
Zhao Z B， Li Y X， Zhen Z， Zhai Y J， Zhang K and Zhao W Q. 2020. Typical fittings detection method with faster R-CNN combining KL divergence and shape constraints. High Voltage Engineering， 46（9）： 3018-3026
赵振兵， 李延旭， 甄珍， 翟永杰， 张珂， 赵文清. 2020. 结合KL散度和形状约束的Faster R-CNN典型金具检测方法. 高电压技术， 46（9）： 3018-3026 ［DOI： 10.13336/j.1003-6520.hve.20200507023http://dx.doi.org/10.13336/j.1003-6520.hve.20200507023］
Zhao Z B， Qi H Y， Qi Y C， Zhang K， Zhai Y J and Zhao W Q. 2020. Detection method based on automatic visual shape clustering for pin-missing defect in transmission lines. IEEE Transactions on Instrumentation and Measurement， 69（9）： 6080-6091 ［DOI： 10.1109/TIM.2020.2969057http://dx.doi.org/10.1109/TIM.2020.2969057］
Zhu X Z， Su W J， Lu L W， Li B， Wang X G and Dai J F. 2020. Deformable DETR： deformable transformers for end-to-end object detection ［EB/OL］. ［2022-11-01］. https://arxiv.org/pdf/2010.04159.pdfhttps://arxiv.org/pdf/2010.04159.pdf
相关作者
相关机构
京公网安备11010802024621