Current Issue Cover

李思聪, 朱枫, 吴清潇(中国科学院沈阳自动化研究所)

摘 要
目的 在工业机器人视觉领域中,经常需要对一些由拼装、冲压或贴合等工艺造成的形变工件进行精准定位,这些工件的大部分特征表现出一定程度的非刚性,其它具备良好一致性的部分通常特征简单,如稀疏边缘,几何形状约束弱,不具备全局唯一性,导致一些常用的目标检测算法精度不足或鲁棒性不强,难以满足实际需求。方法 针对这一问题,提出融合边缘与灰度特征的形变工件精准定位方法,这是一个两阶段的定位方法。第一阶段提出多归一化互相关的模板匹配MNCC(Multi Normalized Cross Correlation)方法检测形变目标,利用余弦距离下的灰度聚类获得均值模板,通过滑动窗口的方式,结合金字塔跟踪,自顶向下地优先搜索类均值模板,得到类匹配候选,然后进行类内细搜索获得最佳位置匹配。第二阶段提出一种改进的形状匹配方法T-SBM(Truncated Shape-Based Matching),通过改变原始SBM(Shape-Based Matching)的梯度方向内积的计算方式,对负梯度极性方向截断,削弱目标背景不稳定导致局部梯度方向反转时对整体评分的负贡献,改善边缘稀疏或特征简单导致检测鲁棒性低的问题。最后提出二维高斯条件密度评价,将灰度特征、形状特征和形变量进行综合加权,获得理想目标评价,实现序贯检测。结果 实验部分分别与SBM、归一化互相关匹配算法NCC (Normalized Cross Correlation)、LINE2d算法和YOLOv5s算法在5种类型工件的472张真实工业图像上进行了对比测试,在检出分值大于0.8(实际常用的阈值区间)时,提出算法的召回率优于其它几种测试算法;在IoU阈值0.9时的平均检测准确率为81.7%,F1-Score为95%,两组指标相比其它测试算法分别至少提升了10.8%和8.3%。在平均定位精度方面,提出算法的定位偏差在IoU阈值0.9时达到了2.44个像素,在5种测试算法中的表现也为最佳。结论 提出了一种两阶段的定位方法,该方法适用于检测工业场景中由拼装、冲压、贴合等工艺制成的形变工件并能够进行精准定位,尤其适用于工业机器人视觉引导定位应用场景,并在实际项目中得到了应用。
Precise Positioning Method of Deformed Workpiece by fusing edge and grayscale features

LI Sicong, ZHU Feng, WU Qingxiao(Shenyang Institute of Automation, Chinese Academy of Sciences)

Objective In industrial robot vision, it is often necessary to accurately detect deformed workpieces caused by assembly, stamping, or lamination. These workpieces sometimes show non-rigid characteristics, such as dislocation or twist deformation. Most features do not maintain good shape consistency, while the remaining undeformed parts are generally simple, e.g., sparse edges, which are not globally unique. In addition, it is not realistic to obtain massive training samples before workpieces are mass-produced. The above reasons lead to some commonly used object detection methods with insufficient accuracy or weak robustness, challenging to meet the actual needs. Method Aiming at the problem, a two-stage precise positioning method of deformed workpieces by fusing edge and grayscale features is proposed. The first stage is the coarse position detection of deformed targets based on grayscale features, and the second stage is precise positioning based on shape features. The innovation of the first stage lies in the fact that a MNCC (Multi Normalized Cross Correlation) matching method is proposed, which includes offline and online parts. In the offline part, the grayscale clustering algorithm at cosine distance is used to obtain the class-mean template, which characterizes a class center of similar features in the target deformation space. Therefore, a small number of class-mean templates can be used to represent the grayscale features of the target"s deformation after discretization. In the online part, by sliding window, combined with pyramid tracking, the class-mean template is searched preferentially from top to bottom to acquire the class-mean candidates. Then a detailed search of the candidates within the class is carried out to obtain the best match, achieve the efficient matching of deformed workpieces and complete the task of coarse position detection during the first stage. A T-SBM(Truncated Shape-Based Matching) method is proposed in the second stage to achieve precise positioning using the target edge. By changing the similarity measurement based on gradient"s inner product, the gradient vector of opposite direction is truncated, so there is no negative evaluation of the local edge points. The improvement restricts the negative contribution on the overall similarity score when the local gradient direction is inverted due to the inconsistency of the target background. Prevent the simple repsentiations of sparse edges leads to low robustness. Finally, a two-dimensional Gaussian conditional density evaluation is proposed to reasonably combine grayscale features, shape features, and deformation quantity. The candidate who holds the top score wins the best match. The proposed evaluation provides a comprehensive estimation for the ideal target position detection under the 3-sigma principle, realizes the precise positioning of the deformed workpiece, and completes the sequential detection.Result In the experimental part, the proposed method was compared with classical SBM(Shape-Based Matching), NCC(normalized cross-correlation), LINE2d, and YOLOv5s on 472 authentic industrial images consisting of five types of workpieces, namely TV back, led panel, screw hole, metal tray, and aluminum plate. Industry Vision Software, HALCON, provides the implementation of SBM and NCC, and LINE2d is from OpenCV. The evaluation contained F1-Score, recall, detection accuracy, and average pixel distance, of which the first three regard detection robustness, and the last regards positioning accuracy. At IoU 0.9, a strict enough threshold for precise positioning, the average detection accuracy and the F1-Score of the proposed method are 81.7% and 95%, improved by 10.8% and 8.3%, respectively, compared with other test methods. When the minscore threshold is less than 0.8, the recall of the proposed method is slightly inferior to the NCC method. However, when the minscore is greater than 0.8, a commonly used threshold interval, it outperforms the other methods significantly. Regarding average positioning accuracy, the positioning error based on the Euclidean distance of the proposed method is as low as 2.44 pixels at the IoU threshold of 0.9, which is far better than the other test methods. Conclusion We proposed a two-stage precision positioning method for deformed workpieces made by assembly, stamping, or lamination. In the experiment, the proposed method outperforms the other test methods on detection robustness and positioning accuracy, which