浏览全部资源
扫码关注微信
Received：02 November 2018，
Revised：2018-12-21，
Published：16 July 2019
移动端阅览
目的
2
目标建模是机器视觉领域的主要研究方向之一，主动目标建模是在保证建模完整度的情况下，通过有计划地调节相机的位姿参数，以更少的视点和更短的运动路径实现目标建模的智能感知方法。为了反映主动目标建模的研究现状和最新进展，梳理分析了2004年以来的相关文献，对国内外研究方法做出概括性总结。
方法
2
以重构模型类型和规划视点所用信息作为划分依据，将无模型的主动目标建模方法分为基于表面的主动目标建模方法、基于搜索的目标建模方法和两者相结合的方法3大类，重点对前两类方法进行综述，首先解释了每类方法的基本思想，总结每类方法涉及的问题，然后对相关问题的主要研究方法进行归纳和分析，最后将各个问题的解决方法进行合理的搭配组合，形成不同的主动目标建模方法，并对各类方法的优势和局限性进行了总结。
结果
2
各类主动目标建模算法在适用场景范围、计算复杂度等方面存在差异，但相对于传统的被动目标建模方法，当前的主动目标建模算法已经能够极大程度地提高建模任务的质量和降低建模所需代价。
结论
2
基于表面的主动目标建模方法思想相对简单，但仅适用于表面简单的目标建模。基于搜索的目标建模方法能够量化地评价每一个候选视点，适用广泛且涉及的问题相对于基于表面的方法有更大的解决空间，有更多的研究成果产生。将二者涉及问题的不同研究方法相搭配，可以构成不同的主动目标建模方法子类。
Objective
2
Target modeling is one of the main research directions in the field of machine vision
and this technology is widely used in various fields. When modeling the geometry of an object
the data obtained from one viewpoint are often incomplete
and large-area losses may even occur. Therefore
obtaining the information of the target from different viewpoints and fusing the information are necessary to achieve a complete geometric modeling of the target. Active object reconstruction is an intelligent perception method that achieves target modeling with few viewpoints and short motion paths by systematically adjusting the pose parameters of the camera while ensuring model integrity. To reflect the research status and latest development of active object reconstruction
relevant studies since 2004 are combed and analyzed
and a summary of domestic and foreign research methods is made.
Method
2
At present
active object reconstruction is mainly aimed at two task types: model-based and non-model active object reconstruction. Model-based methods pre-plan a series of viewpoints before modeling and can achieve full coverage of the target with high quality. Non-model methods have no information on the target at all
and view planning is performed in real time during modeling. In practical applications
the second category appears frequently and is difficult; thus
this study only summarizes non-model methods. On the basis of the rebuilt model type and the information used during view planning
non-model active object reconstruction methods are divided into three categories
namely
surface based
search based
and combined. The basic ideas of each type of method are explained
and the problems involved are summarized. Surface-based methods use point cloud and triangular patch models. They extract shape information from the obtained local model and classify the shape of the unknown region to determine the next viewpoint. Search-based methods use voxel models. A certain method is employed to determine the candidate viewpoints
and then these viewpoints are scored by a reasonable evaluation function. The candidate viewpoint with the highest score is used as the next best view. The combined method uses the surface and voxel models and merges the advantages of the two methods comprehensively to provide effective information for view planning. However
combined methods have not been investigated much recently
and the first two methods have mainly been the focus. Surface-based methods involve problems of detection direction determination
unknown surface prediction
and next-best viewpoint determination. Search-based methods involve problems of model type selection
search space determination
undetected area prediction
and design of the evaluation function to sort candidate viewpoints. The main research methods for these related problems are summarized and analyzed
and the solutions to each problem are combined reasonably to form different active object reconstruction methods.
Result
2
In surface-based active object reconstruction methods
the manner of determining the direction of detection and predicting the unknown area has an important impact on the view planning effect. When selecting an edge point to determine the direction of detection
the use of the quantitative indicator method is more reliable than the use of the spatial position method to express the unknown region
but its computational complexity is higher. In addition
using an indirect method to predict an unknown surface may be simpler than using a direct method
but it results in larger fitting errors. In general
surface-based methods are relatively simple
and the process of each view planning consumes minimal time. However
the unknown region depends on its adjacent surface trend to predict; thus
this method is only suitable for reconstructing objects with regular shapes. Search-based active object reconstruction methods quantitatively evaluate each candidate view. The octomap model is more efficient than other probabilistic voxel models when selecting model types. The selection of candidate viewpoints using dynamic search space methods has higher computational complexity than using fixed search space methods
but such methods have no limitation on the target size
and their application scenario is extensive. When predicting the information contained in an unknown voxel
its relative positional relationship with the known voxel can be utilized; thus
using this method for the next view planning can maximize the known information compared with not updating the unknown voxel. When determining the evaluation function
information gain modules may be added to the evaluation function
and the adjacent frame overlap ratio optimization modules
the neighboring viewpoint distance optimization modules
and the reconstructed surface quality optimization items may be added as needed. The information gain of the viewpoint is obtained by counting the voxel gain in the field of view. Differences in voxel gain calculation and statistical methods directly affect the information gain value of the viewpoint. With these search-based methods
the next view planning works well
but the process is time consuming. Moreover
the problems involved in such methods have a larger solution space than those involved in surface-based methods. Therefore
more research results are generated in search-based active object reconstruction methods. However
such methods are relatively computationally intensive
and in most cases
the views are not continuously pulsating in the search space
and point cloud registration is not considered.
Conclusion
2
Researchers who study active object reconstruction have made some progress at present
but the accuracy and efficiency of active reconstruction can still be improved. Other feasible research directions are provided in the end
and these could serve as a reference for future research in this direction
such as introducing a priori information into the process of view planning
combining surface- and search-based methods
and building perceptual intelligence systems that are suitable for different tasks.
Fan H Q, Su H, Guibas L. A point set generation network for 3D object reconstruction from a single image[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, HI, USA: IEEE, 2017: 1-6.[ DOI: 10.1109/CVPR.2017.264 http://dx.doi.org/10.1109/CVPR.2017.264 ]
Mei F, Liu J, Li C P, et al. Improved RGB-D camera based indoor scene reconstruction[J]. Journal of Image and Graphics, 2015, 20(10):1366-1373.
梅峰, 刘京, 李淳芃, 等.基于RGB-D深度相机的室内场景重建[J].中国图象图形学报, 2015, 20(10):1366-1373. [DOI:10.11834/jig.20151010]
He B W, Chen Z P. Determination of the common view field in hybrid vision system and 3D reconstruction method[J]. Robot, 2011, 33(5):614-620.
何炳蔚, 陈志鹏.混合视觉系统中共同视场的确定与3维重建方法[J].机器人, 2011, 33(5):614-620. [DOI:10.3724/SP.J.1218.2011.00614]
Lin C H, Kong C, Lucey S. Learning efficient point cloud generation for dense 3D object reconstruction[C]//Proceedings of the 32nd AAAI Conference on Artificial Intelligence. New Orleans, Louisiana, USA: AAAI, 2018: 7114-7121.
Choy C B, Xu D F, Gwak J, et al. 3D-R2N2: a unified approach for single and multi-view 3D object reconstruction[C]//Proceedings of the 14th European Conference on Computer Vision. Amsterdam, The Netherlands: Springer, 2016: 628-644.[ 10.1007/978-3-319-46484-8_38 DOI: http://dx.doi.org/10.1007/978-3-319-46484-8_38 ]
Lun Z L, Gadelha M, Kalogerakis E, et al. 3D shape reconstruction from sketches via multi-view convolutional networks[C]//Proceedings of 2017 International Conference on 3D Vision. Qingdao, China: IEEE, 2017: 67-77.[ DOI: 10.1109/3DV.2017.00018 http://dx.doi.org/10.1109/3DV.2017.00018 ]
Chou H L, Chou H L, Chen Z. A quality controllable multi-view object reconstruction method for 3D imaging systems[J]. Journal of Visual Communication and Image Representation, 2010, 21(5-6):427-441.[DOI:10.1016/j.jvcir.2010.03.004]
Connolly C. The determination of next best views[C]//Proceedings of 1985 IEEE International Conference on Robotics and Automation. St. Louis, MO, USA: IEEE, 1985: 432-435.[ DOI: 10.1109/ROBOT.1985.1087372 http://dx.doi.org/10.1109/ROBOT.1985.1087372 ]
Aloimonos J, Weiss I, Bandyopadhyay A. Active vision[J]. International Journal of Computer Vision, 1988, 1(4):333-356.[DOI:10.1007/BF00133571]
Jing W, Polden J, Lin W, et al. Sampling-based view planning for 3D visual coverage task with unmanned aerial vehicle[C]//Proceedings of 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems. Daejeon, South Korea: IEEE, 2016: 1808-1815.[ DOI: 10.1109/IROS.2016.7759288 http://dx.doi.org/10.1109/IROS.2016.7759288 ]
Hepp B, Nießner M, Hilliges O. Plan3D:viewpoint and trajectory optimization for aerial multi-view stereo reconstruction[J]. ACM Transactions on Graphics, 2018, 38(1):1-31.[DOI:1705.09314]
Kaba M D, Uzunbas M G, Lim S N. A reinforcement learning approach to the view planning problem[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, HI, USA: IEEE, 2017: 5094-5102.[ DOI: 10.1109/CVPR.2017.541 http://dx.doi.org/10.1109/CVPR.2017.541 ]
Scott W, Roth G, Rivest J F. View planning for automated three-dimensional object reconstruction and inspection[J]. ACM Computing Surveys, 2003, 35(1):64-96.[DOI:10.1145/641865.641868]
Chen S Y, Li Y F, Kwok N M. Active vision in robotic systems:a survey of recent developments[J]. The International Journal of Robotics Research, 2011, 30(11):1343-1377.[DOI:10.1177/0278364911410755]
He B W. A viewpoint planning method with self-termination[J]. Journal of Image and Graphics, 2006, 11(12):1827-1833.
何炳蔚.一种具有自终止特性的视点规划方法[J].中国图象图形学报, 2006, 11(12):1827-1833. [DOI:10.11834/jig.2006012318]
He B W, Zhou X L. Research of sensor planning method in line laser three-dimensional measurement system[J]. Chinese Journal of Lasers, 2010, 37(6):1618-1625.
何炳蔚, 周小龙.线激光3维测量仪中视觉传感器规划方法研究[J].中国激光, 2010, 37(6):1618-1625.
Fang W, He B W. Automatic view planning for 3D reconstruction and occlusion handling based on the integration of active and passive vision[C]//Proceedings of 2012 IEEE International Symposium on Industrial Electronics. Hangzhou, China: IEEE, 2012: 1116-1121.[ DOI: 10.1109/ISIE.2012.6237245 http://dx.doi.org/10.1109/ISIE.2012.6237245 ]
Huang L P, Zuo J Q, Zhang L. Studies to determine the next best view of the robot visual servo-based approach[J]. Industrial Instrumentation&Automation, 2015, (2):7-11.
黄立平, 左骏秋, 张磊.基于机器人视觉伺服的确定下一最优视点的方法研究[J].工业仪表与自动化装置, 2015, (2):7-11. [DOI:10.3969/j.issn.1000-0682.2015.02.002]
Yao X T, Wu L L, Ma Y L, et al. Research on next best view in automatic 3D reconstruction[J]. Journal of Jiangxi Normal University:Natural Science Edition, 2013, 37(6):569-573.
姚兴田, 吴亮亮, 马永林, 等.自动3维重构中确定下一最优视点的方法研究[J].江西师范大学学报:自然科学版, 2013, 37(6):569-573. [DOI:10.3969/j.issn.1000-5862.2013.06.005]
Scott W R. Model-based view planning[J]. Machine Vision and Applications, 2009, 20(1):47-69.[DOI:10.1007/s00138-007-0110-2]
Chen S Y, Li Y F. Automatic sensor placement for model-based robot vision[J]. IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 2004, 34(1):393-408.[DOI:10.1109/TSMCB.2003.817031]
Mavrinac A, Chen X, Alarcon-Herrera J L. Semiautomatic model-based view planning for active triangulation 3-D inspection systems[J]. IEEE/ASME Transactions on Mechatronics, 2015, 20(2):799-811.[DOI:10.1109/TMECH.2014.2318729]
Munkelt C, Breitbarth A, Notni G, et al. Multi-view planning for simultaneous coverage and accuracy optimisation[C]//Proceedings of the British Machine Vision Conference. Dundee, UK: BMVA, 2010: 1-11.[ DOI: 10.5244/C.24.118 http://dx.doi.org/10.5244/C.24.118 ]
Wakisaka E, Kanai S, Date H. Model-based next-best-view planning of terrestrial laser scanner for HVAC facility renovation[J]. Computer-Aided Design and Applications, 2018, 15(3):353-366.[DOI:10.1080/16864360.2017.1397886]
Schmid K, Hirschmüller H, Dömel A, et al. View planning for multi-view stereo 3D reconstruction using an autonomous multicopter[J]. Journal of Intelligent&Robotic Systems, 2012, 65(1-4):309-323.[DOI:10.1007/s10846-011-9576-2]
Torabi L, Gupta K. An autonomous six-DOF eye-in-hand system for in situ 3D object modeling[J]. The International Journal of Robotics Research, 2012, 31(1):82-100.[DOI:10.1177/0278364911425836]
Kriegel S, Rink C, Bodenmüller T, et al. Efficient next-best-scan planning for autonomous 3D surface reconstruction of unknown objects[J]. Journal of Real-Time Image Processing, 2015, 10(4):611-631.[DOI:10.1007/s11554-013-0386-6]
Pito R. A solution to the next best view problem for automated surface acquisition[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1999, 21(10):1016-1030.[DOI:10.1109/34.799908]
Chen S Y, Li Y F. Vision sensor planning for 3D model acquisition[J]. IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 2005, 35(5):894-904.[DOI:10.1109/TSMCB.2005.846907]
Krumbein W C. The "sorting out" of geological variables illustrated by regression analysis of factors controlling beach firmness[J]. Journal of Sedimentary Research, 1959, 29(4):575-587.[DOI:10.1306/74D7099D-2B21-11D7-8648000102C1865D]
Kriegel S, Bodenmüller T, Suppa M, et al. A surface-based next-best-view approach for automated 3D model completion of unknown objects[C]//Proceedings of 2011 IEEE International Conference on Robotics and Automation. Shanghai, China: IEEE, 2011: 4869-4874.[ DOI: 10.1109/ICRA.2011.5979947 http://dx.doi.org/10.1109/ICRA.2011.5979947 ]
Wilhelms J, Van Gelder A. Octrees for faster isosurface generation[J]. ACM Transactions on Graphics, 1992, 11(3):201-227.[DOI:10.1145/130881.130882]
Pathak K, Birk A, Poppinga J, et al. 3D forward sensor modeling and application to occupancy grid based sensor fusion[C]//Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems. San Diego, CA, USA: IEEE, 2007: 2059-2064.[ DOI: 10.1109/IROS.2007.4399406 http://dx.doi.org/10.1109/IROS.2007.4399406 ]
Ryde J, Hu H S. 3D mapping with multi-resolution occupied voxel lists[J]. Autonomous Robots, 2010, 28(2):169-185.[DOI:10.1007/s10514-009-9158-3]
Payeur P, Hebert P, Laurendeau D, et al. Probabilistic octree modeling of a 3D dynamic environment[C]//Proceedings of the International Conference on Robotics and Automation. Albuquerque, NM, USA: IEEE, 1997: 1289-1296.[ DOI: 10.1109/ROBOT.1997.614315 http://dx.doi.org/10.1109/ROBOT.1997.614315 ]
Wurm K M, Hornung A, Bennewitz M, et al. OctoMap: a probabilistic, flexible, and compact 3D map representation for robotic systems[C]//Proceedings of the ICRA 2010 Workshop on Best Practice in 3D Perception and Modeling for Mobile Manipulation. Anchorage, AK, USA: ICRA, 2010.[ DOI: 10.1.1.176.724 http://dx.doi.org/10.1.1.176.724 ]
Hornung A, Wurm K M, Bennewitz M, et al. OctoMap:an efficient probabilistic 3D mapping framework based on octrees[J]. Autonomous Robots, 2013, 34(3):189-206.[DOI:10.1007/s10514-012-9321-0]
Daudelin J, Campbell M. An adaptable, probabilistic, next-best view algorithm for reconstruction of unknown 3-D objects[J]. IEEE Robotics and Automation Letters, 2017, 2(3):1540-1547.[DOI:10.1109/LRA.2017.2660769]
Potthast C, Sukhatme G S. A probabilistic framework for next best view estimation in a cluttered environment[J]. Journal of Visual Communication and Image Representation, 2014, 25(1):148-164.[DOI:10.1016/j.jvcir.2013.07.006]
Vasquez-Gomez J I, Sucar L E, Murrieta-Cid R, et al. Tree-based search of the next best view/state for three-dimensional object reconstruction[J]. International Journal of Advanced Robotic Systems, 2018, 15(1):1-11.[DOI:10.1177/1729881418754575]
Delmerico J, Isler S, Sabzevari R, et al. A comparison of volumetric information gain metrics for active 3D object reconstruction[J]. Autonomous Robots, 2018, 42(2):197-208.[DOI:10.1007/s10514-017-9634-0]
Vasquez-Gomez J I, Sucar L E, Murrieta-Cid R, et al. Volumetric next-best-view planning for 3D object reconstruction with positioning error[J]. International Journal of Advanced Robotic Systems, 2014, 11(10):159-171.[DOI:10.5772/58759]
Isler S, Sabzevari R, Delmerico J, et al. An information gain formulation for active volumetric 3D reconstruction[C]//Proceedings of 2016 IEEE International Conference on Robotics and Automation. Stockholm, Sweden: IEEE, 2016: 3477-3484.[ DOI: 10.1109/ICRA.2016.7487527 http://dx.doi.org/10.1109/ICRA.2016.7487527 ]
Vasquez-Gomez J I, Sucar L E, Murrieta-Cid R. View planning for 3D object reconstruction with a mobile manipulator robot[C]//Proceedings of 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems. Chicago, IL, USA: IEEE, 2014: 4227-4233.[ DOI: 10.1109/IROS.2014.6943158 http://dx.doi.org/10.1109/IROS.2014.6943158 ]
Yamauchi B. A frontier-based approach for autonomous exploration[C]//Proceedings of 1997 IEEE International Symposium on Computational Intelligence in Robotics and Automation CIRA'97. 'Towards New Computational Principles for Robotics and Automation. Monterey, CA, USA: IEEE, 1997: 146-151.[ DOI: 10.1109/CIRA.1997.613851 http://dx.doi.org/10.1109/CIRA.1997.613851 ]
Amanatides J, Woo A. A fast voxel traversal algorithm for ray tracing[C]//Proceedings of Eurographics. Amsterdam: the Netherlands, 1987: 3-10.[ DOI: 10.2312/egtp19871000 http://dx.doi.org/10.2312/egtp19871000 ]
相关作者
相关机构
京公网安备11010802024621
