Preliminary exploration on the construction of urban video real-scene map

Jun Wu; Qichang Liu; Mingyi Huang

doi:10.11834/jig.190112

Scholar View | Views : 0 下载量: 64 CSCD: 4

PDF
Export
Share
Collection
Album

Preliminary exploration on the construction of urban video real-scene map
Vol. 24, Issue 9, Pages: 1409-1415(2019)
Received：19 April 2019，

Revised：2019-4-30，

Published：16 September 2019
DOI： 10.11834/jig.190112
稿件说明：

移动端阅览

DOI：

Jun Wu, Qichang Liu, Mingyi Huang. Preliminary exploration on the construction of urban video real-scene map[J]. Journal of Image and Graphics, 2019, 24(9): 1409-1415. DOI： 10.11834/jig.190112.

摘要

城市视频实景地图兼具地图立体空间、视频时间4维度层面信息统一表达能力，对于我国城市立体监控系统构建、互联网地图产品发展，以及未来实景3维中国建设战略实施具有重要意义和应用价值。为引起更多研究者进行探索，对城市视频实景地图构建方法、技术及其应用前景进行讨论。从增强虚拟环境技术（AVE）角度出发，对融合全景视频与地理3维模型构建城市视频实景地图涉及的全景摄像机标定、全景视频空间配准及视频纹理映射、实时渲染系列技术、方法进行了梳理。经过分析得出：1）适合传统“针孔”模型的摄像机标定、影像空间配准理论和方法，需根据全景摄像机球面投影模型进行拓展；2）适合静态纹理的大规模3D场景渲染LOD（levels of detail）技术和策略，需结合视频传输带宽限制、高帧率特点进行技术创新。城市视频实景地图构建是一项值得重视的崭新课题，将有力促进互联网、人工智能前沿技术发展，有望给相关行业带来万亿级市场机遇。

Abstract

UVRSM (urban video real-scene map) has important significance for construction of urban monitoring system

the development of Internet map products

and the implementation of future "real-scene China construction" strategy given the ability of unified information expression in three-dimensional space and time dimension. This paper presents a preliminary exploration on the construction of UVRSM for the convenience of researchers in related fields. From the point of view of augmented virtual environment (AVE) technology

this work proposes to construct the UVRSM by integrating panoramic video and geographic three-dimensional models. To this end

the technologies and methods involved in panoramic camera calibration

panoramic video geo-registration

automatic video texture mapping

and real-time rendering are comprehensively discussed. Some valuable research ideas and schemes are given. The theory and method of camera calibration and image geo-registration suitable for traditional pinhole model need to be expanded according to the spherical projection model of panoramic camera. The LOD (level of detail) technology and strategy for large-scale 3D scene rendering suitable for static texture need to be innovated to obtain the characteristics of video transmission bandwidth limitation and high frame rate. The construction of UVRSM is a new topic and will strongly promote the development of Internet and AI (artificial intelliqence) frontier technology. This field is expected to bring trillions of market opportunities to UVRSM-related domains or industries.

关键词

Keywords

references

Wang H W. Reform of public crisis management mode in the perspective of holistic approach to national security[J]. Administrative Tribune, 2018, 25(4): 20-26.

王宏伟.总体国家安全观视角下公共危机管理模式的变革[J].行政论坛, 2018, 25(4): 20-26.[DOI: 10.16637/j.cnki23-1360/d.2018.04.003]

Wang Y S, Xing G L. Research on evaluation index system of networked application for public security video surveillance construction[J]. China Security&Protection, 2018, (4): 96-101.

王永升, 邢更力.公共安全视频监控建设联网应用评价指标体系研究[J].中国安防, 2018, (4):96-101.[DOI: CNKI: SUN: ZGAF.0.2018-04-024]

Li D R. New mission for surveying, mapping and geomatics in Smart Earth era[J]. Geomatics World, 2013, 37(1): 6-7.

李德仁.智慧地球时代测绘地理信息学的新使命[J].地理信息世界, 2013, 37(1): 6-7.

Li D R, Shao Z F. Research on physical city, digital city and smart city[J]. Geospatial Information, 2018, 16(9): 1-4.

李德仁, 邵振峰.论物理城市、数字城市和智慧城市[J].地理空间信息, 2018, 16(9): 1-4.[DOI: 10.3969/j.issn.1672-4623.2018.09.001]

Zhou Z, Zhou Y, Xiao J J. Survey on augmented virtual environment and augmented reality[J]. Scientia Sinica Informationis, 2015, 45(2): 157-180.

周忠, 周颐, 肖江剑.虚拟现实增强技术综述[J].中国科学, 2015, 45(2): 157-1804.[DOI:10.1360/N112014-00076]

Sawhney H S, Arpa A, Kumar R, et al. Video flashlights: real time rendering of multiple videos for immersive model visualization[C]//Proceedings of the 13th Eurographics Workshop on Rendering. Pisa, Italy: Eurographics Association, 2002, 28: 157-168.

Rossi D, Petrucci E, Fazzini S. A framework to increase the video-mapping accuracy of an architectural heritage mock-up[C]//Proceedings of ACM Press the 2014 Virtual Reality International Conference. Laval, France: ACM, 2014: 1-4.[ DOI:10.1145/2617841.2617844 http://dx.doi.org/10.1145/2617841.2617844 ]

闫阿奇, 杨建峰, 曹剑中, 等.航天鱼眼相机光学系统设计的研究[J].光学学报, 2011, 31(10): 1022004.

Wu X J, Guo Y X. A 360° panoramic imaging design method for cylindrical object[J]. Acta Optica Sinica, 2015, 35(9): #0923002.

吴晓军, 郭亚雪.一种圆柱物体360°全景成像装置设计方法[J].光学学报, 2015, 35(9): #0923002.

Wu Z J, Wu Q Y, Zhang B C. A new calibration method for fisheye lens based on spherical model[J]. Chinese Journal of Lasers, 2015, (5): 226-233.

吴泽俊, 吴庆阳, 张佰春.一种新的基于球面模型的鱼眼镜头标定方法[J].中国激光, 2015, (5): 226-233.

Wu J, Wang L R, Huang M Y, et al. High precision calibration of fisheye camera with single image under multiple geometric constraints[J]. Acta Optica Sinica, 2018, 38(11): 1115001.

吴军, 王玲容, 黄明益, 等.多几何约束下的鱼眼相机单像高精度标定[J].光学学报, 2018, 38(11): 1115001.

Pi Y D, Li X, Chen Z Y, et al. Calibration and rectification of fisheye images based on three-dimensional control field[J]. Acta Optica Sinica, 2017, 37(1): 0115001.

皮英冬, 李欣, 陈智勇, 等.基于3维控制场的鱼眼影像检校和纠正方法[J].光学学报, 2017, 37(1): 0115001.

Kanatani K. Calibration of ultrawide fisheye lens cameras by eigenvalue minimization[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013, 35(4): 813-822.[DOI: 10.1109/TPAMI.2012.146]

Aghayari S, Saadatseresht M, Omidalizarandi M, et al. Geometric calibration of full spherical panoramic Ricoh-theta camera[C]//ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 2017, Ⅳ-1/W1: 237-245.

Gurtner A, Greer D G, Glassock R, et al. Investigation of fish-eye lenses for small-UAV aerial photography[J]. IEEE Transactions on Geoscience and Remote Sensing, 2009, 47(3): 709-721.[DOI: 10.1109/TGRS.2008.2009763]

Ji S P, Shi Y. Comparison of two sensor models for multi-camera rig system in measurements[J]. Acta Geodaetica et Cartographica Sinica, 2014, 43(12): 1252-1258.

季顺平, 史云.多镜头组合型全景相机两种成像模型的定位精度比较[J].测绘学报, 2014, 43(12): 1252-1258.[DOI: 10.13485/j.cnki.11-2089.2014.0169]

Lee J, Kim B, Kim K, et al. Rich360: optimized spherical representation from structured panoramic camera arrays[J]ACM Transactions on Graphics, 2016, 35(4), Article 63: 1-11.[ DOI:10.1145/2897824.2925983 http://dx.doi.org/10.1145/2897824.2925983 ]

Wang D, Liu F Y, Chen T E, et al. The method of cylinder panoramic image mosaic based on camera calibration parameters[J]. Science of Surveying and Mapping, 2016, 41(7): 150-154, 143.

王冬, 刘凤英, 陈天恩, 等.一种相机标定参数的柱面全景影像拼接方法[J].测绘科学, 2016, 41(7): 150-154, 143.[DOI: 10.16251/j.cnki.1009-2307.2016.07.027]

Mei W S, Hu S P, Li M S, et al. Rotation panorama photogrammetry method based on digital camera[J]. Geomatics and Information Science of Wuhan University, 2017, 42(2): 243-249.

梅文胜, 胡帅朋, 李谋思, 等.基于普通数码相机的旋转全景摄影测量方法[J].武汉大学学报·信息科学版, 2017, 42(2): 243-249.[DOI: 10.13203/j.whugis20140994]

Kumar R, Samarasekera S, Hsu SC, et al. Registration of highly-oblique and zoomed in aerial video to reference imagery[C]//Proceedings of the 15th International Conference on Pattern Recognition. Barcelona, Spain: IEEE, 2000.[ DOI:10.1109/ICPR.2000.902919 http://dx.doi.org/10.1109/ICPR.2000.902919 ]

Xu D J, Kasparis T. Video registration to geo-referenced imagery[C]//Proceedings of SPIE 5428, Signal and Data Processing of Small Targets 2004. Orlando, Florida, USA: SPIE, 2004: 569-576.[ DOI:10.1117/12.541734 http://dx.doi.org/10.1117/12.541734 ]

Zhou G.Geo-referencing of video flow from small low-cost civilian UAV[J]. IEEE Transactions on Automation Science&Engineering, 2009, 7(1):156-166 [DOI: 10.1109/TASE.2008.2010948]

Huang X, Zhang Y J, Yang J L, et al. Closed-form solution to space resection based on homography matrix[J]. Journal of Remote Sensing, 2016, 20(3): 431-440.

黄旭, 张永军, 杨璟林, 等.单应性几何下的后方交会直接解法[J].遥感学报, 2016, 20(3): 431-440.[DOI: 10.11834/jrs.20165182]

Zhang H W, Zhang Z X, Zhang J Q. Analysis for the redundancy and relativity of the observation in line photogrammetry[J]. Geomatics and Information Science of Wuhan University, 2005, 30(9): 775-777, 824.

张宏伟, 张祖勋, 张剑清.直线摄影测量观测值的冗余度和相关性分析[J].武汉大学学报·信息科学版, 2005, 30(9): 775-777, 824.[DOI: 10.3969/j.issn.1671-8860.2005.09.006]

Zhang Z X, Zhang J Q. Generalized point photogrammetry and its application[J]. Geomatics and Information Science of Wuhan University, 2005, 30(1): 1-5.

张祖勋, 张剑清.广义点摄影测量及其应用[J].武汉大学学报·信息科学版, 2005, 30(1): 1-5.

Zheng S Y, Guo B Y, Li C L. 3D reconstruction and inspection of cylinder based on geometric model and generalized point photogrammetry[J]. Acta Geodaetica et Cartographica Sinica, 2011, 40(4): 477-482.

郑顺义, 郭宝云, 李彩林.基于模型和广义点摄影测量的圆柱体自动3维重建与检测[J].测绘学报, 2011, 40(4): 477-482.

Wu J. Research on rapidly reconstructing texture for facades in 3D city modeling[J]. Acta Geodaetica et Cartographica Sinica, 2005, 34(4): 317-323.

吴军. 3维城市建模中的建筑墙面纹理快速重建研究[J].测绘学报, 2005, 34(4): 317-323.[DOI: 10.3321/j.issn:1001-1595.2005.04.007]

Liu G, Peng Q S, Bao H J. Texture mapping on real world models from multiple photographic images[J]. Journal of Software, 2005, 16(11): 2014-2020.

刘钢, 彭群生, 鲍虎军.基于多幅实拍照片为真实景物模型添加纹理[J].软件学报2005, 16(11): 2014-2020.

Guo L, Wang J Y, Huang Y Y. On texture mapping for realistic 3D reconstruction[J]. Journal of Image and Graphics, 2018, 12(10): 1881-1884.

郭玲, 王建宇, 黄炎焱.真实感3D重建中的纹理映射技术[J].中国图象图形学报, 2018, 12(10): 1881-1884.[DOI: 10.3969/j.issn.1006-8961.2007.10.045]

Li Q, Dai S L, Zhao Y J, et al. A block LOD real-time rendering algorithm for large scale terrain[J]. Journal of Computer-Aided Design&Computer Graphics, 2013, 25(5): 708-713.

李钦, 戴树岭, 赵永嘉, 等.分块LOD大规模地形实时渲染算法[J].计算机辅助设计与图形学学报, 2013, 25(5): 708-713.

Kim J S, Li K J. Simplification of geometric objects in an indoor space[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2019, 147: 146-162.[DOI: 10.1016/j.isprsjprs.2018.11.017]

Volino M, Hilton A. Layered view-dependent texture maps[C]//Proceedings of the 10th European Conference on Visual Media Production. London, UK: ACM, 2013.

Qu H G. Market status and development direction of network video surveillance[J]. China Security&Protection, 2015, (5): 32-35.

瞿洪桂.网络视频监控的市场现状及发展方向[J].中国安防, 2015, (5): 32-35.[DOI: 10.3969/j.issn.1673-7873.2015.05.007]

Guo H J, Mao Y Q, Shen S B. Implementation mechanism of dual stream network video transport technology[J]. Journal of System Simulation, 2013, 25(11): 2693-2698.

郭洪健, 毛燕琴, 沈苏彬.一种双码流网络视频传送技术的实现机制[J].系统仿真学报, 2013, 25(11): 2693-2698.

Zhong Y Z, Xiang Z, Shen H. Media Streaming and Video Server[M]. Beijing: Tsinghua University Press, 2003.

钟玉琢, 向哲, 沈洪.流媒体和视频服务器[M].北京:清华大学出版社, 2003.

Lee C A, Gasster S D, Plaza A, et al. Recent developments in high performance computing for remote sensing: a review[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2011, 4(3): 508-527.[DOI: 10.1109/JSTARS.2011.2162643]

Hinton G E, Osindero S, Teh Y W. A fast learning algorithm for deep belief nets[J]. Neural Computation, 2006, 18(7): 1527-1554.[DOI: 10.1162/neco.2006.18.7.1527]

Zhang H, Wang K F, Wang F Y. Advances and perspectives on applications of deep learning in visual object detection[J]. Acta Automatica Sinica,2017, 43(8): 1289-1305.

张慧, 王坤峰, 王飞跃.深度学习在目标视觉检测中的应用进展与展望[J].自动化学报, 2017, 43(8): 1289-1305.[DOI: 10.16383/j.aas.2017.c160822]