Current Issue Cover
视觉传感成像技术与数据处理进展

王程1, 陈峰2, 汶德胜3, 雷浩3, 宋宗玺3, 赵航芳4(1.福建省智慧城市感知与计算重点实验室(厦门大学), 厦门 361005;2.厦门理工学院计算机与信息工程学院, 厦门 361024;3.中国科学院西安光学精密机械研究所, 西安 710119;4.浙江大学信息与电子工程学院, 杭州 310058)

摘 要
本文以视觉传感的新视觉传感硬件、处理技术和应用场景为主线,通过综合国内外文献和相关报道来梳理该领域在成像技术和数据处理方面的主要进展。从激光扫描成像、大动态范围光学成像技术、偏振成像与传感技术和海洋声学层析成像等研究方向,重点论述视觉传感领域的发展现状、前沿动态、热点问题和趋势。基于激光扫描的3维建模技术虽然取得了一些进展,但仍面临居多挑战。随着硬件设备和数据处理技术的发展,未来激光扫描系统将在众多民用领域得到广泛应用,满足不同的探测和建模任务;大动态范围光学成像相关技术已逐步应用于红外成像、光谱成像、偏振成像、超声成像和单光子成像等领域,将为多维信息获取、智能处理以及数据挖掘等提供有力支撑;充分挖掘偏振成像的应用潜能,与其他先进成像传感技术相结合,实现更优性能,对各个尺度下的成像场景都具有重要的应用价值;海洋声学层析成像需要与其他方法相结合,发展基于分布式水下传感网络、卫星观测、海底电缆、人工与自然噪声机会声源等联合观测的低成本、长期观测网络。对国内外视觉传感领域进展情况进行梳理、总结,有助于发现该领域的发展趋势以及明确下一步的研究方向。
关键词
Review on imaging and data processing of visual sensing

Wang Cheng1, Chen Feng2, Wen Desheng3, Lei Hao3, Song Zongxi3, Zhao Hangfang4(1.Fujian Key Laboratory of Sensing and Computing for Smart Cities, Xiamen University, Xiamen 361005, China;2.College of Computer and Information Engineering, Xiamen University of Technology, Xiamen 361024, China;3.Xi'an Institute of Optics and Precision Mechanics, Chinese Academy of Sciences, Xi'an 710119, China;4.College of Information Science and Electronic Engineering, Zhejiang University, Hangzhou 310058, China)

Abstract
Recently, significant developments of visual sensing have been observed in imaging technology and data processing, thereby providing great opportunities to enhance our ability to perceive and recognize our real world. Therefore, investigations on visual sensing possess important theoretical value and are required for application needs. Surveying the progress to understand the trend in the field of visual sensing and to clarify the future research direction is beneficial. The reviews are generated mainly based on analyzing peer-reviewed academic publications and related reports. A general description on the states of the art and trends about the visual sensing is provided, mainly including laser scanning, high dynamic range (HDR) imaging, polarization imaging, and ocean acoustic tomography. Specifically, for each of these imaging fields, parts discussed include new hardware, processing technology, and application scenarios. Processing of 3D point cloud data has become more effective along with the great progresses in deep learning and the advancement of hardware devices. Meanwhile, applications of 3D point cloud data are increasingly popular for diverse purposes. Over past several years, many domestic institutions and teams focused on developing algorithms for 3D point cloud data processing, such as in feature extraction, semantic labeling and segmentation, and object detection. In particular, several teams have conducted a number of substantive work in the production and sharing of standard data sets, which promote and improve the processing ability and application level of point cloud data. However, at present, the commercial hardware still has some deficiencies. Combining 3D point cloud data with observation from other sensors is a valuable but challenging task. Nevertheless, the laser scanning system is expected to be widely used in transportation, civil engineering, forestry, agriculture, and other civil fields in the future to satisfy different detection and modeling tasks. At the same time, with ongoing advancements in laser scanning equipment, it also plays an important role in understanding natural sciences, such as archaeology and geoscience. High dynamic range imaging is a hot research field in digital image acquisition, processing, display, and applications. Currently, researchers mostly focus on multiple exposure, different modulation methods, and multi detector methods in the HDR imaging. For example, through nonlinear response and multiple exposure imaging, the dynamic range can reach approximately 140 dB, and it can reach approximately 160 dB by using multi detector imaging. Using deep learning directly in HDR image mapping, instead of using traditional methodology, such as optical flow method and the combination of optical flow and neural network, has become a distinguished characteristic. Deep learning neural network has also been gradually applied to single exposure HDR reconstruction and tone mapping. Many domestic research teams have investigated the issues for the combination of deep learning neural network and HDR imaging. As expected, advancements in deep neural network provide a good opportunity for processing HDR imaging, such as in image fusion. With potential advancements in new detector materials, detector design, semiconductor equipment, and technology towards nanotechnology, new detectors with 10 megapixel resolution and dynamic range better than 160 dB will be available and will greatly improve the sensitivity under low illumination. Important fields urgently need breakthrough, including HDR imaging of dynamic scene and acquisition, processing and display of color HDR imaging with large dynamics, and wide color gamut. Compared with the progresses in polarization imaging made by other countries (e.g., the United States of America, Canada, and Japan), the systematicness and practicability in DoFP CMOS chip research domestically still need to be improved. In practice, the domestic institutions have made continuous achievements in many polarization imaging issues, including mosaic removal, polarization defogging, underwater polarization, polarization 3D imaging, imaging polarization spectral remote sensing, airborne polarization imaging, marine environment spectral polarization imaging, and spatial polarization detection. Furthermore, integrated optical detection of land, sea, air, and space is a critical demand, which promotes the rapid development of polarization imaging and sensing. In multisource data fusion, many methods and technologies showed excellent performance in their respective applications correspondingly, including multidimensional data acquisition and intelligent processing of “polarization +”, polarization + infrared, polarization + spectrum, polarization + TOF, polarization structured light, fluorescence polarization imaging (FPI), polarization-sensitive optical coherence tomography (PS-OCT), polarization-dependent optical second harmonic imaging, and polarization confocal microscopy imaging. In ocean acoustic tomography, institutions from the United States of America published the largest number of papers, showing distinguished trends from other countries. At the same time, as a country with the largest number of published patents, Japan shows great importance to ocean acoustic tomography and has certain advantages in technological innovation. Compared with institutions from the United States of America and Japan, institutions from China have published relatively a small number of papers and patents in ocean acoustic tomography. With more than 40 years of its development, great progresses were made in theory and technology. However, the application of ocean acoustic tomography still faces the bottleneck of high cost of sea trial, which is also impossibly used as an observation means alone. In conclusion, 1) the 3D modeling based on laser scanning still faces many challenges, although progresses have been made recently. With the development of hardware and progress in data processing, laser scanning system benefits many civil fields in the future to satisfy different detection and modeling tasks; 2) high dynamic range optical imaging technology has been gradually applied to many fields, mainly including infrared imaging, spectral imaging, polarization imaging, ultrasonic imaging, and single photon imaging, which are valuable for multidimensional information acquisition, intelligent processing, and data mining; 3) fully exploiting the potential of polarization imaging has great value. Furthermore, to achieve its better performance, the combination with other advanced imaging sensing technologies is necessary. 4) Marine acoustic tomography needs to be combined with other means to develop a low-cost, long-term observation network, which is based on distributed underwater sensor networks, satellite observations, submarine cables, as well as using artificial and natural noise as sound source of opportunity.
Keywords

订阅号|日报