浏览全部资源
扫码关注微信
收稿：2017-11-10，
修回：2018-1-30，
纸质出版：2018-08-16
移动端阅览
目的
2
针对当前文物资源由传统的实体文物向虚拟展示和数字文物进行扩展的趋势，如何提供一种多模态的信息呈现方式就显得尤为重要。通过将力触觉技术引入3维文物展示领域，提出一种基于多模感知的3维文物交互式呈现的算法框架。在对文物的基本特征进行视、听、触觉多通道分析的基础上，依据用户与文物模型的接触状态对多通道信息进行计算和整合。
方法
2
在力触觉计算渲染方面，基于嵌入深度构建弹簧系统模拟轮廓形状的接触过程，引入动摩擦和静摩擦因数来反映表面摩擦力这一材质特征，通过法线贴图来实现文物表面纹理的触觉处理；针对交互的环境由2维平面拓展至立体空间，结合力触觉设备将操作时的行为和状态映射为虚拟环境中的操作代理，借助操作代理构建"旋转"和"选择-移动-释放"两种基本的操作范式来实现用户意图；最后，物理引擎的引入将物体的基本运动规律集成至虚拟场景，提升场景交互的真实感.
结果
2
使用Phantom Omni手控器搭建面向馆藏文物的多模感知实验系统，抽取志愿者对实验系统进行测评。实验结果表明：运用本文方法，用户可从视觉、听觉、触觉多个通道对数字文物的整体和细节信息进行感知，且交互的整体过程简单、自然、有效。
结论
2
本文提出的基于多模感知的数字文物交互式呈现方法，可有效实现对各类数字遗产特别是3维文物的多模重现，在保证较高实时性的同时拥有良好的可用性和情感体验效果。
Objective
2
Cultural heritage is a powerful witness to the cultural inheritance and development of a country or region. In view of the current trend in which cultural relics extend from traditional physical relics to virtual exhibitions and digital relics
providing a multi-mode presentation method of digital cultural relics is necessary. By introducing computer haptics into the field of cultural relic exhibitions
we propose an algorithm framework for interactive presentation based on the multi-mode perception of 3D artifacts. The proposed framework is based on multi-channel analysis of tactual
auditory
and visual characteristics of cultural relics. Considering the contact states between users and these digital 3D models of relics
the multi-channel information is identified
calculated
integrated
and processed. Eventually
information is passed to the visual
audio
and haptic devices to complete the multi-mode perception.
Method
2
In this study
the impedance control mode is used as the basic driving mechanism of the haptic computing rendering. The basic idea is that a user inputs the position and direction information
and the force and torque are then calculated and fed back to the user. The calculation and rendering for different characteristics of these artifacts
such as contour shapes
physical materials
and surface textures
are also implemented in stages. For the contour shape of these artifacts
the spring system based on the embedding depth is constructed to simulate the contact process
and the stiffness coefficient of the spring is related with the cultural object. To reflect these material characteristics of the surface friction
the friction during the contact of cultural relics is simulated by introducing the dynamic and static friction factors. With the movement of the operation proxy on the surface of cultural relics
the finite state machine is alternately updated between static and dynamic friction. Although discrete triangular patch groups can identify the complex form
deviations remain because the existing models are usually represented by triangular patches. To address this problem
the normal mapping is used to perform the haptic processing of the surface texture. Compared with the traditional 2D interaction
3D space provides more freedom and more abundant interactive tasks. At this time
through the comparison or abstraction of certain mechanisms existing in the real world
the behaviors and states of users' operation can be mapped to a proxy in a virtual environment. Through the operation proxy
we further build unified "rotation" and "select-move-release" interaction models to realize user intention. The "rotation" operation allows users to freely change their perspective according to their own needs
providing multi-view observation and appreciation of cultural relic models. During 3D interaction
the "select-move-release" operation helps users to freely manipulate the artifact model in a virtual environment. This method can also enhance the overall cognition of the artifact model. Finally
to further enhance the realistic characteristics of the virtual environment
ODE physics engine is introduced to cultural relic interaction. The reasonable introduction of physics engine integrates the basic motion law of objects into the virtual environment to enhance the realism of physical movement and scene interaction.
Result
2
With Phantom Omni haptic device as basis
multi-mode perceptual experimental systems for cultural artifacts are built
and 15 volunteers (eight males and seven females) are selected to evaluate the system. Experimental results show that the method can enable human operators to perceive the overall and detailed information of digital relics through multiple channels of vision
hearing
and touch. In addition
the entire interaction is simple
natural
and effective. On the basis of the user experience theory of Rolls-Royce
we evaluate the experiential effects from three dimensions
namely
usability
sensory experience
and emotional experience. Among these dimensions
usability and emotional experience scores are higher
whereas the sensory experience score is relatively lower. With regard to these set problems
users' acceptance and satisfaction degree are relatively high
satisfying the curiosity of people and the attraction of the presentation manner. However
users' ratings in realistic details of cultural models and the naturalness of interaction between the human and artifacts are relatively low. In general
the interactive mode proposed in this study is more natural than the mouse interaction
but it still maintains a certain distance from the human-object interaction under the real environment.
Conclusion
2
This study presents an interactive presentation method of digital artifacts based on multi-modal perception that can effectively achieve the multi-modal reproduction of various types of digital heritage
especially 3D artifacts. While ensuring high real-time performance
the method has good usability and emotional experience. In our future work
we will continue to explore the new surface haptic-rendering algorithm of the cultural relics. The method of haptic interaction for multi-point grasping of virtual artifacts needs to be further studied. Multi-point interaction can provide higher fidelity and richer operation experience than the existing interaction between single point and digital relics.
Chow S K, Chan K L. Reconstruction of photorealistic 3D model of ceramic artefacts for interactive virtual exhibition[J]. Journal of Cultural Heritage, 2009, 10(2):161-173.[DOI:10.1016/j.culher.2008.08.011]
Bruno F, Bruno S, Sensi G D, et al. From 3D reconstruction to virtual reality:a complete methodology for digital archaeological exhibition[J]. Journal of Cultural Heritage, 2010, 11(1):42-49.[DOI:10.1016/j.culher.2009.02.006]
Fernández-Palacios B J, Morabito D, Remondino F. Access to complex reality-based 3D models using virtual reality solutions[J]. Journal of Cultural Heritage, 2017, 23:40-48.[DOI:10.1016/j.culher.2016.09.003]
Liu Y, Lu D M, Diao C Y, et al. Dunhuang 285 cave multimedia integrated virtual exhibit[J]. Journal of Computer-Aided Design&Computer Graphics, 2004, 16(11):528-1534.
刘洋, 鲁东明, 刁常宇, 等.敦煌285窟多媒体集成虚拟展示[J].计算机辅助设计与图形学学报, 2004, 16(11):1528-1534. [DOI:10.3321/j.issn:1003-9775.2004.11.011]
Duan X Y, Liu X L, Liu C X. 3D display technique of virtual Yin ruin's museum[J]. Journal of System Simulation, 2005, 17(9):2187-2190.
段新昱, 刘学莉, 刘晨曦.虚拟殷墟博物苑的3维展示技术[J].系统仿真学报, 2005, 17(9):2187-2190. [DOI:10.3969/j.issn.1004-731X.2005.09.037]
Lin Y, Chen J, Liu Y, et al. User experience design of VR-AR hybrid mobile browsing system based on mental model[J]. Chinese Journal of Computers, 2015, 38(2):408-422.
林一, 陈靖, 刘越, 等.基于心智模型的虚拟现实与增强现实混合式移动导览系统的用户体验设计[J].计算机学报, 2015, 38(2):408-422. [DOI:10.3724/SP.J.1016.2015.00408]
Leyvand T, Meekhof C, Wei Y C, et al. Kinect identity:technology and experience[J]. Computer, 2011, 44(4):94-96.[DOI:10.1109/mc.2011.114]
Ernst M O, Banks M S. Humans integrate visual and haptic information in a statistically optimal fashion[J]. Nature, 2002, 415(6870):429-433.[DOI:10.1038/415429a]
Zhang F J, Dai G Z, Peng X L. A survey on human-computer interaction in virtual reality[J]. Scientia Sinica Informationis, 2016, 46(12):1711-1736.
张凤军, 戴国忠, 彭晓兰.虚拟现实的人机交互综述[J].中国科学:信息科学, 2016, 46(12):1711-1736. [DOI:10.1360/N112016-00252]
Song A G, Tian L, Ni D J, et al. Multi-mode haptic interaction technique and its application[J]. Scientia Sinica Informationis, 2017, 47(9):1183-1197.
宋爱国, 田磊, 倪得晶, 等.多模态力触觉交互技术及应用[J].中国科学:信息科学, 2017, 47(9):1183-1197. [DOI:10.1360/N112017-00081]
Xue F Z, Liu C J, Li N, et al. Open humanoid robot simulator based on ODE[J]. Robot, 2011, 33(1):84-89.
薛方正, 刘成军, 李楠, 等.基于ODE引擎的开放式仿人机器人仿真[J].机器人, 2011, 33(1):84-89. [DOI:10.3724/SP.J.1218.2011.00084]
Buxton B, Hayward V, Pearson I, et al. Big data:the next Google[J]. Nature, 2008, 455:8-9.[DOI:10.1038/455008a]
Wang D X, Jiao J, Zhang Y R, et al. Computer haptics:haptic modeling and rendering in virtual reality environments[J]. Journal of Computer-Aided Design&Computer Graphics, 2016, 28(6):881-895.
王党校, 焦健, 张玉茹, 等.计算机触觉:虚拟现实环境的力触觉建模和生成[J].计算机辅助设计与图形学学报, 2016, 28(6):881-895. [DOI:10.3969/j.issn.1003-9775.2016.06.003]
Sun P F, Bao J S, Jin Y. Haptic perception of 3D vortex structure[J]. Journal of Shanghai Jiaotong University, 2003, 37(1):50-53.
孙鹏飞, 鲍劲松, 金烨. 3维涡流结构的触觉表达[J].上海交通大学学报, 2003, 37(1):50-53. [DOI:10.3321/j.issn:1006-2467.2003.01.014]
Zhang Y R, Li C B, Wang D X, et al. Task oriented method for designing haptic devices[J]. Journal of Mechanical Engineering, 2015, 51(13):212-217, 226.
张玉茹, 李朝斌, 王党校, 等.面向任务的力觉交互设备设计[J].机械工程学报, 2015, 51(13):212-217, 226. [DOI:10.3901/JME.2015.13.212.]
Zilles C B, Salisbury J K. A constraint-based god-object method for haptic display[C ] //Proceedings of 1995 IEEE/RSJ International Conference on Intelligent Robots and Systems. Human Robot Interaction and Cooperative Robots. Pittsburgh, PA: IEEE, 1995, 3: 146-151. [ DOI: 10.1109/iros.1995.525876 http://dx.doi.org/10.1109/iros.1995.525876 ]
Ruspini D C, Kolarov K, Khatib O. The haptic display of complex graphical environments[C ] //Proceedings of the 24th Annual Conference on Computer Graphics and Interactive Techniques. New York: ACM, 1997: 345-352. [ DOI: 10.1145/258734.258878 http://dx.doi.org/10.1145/258734.258878 ]
Chen H, Sun H Q. Haptic sculpturing in the virtual volume space[J]. Chinese Journal of Computers, 2002, 25(9):994-1000.
陈辉, 孙汉秋.虚拟体空间中的触觉雕刻[J].计算机学报, 2002, 25(9):994-1000. [DOI:10.3321/j.issn:0254-4164.2002.09.015]
Wang D X, Zhang Y R, Wang Y, et al. Haptic rendering for dental training system[J]. Science in China Series F:Information Sciences, 2009, 52(3):529-546.
王党校, 张玉茹, 王勇, 等.面向牙科手术培训的力觉合成技术[J].中国科学F辑:信息科学, 2009, 39(1):159-174. [DOI:10.1360/zf2009-39-1-159]
Wang L, Wan H G, Peng Q S. Computation and haptic application of protein interaction fields based on CHARMM force field[J]. Journal of Computer-Aided Design&Computer Graphics, 2009, 21(7):886-892.
汪莉, 万华根, 彭群生.基于CHARMM力场的蛋白质分子场计算及触觉感知[J].计算机辅助设计与图形学学报, 2009, 21(7):886-892.
Yang G Q, Hou Z X, Guo C, et al. The simulation of brush stroke based on force feedback technology[J]. Journal of Computer-Aided Design&Computer Graphics, 2016, 28(5):849-854.
杨广卿, 侯增选, 郭超, 等.基于力反馈技术的毛笔笔道仿真[J].计算机辅助设计与图形学学报, 2016, 28(5):849-854. [DOI:10.3969/j.issn.1003-9775.2016.05.019]
Qiu Z W, Zhang T W. Key techniques on cultural relic 3D reconstruction[J]. Acta Electronica Sinica, 2008, 36(12):2423-2427.
邱兆文, 张田文.文物3维重建关键技术[J].电子学报, 2008, 36(12):2423-2427. [DOI:10.3321/j.issn:0372-2112.2008.12.028]
Hsu W H, Ma K L, Correa C. A rendering framework for multiscale views of 3D models[J]. ACM Transactions on Graphics, 2011, 30(6):#131.[DOI:10.1145/2070781.2024165]
Chen T D, Jin W W, Chen Y D, et al. Contour optimization of multi-channel 3D Chinese ink rendering model[J]. Journal of Electronics&Information Technology, 2015, 37(2):494-498.
陈添丁, 金炜炜, 陈英旦, 等.轮廓线优化的多通道3维水墨渲染模型[J].电子与信息学报, 2015, 37(2):494-498. [DOI:10.11999/JEIT140434]
Suarez J, Belhadj F, Boyer V. Real-time 3D rendering with hatching[J]. The Visual Computer, 2017, 33(10):1319-1334.[DOI:10.1007/s00371-016-1222-3]
Schüssler V, Heitz E, Hanika J, et al. Microfacet-based normal mapping for robust Monte Carlo path tracing[J]. ACM Transactions on Graphics, 2017, 36(6):#205.[DOI:10.1145/3130800.3130806]
Zhu X J, Guo G L, Zha J F. Reconstruction of 3D laser scanning data based on normal map[J]. Geography and Geo-Information Science, 2012, 28(6):35-38.
朱晓峻, 郭广礼, 查剑锋.基于法线贴图的3维激光扫描数据模型重建[J].地理与地理信息科学, 2012, 28(6):35-38.
Feng Z Q, Yang B, Xu T, et al. 3D direct human-computer interface paradigm based on free hand tracking[J]. Chinese Journal of Computers, 2014, 37(6):1309-1323.
冯志全, 杨波, 徐涛, 等.基于自然手势跟踪的直接操作型3维人机交互范式[J].计算机学报, 2014, 37(6):1309-1323. [DOI:10.3724/SP.J.1016.2014.01309]
Sun J, Content A, Sun P. Studying numerical representation by Weber-Fechner Law[J]. Psychological Research, 2017, 10(5):35-39.
孙霁, Content A, 孙沛.数量表征和韦伯-费希纳定律:应用及发展[J].心理研究, 2017, 10(5):35-39. [DOI:10.3969/j.issn.2095-1159.2017.05.005]
Rouse W B. Design for Success:A Human-Centered Approach to Designing Successful Products and Systems[M]. New York:Wiley, 1991:270.
相关作者
相关机构
京公网安备11010802024621
