浏览全部资源
扫码关注微信
Published： 16 October 2024 ，
移动端阅览
时佳豪， 孙士友， 杨芳雁， 陈建. 2024. 超声波空中触觉图形呈现的时空调制方法. 中国图象图形学报， 29(10):3022-3032
Shi Jiahao， Sun Shiyou， Yang Fangyan， Chen Jian. 2024. Spatiotemporal modulation method for ultrasonic airborne haptic graphic presentation. Journal of Image and Graphics， 29(10):3022-3032
目的
2
超声波空中触觉反馈技术为虚拟现实、混合现实提供非接触、无约束的触觉体验，是混合现实领域触觉呈现的主要途径。使用传统调制方法对多点图形进行聚焦呈现时，在每个调制周期内只进行单点聚焦，发射阵列的使用率较低，聚焦过程中产生噪声较大。针对现有调制方法的不足之处，提出了一种新型时空调制方法来提高阵列利用率并降低聚焦时产生的噪声。
方法
2
首先，根据所需呈现图形获取多个焦点的位置数据，计算每个焦点聚焦时发射器所需延时时间并进行储存；其次，将调制信号的周期按聚焦点个数进行平均分配，并生成超声发射器驱动信号；最后，将驱动信号发送至延时模块，并在调制信号周期的不同时间片内，将各聚焦点的延时数据送入，实现多点聚焦触觉同步的效果。
结果
2
通过实验对新型调制方法进行测试，在进行占空比为10%和20%的两点聚焦时，相比于传统调制方法，调制噪声分别降低了8.4%和13%，聚焦功耗提高了80%和86%；在进行占空比为10%和20%的四点聚焦时，相比于传统调制方法，调制噪声降低了6.3%和10.1%，聚焦功耗分别提高了60%和100%。在主观图形识别实验中，三角形、矩形、圆形的识别率分别提升了25%、 19%、35%。实验结果表明新型调制方法降低了聚焦噪声，提高了阵列利用率以及对呈现图形的识别率。
结论
2
本文所提出的新型时空调制方法有效减小了聚焦噪声，提供了更好的触觉图形反馈效果。
Objective
2
With the continuous development of computer vision， human-computer interaction， virtual reality， augmented reality， the visual and auditory interaction field has reached a commercialization level. Haptic interaction enables users to perceive the tactile characteristics of virtual graphics， and the integration with visual and audio makes virtual graphics have more physical attributes and enhances the user’s immersive experience. For example， purchasers can realistically feel the texture of online materials， drive a car without having to shift their eyes to distract themselves from some screen operations， shop at home， and experience different products， and doctors can remotely perform surgical simulations and planning. Furthermore， science and technology to the good， such as the blind， deaf， and dumb and other special groups through the sense of touch， can also fully enjoy the “era of map reading” to avoid not seeing and hearing is not capable of using the poor situation. The extensive use of haptic feedback technology brings the user a sense of immersion， greatly enhancing the sense of immersion and the sense of reality， virtual reality， augmented reality， mixed reality， and other areas of future research focus direction. Ultrasonic airborne haptic feedback technology provides noncontact and unconstrained haptic experience for virtual reality and mixed reality and is the main way of haptic presentation in the field of mixed reality. The current ultrasonic haptic feedback devices generally use the method of rapidly and repeatedly moving the ultrasonic focus on a given path when performing multipoint focusing to present an ideal pattern， and only single point focusing is performed in one modulation cycle， with low efficiency of array usage and large noise generated during the focusing process， which affects the user’s sense of experience. To solve the above problems， a novel ultrasonic haptic multipoint synchronous focusing method based on spatiotemporal modulation is proposed in this study； it can generate multiple focal points in each modulation period， reduce modulation noise， and enhance haptic synchronization effect while eliminating the directivity between focal points.
Method
2
First， data are received for a plurality of focal points that must be focused， and the focusing information is sent to a calculation module via the top-level module. After receiving the coordinates of the focal points sent by the top-level module， the calculation module calculates the distance S from each ultrasound emitter to the focal point. Given that the ultrasonic emitters used are not omnidirectional， the ultrasonic emission angle has a limit. The maximum effective emission angle should be considered for calculating the path. A comparison can be made by calculating the longest emission distance L within the emission range. To achieve the focus of each transmitter transmitting signal at each focal point， the delay time of each transmitter driving signal must be controlled so that each signal is transmitted at different times， with the farthest emitter emitting ultrasonic waves first and the nearest emitter emitting ultrasonic waves last， thus ensuring that the ultrasonic waves emitted by each transmitter arrive at the same focus at the same time. The time delay required to focus each focal point is calculated and stored. Second， a modulation frequ
ency
f
0
that can be perceived by the human body is chosen， and all focal points are emitted within the same modulation period
T
0
. Under the assumption that there are
I
focal points， the modulation period
T
0
is divided into
I
equal parts of the time slice （
T
1
~
T
I
）， and each focal point occupies a modulation period of
T
0
/
I
， that is， each focusing time is
T
0
/
I
. The required modulation frequency signal and rendering signal are generated， and the cycle time of the modulation signals is equally distributed according to the number of focusing points to obtain the final ultrasonic transmitter driving signal. Finally， the drive signal is sent to the delay module， which sends the delay data of the previous different focus points in different time slices of the modulation signal cycle to achieve the effect of multipoint focusing haptic synchronization.
Result
2
Testing of the new modulation method through experiments showed that compared with traditional modulation methods， the modulation noise decreased by 8.4% and 13% when focusing on two points with duty cycles of 10% and 20%， respectively. However， the focusing power consumption increased by 80% and 86%. When focusing on four points with duty cycles of 10% and 20%， the new modulation method reduced modulation noise by 6.3% and 10.1%， while the focusing power consumption increased by 60% and 100%， respectively. In subjective graphic recognition experiments， the recognition rates for triangles， rectangles， and circles increased by 25%， 19%， and 35%， respectively. Comparative experimental results demonstrate that the new modulation method reduces focusing noise， enhances array utilization， and improves the recognition rates of presented graphics.
Conclusion
2
In this study， we present a novel spatiotemporal modulation method for ultrasonic focused haptics. Experimental results show that this method can considerably reduce noise focusing noise and provide stronger feedback force.
图形呈现时空调制超声波反馈空中触觉多点触觉
graphic presentationspatiotemporal modulationultrasonic feedbackaerial tactilemultitouch
Breitschaft S J， Pastukhov A and Carbon C C. 2022. Where’s my button？ Evaluating the user experience of surface haptics in featureless automotive user interfaces. IEEE Transactions on Haptics， 15（2）： 292-303 ［DOI： 10.1109/TOH.2021.3131058http://dx.doi.org/10.1109/TOH.2021.3131058］
Chen J， Yu F， Wang Z Q and Lin L. 2020. Multichannel ultrasound focusing delay control method based on variable-length shift register for airborne ultrasound tactile feedback. IEEE Access， 8： 24904-24913 ［DOI： 10.1109/ACCESS.2020.2970451http://dx.doi.org/10.1109/ACCESS.2020.2970451］
Frier W， Ablart D， Chilles J， Long B， Giordano M， Obrist M and Subramanian S. 2018. Using spatiotemporal modulation to draw tactile patterns in mid-air//Proceedings of the 11th International Conference on Haptics： Science， Technology， and Applications. Pisa， Italy： Springer： 270-281 ［DOI： 10.1007/978-3-319-93445-7_24http://dx.doi.org/10.1007/978-3-319-93445-7_24］
Hasegawa K and Shinoda H. 2018. Aerial vibrotactile display based on multiunit ultrasound phased array. IEEE Transactions on Haptics， 11（3）： 367-377 ［DOI： 10.1109/TOH.2018.2799220http://dx.doi.org/10.1109/TOH.2018.2799220］
Hoshi T， Iwamoto T and Shinoda H. 2009. Non-contact tactile sensation synthesized by ultrasound transducers//Proceedings of the 3rd Joint EuroHaptics Conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems. Salt Lake City， USA： IEEE： 256-260 ［DOI： 10.1109/WHC.2009.4810900http://dx.doi.org/10.1109/WHC.2009.4810900］
Howard T， Gallagher G， Lécuyer A， Pacchierotti C and Marchal M. 2019. Investigating the recognition of local shapes using mid-air ultrasound haptics//Proceedings of 2019 IEEE World Haptics Conference. Tokyo， Japan： IEEE： 503-508 ［DOI： 10.1109/WHC.2019.8816127http://dx.doi.org/10.1109/WHC.2019.8816127］
Inoue S， Makino Y and Shinoda H. 2015. Active touch perception produced by airborne ultrasonic haptic hologram//Proceedings of 2015 IEEE World Haptics Conference. Evanston， USA： IEEE： 362-367 ［DOI： 10.1109/WHC.2015.7177739http://dx.doi.org/10.1109/WHC.2015.7177739］
Iwamoto T， Tatezono M and Shinoda H. 2008. Non-contact method for producing tactile sensation using airborne ultrasound//Proceedings of the 6th International Conference on Haptics： Perception， Devices and Scenarios. Madrid， Spain： Springer： 504-513 ［DOI： 10.1007/978-3-540-69057-3_64http://dx.doi.org/10.1007/978-3-540-69057-3_64］
Malladi N S S V， Paneva V and Müller J. 2023. Feel the breeze： promoting relaxation in virtual reality using mid-air haptics//Proceedings of 2023 IEEE International Symposium on Mixed and Augmented Reality Adjunct. Sydney， Australia： IEEE： 322-327 ［DOI： 10.1109/ISMAR-Adjunct60411.2023.00070http://dx.doi.org/10.1109/ISMAR-Adjunct60411.2023.00070］
Morisaki T， Fujiwara M， Makino Y and Shinoda H. 2023. Noncontact haptic rendering of static contact with convex surface using circular movement of ultrasound focus on a finger pad. IEEE Transactions on Haptics ［DOI： 10.1109/TOH.2023.3338230http://dx.doi.org/10.1109/TOH.2023.3338230］
O’Hara K， Gonzalez G， Sellen A， Penney G， Varnavas A， Mentis H， Criminisi A， Corish R， Rouncefield M， Dastur N and Carrell T. 2014. Touchless interaction in surgery. Communications of the ACM， 57（1）： 70-77 ［DOI： 10.1145/2541883.2541899http://dx.doi.org/10.1145/2541883.2541899］
Pitts M J， Williams M A， Wellings T and Attridge A. 2009. Assessing subjective response to haptic feedback in automotive touchscreens//Proceedings of the 1st International Conference on Automotive User Interfaces and Interactive Vehicular Applications. Essen， Germany： ACM： 11-18 ［DOI： 10.1145/1620509.1620512http://dx.doi.org/10.1145/1620509.1620512］
Ribeiro M L， Lederman H M， Elias S and Nunes F L S. 2017. Techniques and devices used in palpation simulation with haptic feedback. ACM Computing Surveys， 49（3）： #48 ［DOI： 10.1145/2962723http://dx.doi.org/10.1145/2962723］
Sand A， Rakkolainen I， Isokoski P， Kangas J， Raisamo R and Palovuori K. 2015. Head-mounted display with mid-air tactile feedback//Proceedings of the 21st ACM Symposium on Virtual Reality Software and Technology. Beijing， China： ACM： 51-58 ［DOI： 10.1145/2821592.2821593http://dx.doi.org/10.1145/2821592.2821593］
Shen V， Shultz C and Harrison C. 2022. Mouth haptics in VR using a headset ultrasound phased array//Proceedings of 2022 CHI Conference on Human Factors in Computing Systems. New Orleans， USA： ACM： #275 ［DOI： 10.1145/3491102.3501960http://dx.doi.org/10.1145/3491102.3501960］
Takahashi R， Hasegawa K and Shinoda H. 2020. Tactile stimulation by repetitive lateral movement of midair ultrasound focus. IEEE Transactions on Haptics， 13（2）： 334-342 ［DOI： 10.1109/TOH.2019.2946136http://dx.doi.org/10.1109/TOH.2019.2946136］
Trinitatova D and Tsetserukou D. 2023. Study of the effectiveness of a wearable haptic interface with cutaneous and vibrotactile feedback for VR-based teleoperation. IEEE Transactions on Haptics， 16（4）： 463-469 ［DOI： 10.1109/TOH.2023.3265713http://dx.doi.org/10.1109/TOH.2023.3265713］
Väänänen-Vainio-Mattila K， Heikkinen J， Farooq A， Evreinov G， Mäkinen E and Raisamo R. 2014. User experience and expectations of haptic feedback in in-car interaction//Proceedings of the 13th International Conference on Mobile and Ubiquitous Multimedia. Melbourne， Australia： ACM： 248-251 ［DOI： 10.1145/2677972.2677996http://dx.doi.org/10.1145/2677972.2677996］
Wu J， Song Z Z， Wu W X， Li L Y and Song A G. 2011. Design of image vibrotactile display system for the blind. Journal of Southeast University （Natural Science Edition）， 41（5）： 973-977
吴涓， 宋振中， 吴伟雄， 李莅圆， 宋爱国. 2011. 用于盲人的图像振动触觉显示系统设计. 东南大学学报（自然科学版）， 41（5）： 973-977 ［DOI： 10.3969/j.issn.1001-0505.2011.05.015http://dx.doi.org/10.3969/j.issn.1001-0505.2011.05.015］
Yuan Y Y， Long Z L， Jiang Z T and Li C. 2019. Simulation for multi-point haptic feedback based on ultrasound phased array//Proceedings of 2019 IEEE International Conference on Robotics and Biomimetics. Dali， China： IEEE： 2445-2449 ［DOI： 10.1109/ROBIO49542.2019.8961576http://dx.doi.org/10.1109/ROBIO49542.2019.8961576］
Zenner A， Ullmann K and Krüger A. 2021. Combining dynamic passive haptics and haptic retargeting for enhanced haptic feedback in virtual reality. IEEE Transactions on Visualization and Computer Graphics， 27（5）： 2627-2637 ［DOI： 10.1109/TVCG.2021.3067777http://dx.doi.org/10.1109/TVCG.2021.3067777］
相关文章
相关作者
相关机构
京公网安备11010802024621