浏览全部资源
扫码关注微信
收稿：2020-12-29，
修回：2021-2-18，
录用：2021-2-25，
纸质出版：2021-06-16
移动端阅览
随着现代科学技术的快速发展，立体显示技术为临床医生的双眼视觉功能和临床应用场景，提供了现实模拟度更高的载体，并成为当前计算机视觉和临床医学领域共同研究的热点。在微创手术术前，与传统的平面显示技术相比，沉浸式立体显示技术能够提供更生动、准确的3维人体生理和病理影像，使医生更易于判断病变的层次、形状和血管等复杂结构及解剖关系；同时虚拟现实能够为医学培训及手术预演提供沉浸式的手术情境模拟，帮助医生高效地掌握手术技巧，提高医学术前诊断效率，从而进一步降低手术风险。在微创手术术中，基于增强现实的三维成像导航技术，能够将微创手术过程立体、直观地展现在医生面前，使术区各组织及其与手术器械间的位置关系和距离更加容易判断，同时通过叠加相同区域的术前检查影像，为手术提供实时的路径导航，实现精准微创手术。此外，在临床医疗资源共享中占据重要比重的远程诊疗领域，立体显示技术能够为远程诊断、线上会诊以及机器人手术等提供更为精确的深度信息，以及更多维度的图像信息，使医学数据的远程显示结果更具有真实性和实用性。现阶段立体显示技术在临床医学领域中也存在显示模式转换不舒适、三维重建图像信息缺失以及立体显示软、硬件系统带来的视觉疲劳等问题，但该技术在医学领域已经展露头角，在未来的临床医学进步中会成为不可或缺的一部分。本文详细分析了沉浸式立体显示技术在临床医学中的代表性应用，介绍了微创外科手术以及远程诊疗领域国内外的研究现状，从影像诊断、手术训练、规划与导航、治疗和教育培训4个方面，总结了立体显示技术在临床医学领域中的研究进展。
With the development of modern imaging 3D reconstruction
haptic interaction and three-dimensional (3D) printing technology
the traditional concept of tumor surgery is undergoing unprecedented revolutionary changes. The tumor treatment is not limited to traditional open surgery
large-area radiotherapy
chemotherapy
and other extensive treatment methods. The popularization of individualized precision surgery planning means that "precision medicine" is gradually applied to clinical surgery
and "precision medicine" mode will also make the treatment of tumor enter a new research area. The physiological anatomy and pathological changes of various organs of the human body are presented as a three-dimensional shape
while medical teaching images
diagnostic medical images
and various endoscopic surgery images are mostly plane images. However
how to interpret them often depends on the professional experience of doctors. This plane display mode limits the effect of medical training
the accuracy of interpretation of diagnostic images and the efficiency of surgical operation. Modern science and technology
stereo display technology provides a carrier with a higher realistic simulation degree for binocular vision function and training and has become a joint research hotspot in the field of computer vision and clinical medicine. Before minimally invasive surgery (MIS)
compared with the traditional planar display technology
the immersive stereoscopic display technology can provide more vivid and accurate 3D human physiological and pathological images
making it easier for doctors to judge the layers
shapes
blood vessels and other complex structures and anatomical relations of lesions. At the same time
it can also provide immersive surgical situation simulation for medical training
help doctors quickly master surgical skills
further improve the efficiency of medical diagnosis and reduce the risk of surgery. In the process of minimally invasive surgery
the three-dimensional imaging navigation technology for augmented reality can not only make it easier for doctors to judge the position relationship and distance between the tissues in the surgical area and the surgical instruments
but also provide surgical navigation by superposing the preoperative examination images of the same location. Which is also achieved the accurate minimally invasive surgery. Besides
in the field of remote diagnosis and treatment
which occupies a significant proportion in the sharing of clinical medical resources
stereo display technology can provide the more accurate depth information and more dimensional image information for remote diagnosis
online consultation
and robotic surgery
so that the remote display results of medical data are more authentic and practical. Although at the present stage
the unadaptability of display mode conversion
partial information loss of 3D reconstruction images. Furthermore
the visual fatigue of users is still a problem that needs to be overcome in the field of clinical medicine. The three-dimensional display technology still has a broad space for development in medical science and is a new driving force to promote medical progress in the future. This article comprehensively analyzes the stereo display technology in clinical medicine; the application of minimally invasive surgery is summarized. The research status at home and abroad in remote diagnosis and treatment
from the image diagnosis and surgery training
planning and navigation
the training and education from four aspects
summarizes the stereo display technology research progress in the field of clinical medicine. This article fully analyzes the stereo display technology in the application of clinical medicine
provides an introduction to the field of minimally invasive surgery and remote diagnosis and research status at home and abroad
from the image diagnosis and surgery training
planning and navigation
the training and education from four aspects
summarizes the stereo display technology research progress in the field of clinical medicine. Clinical medicine in the future
with the aid of a 3D image technology
mixed reality technology
interactive technology
and a series of cutting-edge computer technology
can be more intuitive
accurately make the mixed reality device presents visualization of three-dimensional graphics
and provide a series of digital simulation tools
the surgeon can be based on the data communication with others
better-making operation plan. The extended reality technology can be used in the teaching of medical anatomy so that medical students can more intuitively understand the structure of human organs and their spatial position relationship with adjacent organs and tissues. Simultaneously
as an essential achievement to expand the combination of fundamental technology and medical research
virtual surgery is also one of the hot issues in medical research.
Basdogan C, De S, Kim J, Muniyandi M, Kim H and Srinivasan M A. 2004. Haptics in minimally invasive surgical simulation and training. IEEE Computer Graphics and Applications, 24(2): 56-64[DOI:10.1109/MCG.2004.1274062]
Chen H. 2006. Application of computer medical science virtual surgery in the plastic surgery. Chinese Journal of Aesthetic Medicine, 15(5): 600-602
陈华. 2006. 计算机医学手术仿真技术在整形外科领域中的应用. 中国美容医学, 15(5): 600-602[DOI:10.3969/j.issn.1008-6455.2006.05.058]
Cheng Q Q, Liu P X, Lai P H, Xu S P and Zou Y N. 2018. A novel haptic interactive approach to simulation of surgery cutting based on mesh and meshless models. Journal of healthcare engineering: #9204949
Coles T R, Meglan D and John N W. 2011. The role of haptics in medical training simulators: a survey of the state of the art. IEEE Transactions on Haptics, 4(1): 51-66[DOI:10.1109/TOH.2010.19]
Cutolo F, Meola A, Carbone M, Sinceri S, Cagnazzo F, Denaro E, Esposito N, Ferrari M and Ferrari V. 2017. A new head-mounted display-based augmented reality system in neurosurgical oncology: a study on phantom. Computer Assisted Surgery, 22(1): 39-53[DOI:10.1080/24699322.2017.1358400]
de Paolis L T. 2012. Serious game for laparoscopic suturing training//Proceedings of the 6th International Conference on Complex, Intelligent, and Software Intensive Systems. Palermo, Italy: IEEE: 481-485[ DOI: 10.1109/CISIS.2012.175 http://dx.doi.org/10.1109/CISIS.2012.175 ]
Ding Z Z. 2006. Research on Minimally Invasive Surgery System Based on Virtual Reality. Tianjin: Tianjin University
丁再珍. 2006. 基于虚拟现实技术的微创手术系统研究. 天津: 天津大学
Escobar-Castillejos D, Noguez J, Neri L, Magana A and Benes B. 2016. A review of simulators with haptic devices for medical training. Journal of Medical Systems, 40(4): #104[DOI:10.1007/s10916-016-0459-8]
Feng M, Fu Y L, Pan B and Chang L. 2012. Development of a medical robot system for minimally invasive surgery. International Journal of Medical Robotics + Computer Assisted Surgery: MRCAS, 8(1): 85-96[DOI:10.1002/rcs.440]
Fried G M, Feldman L S, Vassiliou M C, Fraser S A, Stanbridge D, Ghitulescu G and Andrew C G. 2004. Proving the value of simulation in laparoscopic surgery. Annals of Surgery, 240(3): 518-528[DOI:10.1097/01.sla.0000136941.46529.56]
Fu Y Y, Cavuoto L, Qi D, Panneerselvam K, Arikatla V S, Enquobahrie A, De S and Schwaitzberg S D. 2020. Characterizing the learning curve of a virtual intracorporeal suturing simulator VBLaST-SS©. Surgical Endoscopy, 34(7): 3135-3144[DOI:10.1007/s00464-019-07081-6]
Gerard I J, Kersten-Oertel M, Drouin S, Hall J A, Petrecca K, De Nigris D, Giovanni D A D, Arbel T and Collins D L. 2018. Combining intraoperative ultrasound brain shift correction and augmented reality visualizations: a pilot study of eight cases. Journal of Medical Imaging, 5(2): #021210[DOI:10.1117/1.jmi.5.2.021210]
Halic T and De S. 2010. Lightweight bleeding and smoke effect for surgical simulators//Proceedings of 2010 IEEE Virtual Reality Conference. Boston, USA: IEEE: 271-272[ DOI: 10.1109/VR.2010.5444771 http://dx.doi.org/10.1109/VR.2010.5444771 ]
Hou Y Z, Ma L C, Zhu R Y, Chen X L and Zhang J. 2016. A low-cost iPhone-assisted augmented reality solution for the localization of intracranial lesions. PLoS One, 11(7): #0159185[DOI:10.1371/journal.pone.0159185]
Huber T, Paschold M, Hansen C, Lang H and Kneist W. 2018. Artificial versus video-based immersive virtual surroundings: analysis of performance and user's preference. Surgical Innovation, 25(3): 280-285[DOI:10.1177/1553350618761756]
Jensen K, Bjerrum F, Hansen H J, Petersen R H, Pedersen J H and Konge L. 2017. Using virtual reality simulation to assess competence in video-assisted thoracoscopic surgery (VATS) lobectomy. Surgical Endoscopy, 31(6): 2520-2528[DOI:10.1007/s00464-016-5254-6]
Jiang D, Hovdebo J, Cabral A, Mora V and Delorme S. 2013. Endoscopic third ventriculostomy on a microneurosurgery simulator. Simulation, 89(12): 1442-1449[DOI:10.1177/0037549713491519]
Khan Z A, Kamal N, Hameed A, Mahmood A, Zainab R, Sadia B, Mansoor S B and Hasan O. 2017. SmartSIM-a virtual reality simulator for laparoscopy training using a generic physics engine. The International Journal of Medical Robotics and Computer Assisted Surgery, 13(3): #1771[DOI:10.1002/rcs.1771]
Li Y, 2017. Research on Image-Guided Technology for Percutaneous Puncture of Minimally Invasive Spine Surgery. Shenyang: Shenyang Institute of Automation, Chinese Academy of Sciences
李杨. 2017. 面向微创脊柱手术经皮穿刺的图像引导技术研究. 沈阳: 中国科学院沈阳自动化研究所
Liebmann F, Roner S, von Atzigen M, Scaramuzza D, Sutter R, Snedeker J, Farshad M and Fürnstahl P. 2019. Pedicle screw navigation using surface digitization on the Microsoft HoloLens. International Journal of Computer Assisted Radiology and Surgery, 14(7): 1157-1165[DOI:10.1007/s11548-019-01973-7]
Liu X Z, Bai H, Song G L, Zhao Y W and Han J D. 2017. Augmented reality system training for minimally invasive spine surgery//Proceedings of 2017 IEEE International Conference on Robotics and Biomimetics. Macau, China: IEEE: 1200-1205[ DOI: 10.1109/ROBIO.2017.8324581 http://dx.doi.org/10.1109/ROBIO.2017.8324581 ]
Maciel A, Liu Y Q, Ahn W, Singh T P, Dunnican W and De S. 2008. Development of the VBLaST TM : a virtual basic laparoscopic skill trainer. The International Journal of Medical Robotics and Computer Assisted Surgery, 4(2): 131-138[DOI:10.1002/rcs.185]
Nemani A, Kruger U, Cooper C A, Schwaitzberg S D, Intes X and De S. 2019. Objective assessment of surgical skill transfer using non-invasive brain imaging. Surgical Endoscopy, 33(8): 2485-2494[DOI:10.1007/s00464-018-6535-z]
Niu Y H, Wang Y M and Duan H L. 2004, A survey on augmented-reality-based IGS system. Chinese Journal of Medical Devices, (01): 50-54
钮艳华, 汪元美, 段会龙. 2004, 基于增强现实的外科手术导航技术. 中国医疗器械杂志, (01): 50-54
Qian K, Bai J X, Yang X S, Pan J J and Zhang J J. 2015. Virtual reality based laparoscopic surgery simulation//The 21st ACM Symposium on Virtual Reality Software and Technology. Beijing, China: ACM: 69-78[ DOI: 10.1145/2821592.2821599 http://dx.doi.org/10.1145/2821592.2821599 ]
Qian K, Xuan J X, Yang X S, Pan J J and Zhang J J. 2017. Essential techniques for laparoscopic surgery simulation. Computer Animation and Virtual Worlds, 28(2): #1724[DOI:10.1002/cav.1724]
Shan Q, Doyle T E, Samavi R and Al-Rei M. 2017. Augmented reality based brain tumor 3D visualization. Procedia Computer Science, 113: 400-407[DOI:10.1016/j.procs.2017.08.356]
Shi Y L. 2014. Design and Study of Remote MIS Robot Under Large Delay. Tianjin: Tianjin University
师云雷. 2014. 大延迟环境下远程微创手术机器人系统设计与实验研究. 天津: 天津大学
Song Z J, Hu L, Wang M N, Yao D M, Li W S and Du W J. 2010. A surgical navigation system and method based on optical augmented reality technology. China, No. CN101904770A
宋志坚, 胡亮, 王满宁, 姚德民, 李文生, 杜文健. 2010, 一种基于光学增强现实技术的手术导航系统及方法. 中国. CN101904770A
Stepan K, Zeiger J, Hanchuk S, Del Signore A, Shrivastava R, Govindaraj S and Iloreta A. 2017. Immersive virtual reality as a teaching tool for neuroanatomy. International Forum of Allergy and Rhinology, 7(10): 1006-1013[DOI:10.1002/alr.21986]
Wang D X, Zhang Y R, Hou J X, Wang Y, Lv P J, Chen Y G and Zhao H. 2012. iDental: a haptic-based dental simulator and its preliminary user evaluation. IEEE Transactions on Haptics, 5(4): 332-343[DOI:10.1109/TOH.2011.59]
Wang J and Liu Dong. 2016. Research on cholecystectomy virtual surgery simulation training platform. Journal of System Simulation, 28(8): 1853-1862
汪军, 刘冬. 2016. 一种胆囊切除虚拟手术仿真训练平台研究. 系统仿真学报, 28(8): 1853-1862[DOI:10.16182/j.cnki.joss.2016.08.021]
Wang Z. 2006. Study of the Simulation of Soft Tissue Deformation in Virtual Surgery. Nanjing: Southeast University
王征. 2006. 虚拟手术中的软组织形变仿真算法研究. 南京: 东南大学
Wei L, Najdovski Z, Nahavandi S and Weisinger H. 2014. Towards a haptically enabled optometry training simulator. Network Modeling Analysis in Health Informatics and Bioinformatics, 3(1): #60[DOI:10.1007/s13721-014-0060-3]
Xu X F, Zhang D L, Qu G H and Pan H B. 2001. A three-dimensional medical visualization system. Computer Engineering, 27(2): 159-160
徐晓峰, 张大力, 曲桂红, 潘洪波. 2001. 三维医学可视化系统. 计算机工程, 27(2): 159-160[DOI:10.3969/j.issn.1000-3428.2001.02.063]
Yu P, Pan J J, Qin H, Hao A M and Wang H P. 2020. Real-time suturing simulation for virtual reality medical training. Computer Animation and Virtual Worlds, 31(4/5): #1940[DOI:10.1002/cav.1940]
Yuan Y, Weng D D, Wang Y T and Liu Y. 2008, Navigating system for minimally invasive endoscopic sinus surgrey based on augmented reality. Journal of System Simulation, 20(S1): 150-153
袁媛, 翁冬冬, 王涌天, 刘越. 2008, 基于增强现实的鼻内镜微创手术导航系统. 系统仿真学报, 20(S1): 150-153
Zang X J, Weng D D, Wang Y T and Liu Y. 2010. Navigation system for endoscopic sinus surgery based on augmented reality. Journal of Beijing Institute of Technology, 30(1): 69-73
臧晓军, 翁冬冬, 王涌天, 刘越. 2010, 基于增强现实的鼻内窥镜手术导航系统. 北京理工大学学报, 30(1): 69-73[DOI:10.15918/j.tbit1001-0645.2010.01.014]
Zhang X R, Chen G W and Liao H E. 2015. A minimally invasive surgical navigation system based on enhanced holographic in situ fusion//Proceedings of 2015 Chinese Biomedical Engineering Association Annual Conference. China Institute of Electronics; Chinese Institute of Instrumentation; Chinese Optical Society; China Society of Image and Graphics; Chinese Society of Biomedical Engineering
张欣然, 陈国文, 廖洪恩. 2015, 基于增强立体全像原位融合的微创手术导航系统//2015年中国生物医学工程联合学术年会. 中国电子学会; 中国仪器仪表学会; 中国光学学会; 中国图象图形学会; 中国生物医学工程学会
Zhao H and Wang D X. 2010. Soft tissue simulation with bimanual force feedback//Proceedings of 2010 International Conference on Audio, Language and Image Processing. Shanghai, China: IEEE: 903-907[ DOI: 10.1109/ICALIP.2010.5685189 http://dx.doi.org/10.1109/ICALIP.2010.5685189 ]
相关作者
相关机构
京公网安备11010802024621
