浏览全部资源
扫码关注微信
收稿：2019-11-22，
修回：2020-1-22，
录用：2020-1-29，
纸质出版：2020-07-16
移动端阅览
严肃游戏是计算机游戏一个新的发展方向，可以提供形象互动的模拟教学环境，已经广泛应用于科学教育、康复医疗、应急管理、军事训练等领域。虚拟角色是严肃游戏中模拟具有生命特征的图形实体，行为可信的虚拟角色能够提升用户使用严肃游戏的体验感。严肃游戏中的图形渲染技术已经逐步成熟，而虚拟角色行为建模的研究尚在初级阶段。可信的虚拟角色必须能够具有感知、情绪和行为能力。本文分别从游戏剧情与行为、行为建模方法、行为学习和行为建模评价等4个方面来分析虚拟角色行为建模研究。分析了有限状态机和行为树的特点，讨论了虚拟角色的行为学习方法。指出了强化学习的关键要素，探讨了深度强化学习的应用途径。综合已有研究，归纳了虚拟角色行为框架，该框架主要包括感觉输入、知觉分析、行为决策和动作4大模块。从情感计算的融入、游戏剧情和场景设计、智能手机平台和多通道交互4个角度讨论需要进一步研究的问题。虚拟角色的行为建模需要综合地考虑游戏剧情、机器学习和人机交互技术，构建具有自主感知、情绪、行为、学习能力、多通道交互的虚拟角色能够极大地提升严肃游戏的感染力，更好地体现寓教于乐。
Serious games are a new development direction of computer games. They can simulate interactive professional teaching environment and have been widely used in many fields
such as science education
rehabilitation
emergency management
and military training. Serious medical rehabilitation games are mainly used for medical technology training and rehabilitation-assisted treatment. They can train and improve the professional skills of medical staff
reduce the pain and boredom of patients during rehabilitation
and assist patients in rehabilitation treatment. Serious games in military training are mainly used in military modeling and simulation and possess controllability
security
and low cost. Virtual characters are simulated entities with life characteristics for serious games. They have credible behavior and can improve users' experience in serious games. At present
graphics rendering in serious games has gradually matured
but the research on virtual character behavior modeling is only in its first stage. A credible virtual character should have perception
emotion
and behavior. It can respond to the user's operation in time and has a certain reasoning ability. Modeling the behavior of virtual characters requires the knowledge of psychology
cognitive science
and specific domain knowledge. This paper aims to present a control algorithm that reflects the behavior of a virtual character under certain circumstances. This algorithm is an interdisciplinary application
which involves computer graphics
human-computer interaction
and artificial intelligence technologies. This paper summarized the existing studies on virtual character's behavior modeling from four aspects
namely
game plot and behavior
behavior modeling method
behavior learning
and behavior modeling evaluation. In the aspect of game plots and behavior
the design of behavior should contain the plot. The behavior of virtual characters accordingly change with the change in the plot. This condition was illustrated by the virtual character behavior design in the rehabilitation and military training plots. In the aspect of behavior modeling
the behavior modeling methods of virtual characters were summarized. The characteristics of finite state machines and behavior trees were analyzed
the behavioral learning methods were compared
the key elements of reinforcement learning were indicated
and the application of deep reinforcement learning was discussed. Reinforcement learning has four elements
namely
environment
state
action
and reward. Virtual character behavior decision learning could be completed by constantly attempting to obtain environmental rewards. In the aspect of behavior modeling evaluation
three indexes were summarized to evaluate the effectiveness of the model
namely
the real-time behavior of virtual character
the emotion-integrated behavior
and the behavior interactivity. An extremely slow character behavior response will reduce the user's interest in participating in the game
and a timely response will provide the user an efficient and pleasant experience. Users aim to achieve emotional communication with virtual characters in a virtual environment to have a good sense of immersion. A virtual character behavior framework was summarized on the basis of existing studies. It included four modules
namely
sensory input
perception analysis
behavior decision making
and action. Emotional factors are valuable to create virtual characters with credible behavior and realistic movements. They could also increase the appeal of the game. Issues that need additional research were discussed from the perspective of affective computing intervention
game story and scene design
smart phone platform
and multichannel interaction. Virtual characters should be introduced to expand the game plot and assist in teaching to completely realize the function of serious games. Serious games are different from movies. Users do not passively accept
and they learn through constant interaction. A virtual scene without any plot and any virtual character is difficult to attract users. A narrative plot can make it easy to learn. Virtual characters with behavior and emotional expressiveness can enhance users' emotional experience and guide users to learn in a real-life-like situation. The popularity of mobile terminals provides various applications for serious games and attracts many users to use serious games. Multimodel interaction would be popular in the future because single-model interaction will limit user's interaction with the virtual character. A multichannel interaction method could improve the intelligence of virtual characters and could be realized with visual
auditory
tactile
and somatosensory models. The behavior modeling of virtual characters requires comprehensive consideration of game plots
machine learning
and human-computer interaction technologies. Rigid behaviors could not attract users. Building virtual models with autonomous perception
emotion
behavior
learning ability
and multimodel interaction could immensely enhance user's immersion. These models are the development direction of serious games. Serious games provide an intuitive means for education and training. The behavior modeling of virtual characters is a developing core technology of serious games. At present
many problems need to be urgently solved. Affective computing methods
game plots
smartphone operating platforms
and multimodel human-computer interaction should be considered to improve the behavior model of virtual characters. The integration of interdisciplinary knowledge is required to create behavioral credibility
autonomous emotional expression
and convenient interactive virtual characters. These characters could provide a better learning experience for users and are easy to be promoted.
Afyouni I, Rehman F U, Qamar A M, Ghani S, Hussain S O, Sadiq B, Rahman M A, Murad A and Basalamah S. 2017. A therapy-driven gamification framework for hand rehabilitation. User Modeling and User-Adapted Interaction, 27(2):215-265[DOI:10.1007/s11257-017-9191-4]
Andersen P A, Goodwin M and Granmo O C. 2018. Deep RTS: a game environment for deep reinforcement learning in real-time strategy games//Proceedings of 2018 IEEE Conference on Computational Intelligence and Games. Maastricht, Netherlands: IEEE: 1-8[ DOI: 10.1109/CIG.2018.8490409 http://dx.doi.org/10.1109/CIG.2018.8490409 ]
Andersen P A. 2018. Deep Reinforcement Learning Using Capsules in Advanced Game Environments. Norway:University of Agder
Andrade K D O, Fernandes G, Caurin G A P, Siqueira A A G, Romero R A F and Pereira R D L. 2014. Dynamic player modelling in serious games applied to rehabilitation robotics//Proceedings of Joint Conference on Robotics: SBR-LARS Robotics Symposium and Robocontrol. Sao Carlos: IEEE: 211-216[ DOI: 10.1109/SBR.LARS.Robocontrol.2014.41 http://dx.doi.org/10.1109/SBR.LARS.Robocontrol.2014.41 ]
Asadipour A, Debattista K and Chalmers A. 2017. Visuohaptic augmented feedback for enhancing motor skills acquisition. The Visual Computer, 33(4):401-411[DOI:10.1007/s00371-016-1275-3]
Avola D, Cinque L, Foresti G L, Marini M R and Pannone D. 2018. VRheab:a fully immersive motor rehabilitation system based on recurrent neural network. Multimedia Tools and Applications, 77(19):24955-24982[DOI:10.1007/s11042-018-5730-1]
Cangas A J, Navarro N, Parra J M A, Ojeda J J, Cangas D, Piedra J A and Gallego J. 2017. Stigma-stop:a serious game against the stigma toward mental health in educational settings. Frontiers in Psychology, 8:#1385[DOI:10.3389/fpsyg.2017.01385]
Chen H, Liu Z, Chen T, Liu T T, Liu C J and Zhu X Y. 2016. Kinect based serious game of entertainment and body exercise for elderly. Journal of System Simulation, 28(10):2586-2592
陈浩, 刘箴, 陈田, 刘婷婷, 刘翠娟, 朱宪莹. 2016.一个面向老年人娱乐健身的体感严肃游戏研究.系统仿真学报, 28(10):2586-2592)[DOI:10.16182/j.cnki.joss.2016.10.042]
Cheng M T, Lin Y W, She H C and Kuo P C. 2017. Is immersion of any value? Whether, and to what extent, game immersion experience during serious gaming affects science learning. British Journal of Educational Technology, 48(2):246-263[DOI:10.1111/bjet.12386]
Cheng X Y, Wang Y C and Sankar C S. 2018. Using serious games in data communications and networking management course. Journal of Computer Information Systems, 58(1):39-48[DOI:10.1080/08874417.2016.1183465]
Chiang F K, Chang C H, Hu D, Zhang G R and Liu Y. 2019. Design and development of a safety educational adventure game. International Journal of Emerging Technologies in Learning, 14(3):201-218[DOI:10.3991/ijet.v14i03.9268]
Chittaro L and Buttussi F. 2015. Assessing knowledge retention of an immersive serious game vs. a traditional education method in aviation safety. IEEE Transactions on Visualization and Computer Graphics, 21(4):529-538[DOI:10.1109/TVCG.2015.2391853]
Ciman M, Gaggi O, Sgaramella T M, Nota L, Bortoluzzi M and Pinello L. 2018. Serious games to support cognitive development in children with cerebral visual impairment. Mobile Networks and Applications, 23(6):1703-1714[DOI:10.1007/s11036-018-1066-3]
Dankbaar M E W, Roozeboom M B, Oprins E A P B, Rutten F, van Merrienboer J J G, van Saase J L C M and Schuit S C E. 2017. Preparing residents effectively in emergency skills training with a serious game. Simulation in Healthcare, 12(1):9-16[DOI:10.1097/SIH.0000000000000194]
de Sena D P, Fabrício D D, da Silva V D, Bodanese L C and Franco A R. 2019. Comparative evaluation of video-based on-line course versus serious game for training medical students in cardiopulmonary resuscitation:a randomised trial. PLoS One, 14(4):#e0214722[DOI:10.1371/journal.pone.0214722]
Derbali L and Frasson C. 2012. Assessment of learners' motivation during interactions with serious games:a study of some motivational strategies in food-force. Advances in Human-Computer Interaction, 2012:#624538[DOI:10.1155/2012/624538]
Dobrovsky A, Borghoff U M and Hofmann M. 2017. An approach to interactive deep reinforcement learning for serious games//Proceedings of the 7th IEEE International Conference on Cognitive Infocommunications. Wroclaw: IEEE: 85-90[ DOI: 10.1109/CogInfoCom.2016.7804530 http://dx.doi.org/10.1109/CogInfoCom.2016.7804530 ]
Dobrovsky A, Borghoff U M and Hofmann M. 2019. Improving adaptive gameplay in serious games through interactive deep reinforcement learning//Klempous R, Nikodem J and Baranyi P Z, eds. Cognitive Infocommunications, Theory and Applications. Cham: Springer: 411-432[ DOI: 10.1007/978-3-319-95996-2_19 http://dx.doi.org/10.1007/978-3-319-95996-2_19 ]
Ferracani A, Pezzatini D, Seidenari L and Bimbo A D. 2015. Natural and virtual environments for the training of emergency medicine personnel. Universal Access in the Information Society, 14(3):351-362[DOI:10.1007/s10209-014-0364-1]
Fridenson-Hayo S, Berggren S, Lassalle A, Tal S, Pigat D, Meir-Goren N, O'Reilly H, Ben-Zur S, Bölte S, Baron-Cohen S and Golan O. 2017. 'Emotiplay':a serious game for learning about emotions in children with autism:results of a cross-cultural evaluation. European Child and Adolescent Psychiatry, 26(8):979-992[DOI:10.1007/s00787-017-0968-0]
Georgila K, Core M G, Nye B D, Karumbaiah S, Auerbach D and Ram M. 2019. Using reinforcement learning to optimize the policies of an intelligent tutoring system for interpersonal skills training//Proceedings of the 18th International Conference on Autonomous Agents and Multi Agent Systems. Montreal: International Foundation for Autonomous Agents and Multiagent Systems: 737-745
Gongora M and Irvine D. 2010. ReAd:reactive-adaptive methodology to enable evolving intelligent agents for virtual environments. Evolving Systems, 1(2):111-127[DOI:10.1007/s12530-010-9011-0]
Gulec U and Yilmaz M. 2016. A serious game for improving the decision making skills and knowledge levels of Turkish football referees according to the laws of the game. SpringerPlus, 5:#622[DOI:10.1186/s40064-016-2227-0]
Hocine N, Gouaïch A, Cerri S A, Mottet D, Froger J and Laffont I. 2015. Adaptation in serious games for upper-limb rehabilitation:an approach to improve training outcomes. User Modeling and User-Adapted Interaction, 25(1):65-98[DOI:10.1007/s11257-015-9154-6]
Ilachinski A. 2010. Artificial War: Multiagent-Based Simulation of Combat. Zhang Z X, Gao C R and Guo F L, trans. Beijing: Publishing House of Electronics Industry
安德鲁. 2010.人工战争: 基于多Agent的作战仿真.张志祥, 高春蓉, 郭福亮, 译.北京: 电子工业出版社
Ji L X and Ma J H. 2014. Research on the behavior of intelligent role in computer games based on behavior tree. Applied Mechanics and Materials, 509:165-169[DOI:10.4028/www.scientific.net/AMM.509.165]
Khowaja K and Salim S S. 2019. Serious game for children with autism to learn vocabulary:an experimental evaluation. International Journal of Human-Computer Interaction, 35(1):1-26[DOI:10.1080/10447318.2017.1420006]
Koutník J, Schmidhuber J and Gomez F. 2014. Online evolution of deep convolutional network for vision-based reinforcement learning//Proceedings of the 13th International Conference on Simulation of Adaptive Behavior. Castellón, Spain: Springer: 260-269[ DOI: 10.1007/978-3-319-08864-8_25 http://dx.doi.org/10.1007/978-3-319-08864-8_25 ]
Lala D and Nishida T. 2017. A data-driven passing interaction model for embodied basketball agents. Journal of Intelligent Information Systems, 48(1):27-60[DOI:10.1007/s10844-015-0386-z]
Lee S Y, Rowe J P, Mott B W and Lester J C. 2014. A supervised learning framework for modeling director agent strategies in educational interactive narrative. IEEE Transactions on Computational Intelligence and AI in Games, 6(2):203-215[DOI:10.1109/TCIAIG.2013.2292010]
Liang H, Chang J, Kazmi I K, Zhang J J and Jiao P F. 2017. Hand gesture-based interactive puppetry system to assist storytelling for children. The Visual Computer, 33(4):517-531[DOI:10.1007/s00371-016-1272-6]
Liu Q, Zhai J W, Zhang Z Z, Zhong S, Zhou Q, Zhang P and Xu J. 2018. A survey on deep reinforcement learning. Chinese Journal of Computers, 41(1):1-27
刘全, 翟建伟, 章宗长, 钟珊, 周倩, 章鹏, 徐进. 2018.深度强化学习综述.计算机学报, 41(1):1-27)[DOI:10.11897/SP.J.1016.2018.00001]
Liu Z, He S H and Chai Y J. 2011. An emotion cognitive appraisal model for virtual characters. Pattern Recognition and Artificial Intelligence, 24(2):160-167
刘箴, 何少华, 柴艳杰. 2011.一种虚拟角色的情绪认知评价模型.模式识别与人工智能, 24(2):160-167)[DOI:10.3969/j.issn.1003-6059.2011.02.002]
Matsas E and Vosniakos G C. 2017. Design of a virtual reality training system for human-robot collaboration in manufacturing tasks. International Journal on Interactive Design and Manufacturing (IJIDeM), 11(2):139-153[DOI:10.1007/s12008-015-0259-2]
Merrick K E and Maher M L. 2009. Non-player characters in multiuser games//Merrick K E and Maher M L, eds. Motivated Reinforcement Learning. Berlin, Heidelberg: Springer: 3-16[ DOI: 10.1007/978-3-540-89187-1_1 http://dx.doi.org/10.1007/978-3-540-89187-1_1 ]
Merriman N A, Roudaia E, Romagnoli M, Orvieto I and Newell F N. 2018. Acceptability of a custom-designed game, CityQuest, aimed at improving balance confidence and spatial cognition in fall-prone and healthy older adults. Behaviour and Information Technology, 37(6):538-557[DOI:10.1080/0144929X.2018.1462402]
Milne M, Powers D and Leibbrandt R. 2009. Development of a software-based social tutor for children with autism spectrum disorders//Proceedings of the 21st Annual Conference of the Australian Computer-Human Interaction Special Interest Group: Design: Open. Melbourne: ACM: 265-268[ DOI: 10.1145/1738826.1738870 http://dx.doi.org/10.1145/1738826.1738870 ]
Nicolau M, Perez-Liebana D, O'Neill M and Brabazon A. 2017. Evolutionary behavior tree approaches for navigating platform games. IEEE Transactions on Computational Intelligence and AI in Games, 9(3):227-238[DOI:10.1109/TCIAIG.2016.2543661]
Oh I, Rho S, Moon S, Son S, Lee H and Chung J. 2019. Creating pro-level AI for a real-time fighting game using deep reinforcement learning[EB/OL]. https://arxiv.org/pdf/1904.03821.pdf https://arxiv.org/pdf/1904.03821.pdf
Pacella D and López-Pérez B. 2018. Assessing children's interpersonal emotion regulation with virtual agents:the serious game Emodiscovery. Computers and Education, 123:1-12[DOI:10.1016/j.compedu.2018.04.005]
Perez D, Nicolau M, O'Neill M and Brabazon A. 2011. Evolving behaviour trees for the Mario AI competition using grammatical evolution//Proceedings of the International Conference on Applications of Evolutionary Computation. Torino: Springer: 123-132[ DOI: 10.1007/978-3-642-20525-5_13 http://dx.doi.org/10.1007/978-3-642-20525-5_13 ]
Perlin K and Goldberg A. 1996. Improv: a system for scripting interactive actors in virtual worlds//Proceedings of the 23rd Annual Conference on Computer Graphics and Interactive Techniques. New Orleans, USA: ACM: 205-216[ DOI: 10.1145/237170.237258 http://dx.doi.org/10.1145/237170.237258 ]
Psaltis A, Apostolakis K C, Dimitropoulos K and Daras P. 2018. Multimodal student engagement recognition in prosocial games. IEEE Transactions on Games, 10(3):292-303[DOI:10.1109/TCIAIG.2017.2743341]
Ravyse W S, Blignaut A S, Leendertz V and Woolner A. 2017. Success factors for serious games to enhance learning:a systematic review. Virtual Reality, 21(1):31-58[DOI:10.1007/s10055-016-0298-4]
Roozeboom M B, Visschedijk G and Oprins E. 2017. The effectiveness of three serious games measuring generic learning features. British Journal of Educational Technology, 48(1):83-100[DOI:10.1111/bjet.12342]
Sajjadi P, Hoffmann L, Cimiano P and Kopp S. 2019. A personality-based emotional model for embodied conversational agents:effects on perceived social presence and game experience of users. Entertainment Computing, 32:#100313[DOI:10.1016/j.entcom.2019.100313]
Saridis G N. 1977. Self-Organizing Control of Stochastic Systems. New York:Marcel Dekker
Sekhavat Y A. 2017. MPRL: multiple-periodic reinforcement learningfor difficulty adjustment in rehabilitation games//Proceedings of the 5th IEEE International Conference on Serious Games and Applications for Health. Perth: IEEE: 1-7[ DOI: 10.1109/SeGAH.2017.7939260 http://dx.doi.org/10.1109/SeGAH.2017.7939260 ]
Smith R D. 2009. Military Simulation and Serious Games:Where We Came From and Where We Are Going.[s.l.]:Modelbenders Press
Syufagi M A, Hariadi M and Purnomo M H. 2013. Petri net model for serious games based on motivation behavior classification. International Journal of Computer Games Technology, 2013:#851287[DOI:10.1155/2013/851287]
Tampuu A, Matiisen T, Kodelja D, Kuzovkin I, Korjus K, Aru J, Aru J and Vicente R. 2017. Multiagent cooperation and competition with deep reinforcement learning. PLoS One, 12(4):#e0172395[DOI:10.1371/journal.pone.0172395]
Terzidou T, Tsiatsos T, Miliou C and Sourvinou A. 2016. Agent supported serious game environment. IEEE Transactions on Learning Technologies, 9(3):217-230[DOI:10.1109/TLT.2016.2521649]
Tian Z H, Zhao L and Jia Y. 2010. Research on modeling action with uncertain results based on probability interval. Computer Science, 37(1):201-203, 207
田尊华, 赵龙, 贾焰. 2010.基于概率区间的不确定性动作建模研究.计算机科学, 37(1):201-203, 207)[DOI:10.3969/j.issn.1002-137X.2010.01.047]
Tu H and Liu H X. 2017. An improved Q-learning algorithm and its application in behavior tree. Computer Applications and Software, 34(12):235-239
涂浩, 刘洪星. 2017.一种改进型Q学习算法及其在行为树中的应用.计算机应用与软件, 34(12):235-239)[DOI:10.3969/j.issn.1000-386x.2017.12.045]
Wu J J, Liu Z, Liu T T and Wang J. 2019. Uncertainty model to generate virtual characters' behavior in serious games for social training. Journal of Image and Graphics, 25(9):1558-1568
吴佳佳, 刘箴, 刘婷婷, 王瑾. 2019.辅助社交训练严肃游戏中虚拟角色行为表现的不确定性模型.中国图象图形学报, 25(9):1558-1568)[DOI:10.11834/jig.180581]
Zhang Q, Xu K, Jiao P and Yin Q J. 2018. Behavior modeling for autonomous agents based on modified evolving behavior trees//Proceedings of the 7th IEEE Data Driven Control and Learning Systems Conference (DDCLS). Enshi: IEEE: 1140-1145[ DOI: 10.1109/DDCLS.2018.8515939 http://dx.doi.org/10.1109/DDCLS.2018.8515939 ]
京公网安备11010802024621
