Abstract
Educational games are one prominent type of Serious Games that are designed for educational purposes. One goal of evaluating their benefits is ensuring that they meet these objectives. Some measures for this purpose are defined in fields like Learning Analytics and Technology-enhanced Learning. However, unlike other educational methods, meeting learning goals is not the only success measure of Serious Games. Contrary, complex requirements arise from their nature as games which are discussed in fields like Game User Research and Game Analytics and need to be considered. This thesis investigates how the use of new data collection opportunities enabled by smartphones as an interaction platform can benefit research in the field of Serious Games evaluation. Unobtrusive integrated sensors are used for observing user interactions in natural contexts. This can be vital for capturing factors that can help evaluate a Serious Game from multiple perspectives and with deeper insights. With the l...
References (342)
- Sam Adkins. Serious games industry in boom phase: The 2019-2024 worldwide game-based learning market. Ambient Insight (Metaari), August 2019.
- Kingsley Oryina Akputu, Kah Phooi Seng, and Yun Li Lee. Facial emotion recognition for intelligent tutoring environment. In Proceedings of the 2nd International Conference on Machine Learning and Computer Science (IMLCS), pages 9-13, Kuala Lumpur, Malaysia, August 2013.
- Naif R. Aljohani and Hugh C. Davis. Learning analytics in mobile and ubiquitous learning environ- ments. In Proceedings of the 11th World Conference on Mobile and Contextual Learning (mLearn), Finland, October 2012. CEUR-WS.
- Naif R. Aljohani and Hugh C. Davis. Significance of learning analytics in enhancing the mobile and pervasive learning environments. In Proceedings of the 6th International Conference on Next Generation Mobile Applications, Services and Technologies (NGMAST), pages 70-74, Paris, France, September 2012. IEEE.
- Anissa All, Elena Patricia Nunez Castellar, and Jan Van Looy. Measuring effectiveness in digital game-based learning: A methodological review. International Journal of Serious Games (IJSG), 1 (2):3-20, June 2014.
- Asmaa Alsumait and Asma Al-Osaimi. Usability heuristics evaluation for child e-learning ap- plications. In Proceedings of the 11th International Conference on Information Integration and Web-based Applications & Services (iiWAS), pages 425-430, Kuala Lumpur, Malaysia, December 2009. Association for Computing Machinery.
- Luigi Anolli, Fabrizia Mantovani, Linda Confalonieri, Antonio Ascolese, and L. Peveri. Emotions in serious games: From experience to assessment. International Journal of Emerging Technologies in Learning (iJET), 5(SI3):7-16, November 2010.
- Aitor Apaolaza, Simon Harper, and Caroline Jay. Understanding users in the wild. In Proceedings of the 10th International Cross-Disciplinary Conference on Web Accessibility (W4A), 13, pages 1-4, Rio de Janeiro, Brazil, May 2013. Association for Computing Machinery.
- Kimberly E. Arnold and Matthew D. Pistilli. Course signals at Purdue: Using learning analytics to increase student success. In Proceedings of the 2nd International Conference on Learning An- alytics and Knowledge (LAK), pages 267-270, Vancouver, British Columbia, Canada, April 2012. Association for Computing Machinery.
- Ivon Arroyo, David G. Cooper, Winslow Burleson, Beverly Park Woolf, Kasia Muldner, and Robert Christopherson. Emotion sensors go to school. In Proceedings of the 2009 Conference on Artificial Intelligence in Education (AIED), pages 17-24, NLD, July 2009. IOS Press.
- Ivon Arroyo, Beverly Park Woolf, Winslow Burelson, Kasia Muldner, Dovan Rai, and Minghui Tai. A multimedia adaptive tutoring system for mathematics that addresses cognition, metacognition and affect. International Journal of Artificial Intelligence in Education (IJAIED), 24(4):387-426, December 2014.
- Nikolaos Avouris, Vassilis Komis, Meletis Margaritis, and Georgios Fiotakis. An environment for studying collaborative learning activities. Educational Technology & Society, 7(2):34-41, April 2004.
- Nikolaos Avouris, Vassilis Komis, Georgios Fiotakis, Meletis Margaritis, and Eleni Voyiatzaki. Log- ging of fingertip actions is not enough for analysis of learning activities. In Proceedings of Workshop on Usage Analysis in Learning Systems in the 2005 Conference on Artificial Intelligence in Education (AIED), 7, pages 1-8, Brighton, UK, 2005.
- Kiavash Bahreini, Rob Nadolski, and Wim Westera. Multimodal emotion recognition for assess- ment of learning in a game-based communication skills training. In Proceedings of the Workshop Learning Analytics for and in Serious Games at the European Conference on Technology Enhanced Learning (ECTEL), pages 22-25, Graz, Austria, 2014.
- Kiavash Bahreini, Rob Nadolski, and Wim Westera. Towards multimodal emotion recognition in e-learning environments. Interactive Learning Environments, 24(3):590-605, April 2016.
- Kiavash Bahreini, Rob Nadolski, and Wim Westera. Towards real-time speech emotion recognition for affective e-learning. Education and Information Technologies, 21(5):1367-1386, September 2016.
- Ryan S. J. D. Baker and Kalina Yacef. The state of educational data mining in 2009: A review and future visions. Journal of Educational Data Mining (JEDM), 1(1):3-17, October 2009.
- Ryan S. J. d. Baker, Sidney K. D'Mello, Ma. Mercedes T. Rodrigo, and Arthur C. Graesser. Better to be frustrated than bored: The incidence, persistence, and impact of learners' cognitive-affective states during interactions with three different computer-based learning environments. Interna- tional Journal of Human-Computer Studies, 68(4):223-241, April 2010.
- Ryan Shaun Baker, Albert T. Corbett, Kenneth R. Koedinger, and Angela Z. Wagner. Off-task behavior in the cognitive tutor classroom: when students "game the system". In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI), pages 383-390, Vienna, Austria, April 2004. ACM.
- Ntombikayise Banda and Peter Robinson. Multimodal affect recognition in intelligent tutoring systems. In Proceedings of the 5th International Conference of Affective Computing and Intelligent Interaction (ACII), pages 200-207, Berlin, Heidelberg, 2011. Springer.
- Xuan Bao, Songchun Fan, Alexander Varshavsky, Kevin Li, and Romit Roy Choudhury. Your reac- tions suggest you liked the movie: Automatic content rating via reaction sensing. In Proceedings of the 2013 ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp), pages 197-206, Zurich, Switzerland, September 2013. Association for Computing Machinery.
- Richard G. Baraniuk. Compressive sensing [lecture notes]. IEEE Signal Processing Magazine, 24 (4):118-121, July 2007.
- Shaowen Bardzell, Jeffrey Bardzell, and Tyler Pace. Understanding affective interaction: emotion, engagement, and internet videos. In Proceedings of the 3rd International Conference on Affective Computing and Intelligent Interaction (ACII), 40, pages 1-8, Amsterdam, Netherlands, September 2009. IEEE.
- Richard Bartle. Hearts, clubs, siamonds, apades: Players who suit MUDs. The Journal of Virtual Environments, 1(1), 1996.
- Gian-Marco Baschera, Alberto Giovanni Busetto, Severin Klingler, Joachim M. Buhmann, and Markus Gross. Modeling engagement dynamics in spelling learning. In Proceedings of the 15th International Conference on Artificial Intelligence in Education (AIED), pages 31-38, Auckland, New Zealand, June 2011. Springer.
- Marek Bell, Matthew Chalmers, L. Fontaine, Matthew Higgs, Alistair Morrison, John Rooksby, Mattias Rost, and Scott Sherwood. Experiences in logging everyday app use. In Proceedings of the Digital Economy (DE), pages 4-6, Salford, UK, 2013. ACM.
- Lisa Benson, Dean Elliott, Michael Grant, Doug Holschuh, Beaumie Kim, Hyeonjin Kim, Erick Lauber, Sebastian Loh, and Thomas C. Reeves. Usability and instructional design heuristics for e-learning evaluation. In Proceedings of World Conference on Educational Multimedia, Hypermedia and Telecommunications (ED-MEDIA), pages 1615-1621, Denver, Colorado, 2002. Association for the Advancement of Computing in Education (AACE).
- Niels Ole Berntsen and Laila Dybkjaer. Usability data analysis and evaluation. In Multimodal Usability, chapter 16, pages 351-385. Springer-Verlag, London, September 2010.
- Nagella Uday Bhaskar and P. Govindarajulu. Context exploration for requirements elicitation in mobile learning application development. International Journal of Computer Science and Network Security (IJCSNS), 8(8):292-299, 2008.
- Ariel Bleicher. Rise of the Eye Phones [News]. IEEE Spectrum, 50(5):9-10, May 2013.
- Herbert Bless, Gerald L. Clore, Norbert Schwarz, Verena Golisano, Christina Rabe, and Marcus Wölk. Mood and the use of scripts: Does a happy mood really lead to mindlessness? Journal of Personality and Social Psychology, 71(4):665-679, 1996.
- Paulo Blikstein and Marcelo Worsley. Multimodal Learning Analytics and Education Data Min- ing: using computational technologies to measure complex learning tasks. Journal of Learning Analytics, 3(2):220-238, September 2016.
- Paris Mavromoustakos Blom, Sander Bakkes, Chek Tien Tan, Shimon Whiteson, Diederik Roijers, Roberto Valenti, and Theo Gevers. Towards personalised gaming via facial expression recogni- tion. In Proceedings of the Tenth AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment (AIIDE), pages 30-36, Raleigh, NC, October 2014. AAAI Press.
- Matthias Böhmer, Brent Hecht, Johannes Schöning, Antonio Krüger, and Gernot Bauer. Falling asleep with Angry Birds, Facebook and Kindle: a large scale study on mobile application usage. In Proceedings of the 13th International Conference on Human Computer Interaction with Mobile Devices and Services (MobileHCI), pages 47-56, Stockholm, Sweden, August 2011. Association for Computing Machinery.
- Nigel Bosch. Multimodal affect detection in the wild: Accuracy, availability, and generalizability. In Proceedings of the 2015 ACM on International Conference on Multimodal Interaction (ICMI), pages 645-649, Seattle, Washington, USA, November 2015. Association for Computing Machinery.
- Nigel Bosch, Huili Chen, Sidney D'Mello, Ryan Baker, and Valerie Shute. Accuracy vs. availability heuristic in multimodal affect detection in the wild. In Proceedings of the 2015 ACM on Interna- tional Conference on Multimodal Interaction (ICMI), pages 267-274, Seattle, Washington, USA, November 2015. Association for Computing Machinery.
- LaVonda Brown and Ayanna M. Howard. Assessment of engagement for intelligent educational agents: A pilot study with middle school students. ASEE Computers in Education (CoED) Journal, 5(4):96-106, 2014.
- Patrick Brundell, Paul Tennent, Chris Greenhalgh, Dawn Knight, Andy Crabtree, Claire O'Malley, Shaaron Ainsworth, David Clarke, Ronald Carter, and Svenja Adolphs. Digital Replay System (DRS): a tool for interaction analysis. In Proceedings of the International Conference on Learning Sciences (ICLS) -Workshop on Interaction Analysis, Utrecht, The Netherlands, 2008.
- Susan Bull. Supporting learning with Open Learner Models. In Proceedings of the 4th Hellenic Conference on Information and Communication Technologies in Education (ICT in Education), pages 47-61, Athens, Greece, September 2004. Hellenic Association of ICT in Education (HAICTE).
- Susan Bull and Judy Kay. Open Learner Models. In Roger Nkambou, Jacqueline Bourdeau, and Riichiro Mizoguchi, editors, Advances in Intelligent Tutoring Systems, pages 301-322. Springer, Berlin, Heidelberg, 2010.
- Paolo Burelli, Georgios Triantafyllidis, and Ioannis Patras. Non-Invasive player experience estima- tion from body motion and game context. In Proceedings of the IEEE Conference on Computational Intelligence and Games (CIG), 68, pages 1-7, Dortmund, Germany, 2014. IEEE.
- JaeHwan Byun, Christian S. Loh, and Ting Zhou. Assessing play-learners' performance in se- rious game environments by using in situ data: Using eye tracking for serious game analytics. In Paper presented at the Annual Conference of the Association for Educational Communications and Technology (AECT), Jacksonville, FL, 2014. Research and Theory Division Jacksonville, FL, aect.org.
- Patrick Carmichael and Katy Jordan. Semantic web technologies for education -time for a "turn to practice"? Technology, Pedagogy and Education, 21(2):153-169, 2012.
- Keng-hao Chang, Drew Fisher, John Canny, and Björn Hartmann. How's my mood and stress? an efficient speech analysis library for unobtrusive monitoring on mobile phones. In Proceedings of the 6th International Conference on Body Area Networks, pages 71-77, Beijing, China, November 2011. ICST (Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering).
- Maher Chaouachi, Imane Jraidi, and Claude Frasson. MENTOR: A physiologically controlled tutoring system. In Proceedings of the 23rd International Conference on User Modeling, Adaptation and Personalization (UMAP), pages 56-67, Cham, 2015. Springer International Publishing.
- Kyoung-Sic Cho, In-Ho Choi, and Yong-Guk Kim. Robust facial expression recognition using a smartphone working against illumination variation. Applied Mathematics & Information Sciences, 6(2S):403S-408S, 2012.
- M. Chuah and M. Diblasio. Smartphone based Autism social alert system. In Proceedings of the 8th International Conference on Mobile Ad-hoc and Sensor Networks (MSN), pages 6-13, Chengdu, China, December 2012. IEEE.
- Meng-Che Chuang, Raja Bala, Edgar A. Bernal, Peter Paul, and Aaron Burry. Estimating gaze direction of vehicle drivers using a smartphone camera. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pages 165-170, Columbus, OH, June 2014. IEEE.
- Gregory K. W. K. Chung. Guidelines for the design and implementation of game telemetry for serious games analytics. In Christian Sebastian Loh, Yanyan Sheng, and Dirk Ifenthaler, editors, Serious Games Analytics: Methodologies for Performance Measurement, Assessment, and Improve- ment, pages 59-79. Springer International Publishing, Cham, 2015.
- Gregory K. W. K. Chung and Deirdre S. Kerr. A primer on data logging to support extraction of meaningful information from educational games: An example from save patch. Technical Report CRESST Report 814, National Center for Research on Evaluation, Standards, and Student Testing (CRESST), California, LA, March 2012.
- Cristina Conati. How to evaluate models of user affect? In Affective Dialogue Systems, Tutorial and Research Workshop (ADS), chapter 30, pages 288-300. Springer, Kloster Irsee, Germany, 2004.
- Cristina Conati and Mirela Gutica. Interaction with an edu-game: A detailed analysis of stu- dent emotions and judges' perceptions. International Journal of Artificial Intelligence in Education (IJAIED), 26(4):975-1010, December 2016.
- Cristina Conati and Heather Maclaren. Modeling user affect from causes and effects. In Proceed- ings of the 23rd International Conference on User Modeling, Adaptation and Personalization (UMAP), pages 4-15, Berlin, Germany, 2009. Springer.
- Adam Cooper. Learning analytics and interoperability -The big picture in brief. Technical report, Learning Analytics Community Exchange (LACE), March 2014. Project Description.
- Ben Cowley, Niklas Ravaja, and Tuija Heikura. Cardiovascular physiology predicts learning effects in a serious game activity. Computers & Education, 60(1):299-309, January 2013.
- Benjamin Cowley and Niklas Ravaja. Learning in balance: Using oscillatory EEG biomarkers of attention, motivation and vigilance to interpret game-based learning. Cogent Education, 1(1): 962236, December 2014.
- Mark Cox, Jesus Nuevo-Chiquero, Jason Saragih, and Simon Lucey. CSIRO Face Analysis SDK. Technical report, AFGR, Brisbane, Australia, 2013.
- Heather Desurvire and Charlotte Wiberg. Game usability heuristics (PLAY) for evaluating and designing better games: The next iteration. In Proceedings of the Third International Conference on Online Communities and Social Computing (OCSC), pages 557-566, Berlin, Heidelberg, 2009. Springer.
- Heather Desurvire, Martin Caplan, and Jozsef A. Toth. Using heuristics to evaluate the playability of games. In Proceedings of the ACM CHI Annual Conference on Human Factors in Computing Systems Extended Abstracts (CHI EA), pages 1509-1512, Vienna, Austria, April 2004. Association for Computing Machinery.
- Beth Dietz-Uhler and Janet E. Hurn. Using learning analytics to predict (and improve) student success: A faculty perspective. Journal of Interactive Online Learning, 12(1):17-26, 2013.
- Priyesh N. Dixit and G. Michael Youngblood. Understanding information observation in interactive 3D environments. In Proceedings of the 5th ACM SIGGRAPH Symposium on Video Games (Sandbox), pages 163-170, Los Angeles, California, August 2008. Association for Computing Machinery.
- Anders Drachen, Lennart E. Nacke, Georgios Yannakakis, and Anja Lee Pedersen. Correlation between heart rate, electrodermal activity and player experience in first-person shooter games. In Proceedings of the 5th ACM SIGGRAPH Symposium on Video Games (Sandbox), pages 49-54, Los Angeles, California, July 2010. Association for Computing Machinery.
- Brandon Drenikow and Pejman Mirza-Babaei. Vixen: Interactive visualization of gameplay expe- riences. In Proceedings of the 12th International Conference on the Foundations of Digital Games (FDG), 3, pages 1-10, Hyannis, MA, August 2017. Association for Computing Machinery.
- Erik Duval. Attention Please! Learning analytics for visualization and recommendation. In Pro- ceedings of the 1st International Conference on Learning Analytics and Knowledge (LAK), pages 9-17, Banff, Alberta, Canada, February 2011. Association for Computing Machinery.
- Anna Lea Dyckhoff, Dennis Zielke, Mareike Bültmann, Mohamed Amine Chatti, and Ulrik Schroeder. Design and implementation of a learning analytics toolkit for teachers. Educational Technology & Society, 15(3):58-76, 2012.
- Gregory Dyke, Kristine Lund, and Jean-Jacques Girardot. Tatiana: An environment to support the CSCL analysis process. In Proceedings of the 9th international conference on Computer supported collaborative learning -Volume 1 (CSCL), pages 58-67, Rhodes, Greece, June 2009. International Society of the Learning Sciences.
- Anastasios A. Economides. Adaptive context-aware pervasive and ubiquitous learning. Interna- tional Journal of Technology Enhanced Learning (IJTEL), 1(3):169-192, May 2009.
- Paul Ekman. Universals and cultural differences in facial expressions of emotion. Nebraska Sym- posium on Motivation, 19:207-283, 1971.
- Rana el Kaliouby and Peter Robinson. Real-time inference of complex mental states from facial expressions and head gestures. In Branislav Kisačanin, Vladimir Pavlović, and Thomas S. Huang, editors, Real-Time Vision for Human-Computer Interaction, pages 181-200. Springer US, Boston, MA, 2005.
- Magy Seif El-Nasr, Anders Drachen, and Alessandro Canossa. Game analytics: Maximizing the value of player data. Springer-Verlag, London, 2013.
- Clayton Epp, Michael Lippold, and Regan L. Mandryk. Identifying emotional states using keystroke dynamics. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI), pages 715-724, Vancouver, BC, Canada, May 2011. Association for Computing Machinery.
- Aysu Ezen-Can, Joseph F. Grafsgaard, James C. Lester, and Kristy Elizabeth Boyer. Classifying student dialogue acts with multimodal learning analytics. In Proceedings of the Fifth International Conference on Learning Analytics And Knowledge (LAK), pages 280-289, Poughkeepsie, New York, March 2015. Association for Computing Machinery.
- Jean-Claude Falmagne, Dietrich Albert, Christopher Doble, David Eppstein, and Xiangen Hu. Knowledge spaces: Applications in education. Springer, Berlin, Heidelberg, 2013.
- Ian Timothy Fischer, Cynthia Kuo, Ling Huang, and Mario Frank. Smartphones: Not smart enough? In Proceedings of the second ACM Workshop on Security and Privacy in Smartphones and Mobile Devices (SPSM), pages 27-32, Raleigh, North Carolina, USA, October 2012. Associa- tion for Computing Machinery.
- Adam Fouse, Nadir Weibel, Edwin Hutchins, and James D. Hollan. ChronoViz: a system for supporting navigation of time-coded data. In Proceedings of the ACM CHI Annual Conference on Human Factors in Computing Systems Extended Abstracts (CHI EA), pages 299-304, Vancouver, BC, Canada, May 2011. Association for Computing Machinery.
- Sara Freitas, Genaro Rebolledo-Mendez, Fotis Liarokapis, George Magoulas, and Alexandra Poulo- vassilis. Learning as immersive experiences: Using the Four-dimensional framework for designing and evaluating immersive learning experiences in a virtual world. British Journal of Educational Technology, 41(1):69-85, January 2010.
- Jon Froehlich, Mike Y. Chen, Sunny Consolvo, Beverly Harrison, and James A. Landay. MyExperi- ence: a system for in situ tracing and capturing of user feedback on mobile phones. In Proceedings of the 5th International Conference on Mobile Systems, Applications and Services (MobiSys), pages 57-70, San Juan, Puerto Rico, June 2007. Association for Computing Machinery.
- Fong-Ling Fu and Sheng-Chin Yu. Three layered thinking model for designing web-based educa- tional games. In Proceedings of the 17th International Conference on Web-Based Learning: Advances in Web Based Learning (ICWL), pages 265-274, Berlin, Heidelberg, 2008. Springer.
- Rosemary Garris, Robert Ahlers, and James E. Driskell. Games, motivation, and learning: A research and practice model. Simulation & Gaming, 33(4):441-467, December 2002.
- Sari Gilbert. Designing gamified systems: Meaningful play in interactive entertainment, marketing and education. Routledge, New York, September 2015.
- Stefan Göbel, André De, Carvalho Rodrigues, Florian Mehm, and Ralf Steinmetz. Narrative game- based learning objects for story-based digital educational games. In Proceedings of the 1st Interna- tional Open Workshop on Intelligent Personalization and Adaptation in Digital Educational Games, 14, pages 43-53, Graz, January 2009.
- Stefan Göbel, Viktor Wendel, Christopher Ritter, and Ralf Steinmetz. Personalized, adaptive digital educational games using narrative game-based learning objects. In Entertainment for Education. Digital Techniques and Systems, pages 438-445, Berlin, Heidelberg, 2010. Springer.
- Stefan Göbel, Michael Gutjahr, and Sandro Hardy. Evaluation of serious games. In Klaus Bredl and Wolfgang Bösche, editors, Serious Games and Virtual Worlds in Education, Professional Devel- opment, and Healthcare, chapter 7, pages 105-115. IGI Global, 2013.
- July Silveira Gomes, Mohamed Yassine, Marcelo Worsley, and Paulo Blikstein. Analysing engi- neering expertise of high school students using eye tracking and multimodal learning analytics. In Proceedings of the 6th International Conference on Educational Data Mining (EDM), pages 375- 377. International Educational Data Mining Society, January 2013.
- Sergio Gómez, Panagiotis Zervas, Demetrios G. Sampson, and Ramón Fabregat. Context-aware adaptive and personalized mobile learning delivery supported by UoLmP. Journal of King Saud University Computer and Information Sciences, 26(1):47-61, January 2014.
- Arthur Graesser, Patrick Chipman, Frank Leeming, and Suzanne Biedenbach. Deep learning and emotion in serious games. In Peter Vorderer Ute Ritterfeld, Michael Cody, editor, Serious Games: Mechanisms and Effects, chapter 6, pages 81-100. Routledge, Taylor & Francis New York and London, 2009.
- Joseph F. Grafsgaard, Joseph B. Wiggins, Kristy Elizabeth Boyer, Eric N. Wiebe, and James C. Lester II. Embodied affect in tutorial dialogue. In Proceedings of the 16th International Conference on Artificial Intelligence in Education (AIED), 1, pages 1-10. Springer, 2013.
- Joseph F. Grafsgaard, Joseph B. Wiggins, Alexandria Katarina Vail, Kristy Elizabeth Boyer, Eric N. Wiebe, and James C. Lester. The additive value of multimodal features for predicting engagement, frustration, and learning during tutoring. In Proceedings of the 16th International Conference on Multimodal Interaction (ICMI), pages 42-49, Istanbul, Turkey, November 2014. Association for Computing Machinery.
- Joseph F. Grafsgaard, Seung Y. Lee, Bradford W. Mott, Kristy Elizabeth Boyer, and James C. Lester. Modeling self-efficacy across age groups with automatically tracked facial expression. In Proceed- ings of the International Conference on Artificial Intelligence in Education (AIED), pages 582-585, Cham, 2015. Springer International Publishing.
- Dan J. Graham and J. Aaron Hipp. Emerging technologies to promote and evaluate physical activity: Cutting-edge research and future directions. Frontiers in Public Health, 2:66, 2014.
- Mirela Gutica and Cristina Conati. Student emotions with an edu-game: A detailed analysis. In Proceedings of the Conference on Affective Computing and Intelligent Interaction (ACII), pages 534-539, Geneva, September 2013. IEEE.
- Eun Y. Ha, Jonathan P. Rowe, Bradford W. Mott, and James C. Lester. Recognizing player goals in open-ended digital games with Markov Logic networks. In Gita Sukthankar, Christopher Geib, Hung Hai Bui, David Pynadath, and Robert P. Goldman, editors, Plan, Activity, and Intent Recog- nition: Theory and Practice, pages 289-311. Morgan Kaufmann, Amsterdam ; Boston, February 2014.
- Richard Halverson and V. Elizabeth Owen. Game-based assessment: an integrated model for capturing evidence of learning in play. International Journal of Learning Technology (IJLT), 9(2): 111-138, August 2014.
- Seungyeop Han, Rajalakshmi Nandakumar, Matthai Philipose, Arvind Krishnamurthy, and David Wetherall. GlimpseData: Towards continuous vision-based personal analytics. In Proceedings of the 2014 Workshop on Physical Analytics (ICMI), pages 31-36, Bretton Woods, New Hampshire, USA, June 2014. Association for Computing Machinery.
- Erik Harpstead, Brad A. Myers, and Vincent Aleven. In search of learning: Facilitating data analysis in educational games. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI), pages 79-88, Paris, France, April 2013. Association for Computing Machinery.
- Erik Harpstead, Christopher J. MacLellan, Vincent Aleven, and Brad A. Myers. Replay analysis in open-ended educational games. In Christian Sebastian Loh, Yanyan Sheng, and Dirk Ifenthaler, editors, Serious Games Analytics: Methodologies for Performance Measurement, Assessment, and Improvement, pages 381-399. Springer International Publishing, Cham, 2015.
- Jannicke Baalsrud Hauge, Riccardo Berta, Giusy Fiucci, Baltasar Fernández Manjón, Carmen Padrón-Nápoles, Wim Westra, and Rob Nadolski. Implications of learning analytics for serious game design. In Proceedings of the IEEE 14th International Conference on Advanced Learning Tech- nologies ((ICALT)), pages 230-232. IEEE, July 2014.
- Jannicke Balsrud Hauge, Ioana Andreea Stanescu, Antoniu Stefan, Theodore Lim, Maira B. Car- valho, Sandy Louchart, and Sylvester Arnab. Serious game mechanics and opportunities for reuse. In Proceedings of the 11th International Conference eLearning and Software for Education (eLSE), volume 2, pages 19-27, Bucharest, 2015. Carol I NDU Publishing House.
- Yuko Hirabe, Yutaka Arakawa, and Keiichi Yasumoto. Logging all the touch operations on An- droid. In Proceedings of the Seventh International Conference on Mobile Computing and Ubiquitous Networking (ICMU), pages 93-94, Singapore, January 2014. IEEE.
- Camillan Huang. Designing high-quality interactive multimedia learning modules. Computerized Medical Imaging and Graphics, 29(2):223-233, March 2005.
- Eva Hudlicka. Affective game engines: Motivation and requirements. In Proceedings of the 4th international conference on foundations of digital games, pages 299-306, Orlando, Florida, USA, January 2009. ACM.
- William Huitt. Bloom et al.'s taxonomy of the cognitive domain. Educational Psychology Interactive, 22, 2004.
- Ioanna Iacovides and Anna L. Cox. Moving beyond fun: Evaluating serious experience in digital games. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI), pages 2245-2254, New York, USA, April 2015. ACM.
- Curtis S. Ikehara, Jiecai He, and Martha E. Crosby. Issues in Implementing augmented cognition and gamification on a mobile platform. In Dylan D. Schmorrow and Cali M. Fidopiastis, editors, Foundations of Augmented Cognition, pages 685-694. Springer, Berlin, Heidelberg, 2013.
- Klaus P. Jantke. Patterns of game playing behavior as indicators of mastery. In Dirk Ifenthaler, Deniz Eseryel, and Xun Ge, editors, Assessment in Game-Based Learning: Foundations, Innovations, and Perspectives, pages 85-103. Springer, New York, NY, 2012.
- Larry Johnson, Becker S. Adams, Malcolm Cummins, Victoria Estrada, Alex Freeman, and Holly Ludgate. The NMC Horizon Report: 2013 Higher Education Edition. New Media Consortium, Austin, TX, 2013.
- Michael Johnston and Srinivas Bangalore. Finite-state multimodal integration and understanding. Natural Language Engineering, 11(2):159-187, June 2005.
- Simon Joslin, Ross Brown, and Penny Drennan. The gameplay visualization manifesto: a frame- work for logging and visualization of online gameplay data. Computers in Entertainment, 5(3):6, July 2007.
- Anne Joutsenvirta, A. Kekäläinen, M. Cankar, Titti Kallio, and Anu Kankainen. Usability testing of mobile applications: A comparison between laboratory and field testing. Journal of Usability Studies, 1(1):4-16, January 2005.
- Shiming Kai, Luc Paquette, Ryan S. Baker, Philip N. Bosch, Sidney K. D'Mello, Jaclyn Ocumpaugh, Valerie J. Shute, and Matthew Ventura. A comparison of face-based and interaction-based affect detectors in physics playground. In Proceedings of the 8th International Conference on Educational Data Mining (EDM), pages 77-84, Madrid, Spain, 2015. EDM.
- Vani Kalloo, Kinshuk, and Permanand Mohan. Personalized game based mobile learning to assist high school students with mathematics. In Proceedings of the 10th IEEE International Conference on Advanced Learning Technologies (ICALT), pages 485-487, Sousse, Tunisia, July 2010. IEEE.
- Vaiva Kalnikaite, Abigail Sellen, Steve Whittaker, and David Kirk. Now let me see where I was: Understanding how lifelogs mediate memory. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI), volume 3, pages 2045-2054, Atlanta, Georgia, June 2010. ACM.
- Ruth Kanfer and Phillip Ackerman. Motivation and cognitive abilities: An integrative/aptitude- treatment interaction approach to skill acquisition. Journal of Applied Psychology, 74(4):657-690, August 1989.
- Seungwoo Kang, Sungjun Kwon, Chungkuk Yoo, Sangwon Seo, Kwang Park, Junehwa Song, and Youngki Lee. Sinabro: Opportunistic and unobtrusive mobile ECG monitoring system. In Proceedings of the 15th Workshop on Mobile Computing Systems and Applications (HotMobile ), 11, pages 1-6, New York, NY, February 2014. ACM.
- Deirdre Kerr and Gregory K. W. K. Chung. Identifying key features of student performance in educational video games and simulations through cluster analysis. Journal of Educational Data Mining (JEDM), 4(1):144-182, October 2012.
- Deirdre S. Kerr and Gregory K. W. K. Chung. Using cluster analysis to extend usability testing to instructional content. Technical Report CRESST Report 816, National Center for Research on Evaluation, Standards, and Student Testing (CRESST), May 2012.
- Myint Swe Khine. Learning to play: Exploring the future of education with video games. Peter Lang Inc., New York, November 2010.
- Michael D. Kickmeier-Rust. Talking digital educational games. In Proceedings of the 1st Interna- tional Open Workshop on Intelligent Personalization and Adaptation in Digital Educational Games, pages 55-66. Springer, 2009.
- Michael D. Kickmeier-Rust and Dietrich Albert. An alien's guide to multi-adaptive educational computer games. Informing Science Press, February 2012.
- Michael D. Kickmeier-Rust and Dietrich Albert. Learning Analytics to support the use of virtual worlds in the classroom. In Proceedings of the International Workshop on Human-Computer Inter- action and Knowledge Discovery in Complex, Unstructured, Big Data (HCI-KDD), pages 358-365, Berlin, Heidelberg, 2013. Springer.
- Kristian Kiili, Harri Ketamo, and Michael D. Kickmeier-Rust. Eye tracking in game-based learning research and game design. International Journal of Serious Games (IJSG), 1(2):51-65, 2014.
- Kristian Kiili, Harri Ketamo, and Michael D. Kickmeier-Rust. Evaluating the usefulness of eye tracking in game-based learning. International Journal of Serious Games (IJSG), 8(2), 2014.
- Kristian Kiili, Timo Lainema, Sara de Freitas, and Sylvester Arnab. Flow framework for analyzing the quality of educational games. Entertainment Computing, 5(4):367-377, December 2014.
- Jun H. Kim, Daniel V. Gunn, Eric Schuh, Bruce Phillips, Randy J. Pagulayan, and Dennis Wixon. Tracking real-time user experience (TRUE): a Comprehensive instrumentation solution for com- plex systems. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI), pages 443-452, Florence, Italy, April 2008. Association for Computing Machinery.
- Kenneth R. Koedinger, Ryan S. J. d. Baker, Kyle Cunningham, Alida Skogsholm, Brett Leber, and John Stamper. A data repository for the EDM community: The PSLC dataShop. In Cristobal Romero, Sebastian Ventura, Mykola Pechenizkiy, and Ryan S. J. d Baker, editors, Handbook of Educational Data Mining, pages 43-55. CRC Press, Boca Raton, October 2010.
- Johannes Konert, Kristina Richter, Florian Mehm, Stefan Göbel, Regina Bruder, and Ralf Stein- metz. PEDALE -A peer education diagnostic and learning environment. Journal of Educational Technology & Society, 15(4):27-38, 2012.
- Marguerite L. Koole. Practical issues in mobile education. In Proceedings of the Fourth IEEE International Workshop on Wireless, Mobile and Ubiquitous Technology in Education (WMTE), pages 142-146, USA, November 2006. IEEE Computer Society.
- Klaus Korossy. Modeling knowledge as competence and performance. In Dietrich Albert and Josef Lukas, editors, Knowledge Spaces: Theories, Empirical Research, and Applications, pages 103-132. Psychology Press, New York, NY, August 1999.
- Kai Kunze, Masakazu Iwamura, Koichi Kise, Seiichi Uchida, and Shinichiro Omachi. Activity recognition for the mind: Toward a cognitive "Quantified Self". Computer (IEEE), 46(10):105- 108, October 2013.
- Tracey L. Leacock and John C. Nesbit. A framework for evaluating the quality of multimedia learning resources. Journal of Educational Technology & Society, 10(2):44-59, 2007.
- Hosub Lee, Young Sang Choi, Sunjae Lee, and I. P. Park. Towards unobtrusive emotion recognition for affective social communication. In Proceedings of the IEEE Consumer Communications and Networking Conference (CCNC), pages 260-264, Las Vegas, NV, January 2012. IEEE.
- James Lester, Bradford W. Mott, Jonathan P. Rowe, and Jennifer Sabourin. Learner modeling to predict real-time affect in serious games. Design Recommendations for Intelligent Tutoring Systems, 1:199-208, 2013.
- Susan M. Letourneau. Intensive data at CREATE -Research approaches & future directions. In Proceedings of the Simulations and Games Learning Analytics and Data Mining Workshop, 2013.
- Timo Leuders, Regina Bruder, Ulf Kroehne, Dominik Naccarella, Renate Nitsch, Jan Henning- Kahmann, Augustin Kelava, and Markus Wirtz. Development, validation, and application of a competence model for Mathematical problem solving by using and translating representations of functions. In Detlev Leutner, Jens Fleischer, Juliane Grünkorn, and Eckhard Klieme, editors, Competence Assessment in Education: Research, Models and Instruments, pages 389-406. Springer International Publishing, Cham, 2017.
- Roy Levy. Dynamic Bayesian network modeling of game-based diagnostic assessments. Multivari- ate Behavioral Research, 54(6):771-794, November 2019.
- Jian-Sheng Li, Yun-Xia Han, and Shan Liu. Physiological evaluation of the players' emotions in different educational games. American Journal of Educational Research, 2(9):735-739, January 2014.
- Zhung-Xun Liao, Yi-Chin Pan, Wen-Chih Peng, and Po-Ruey Lei. On mining mobile apps usage behavior for predicting apps usage in smartphones. In Proceedings of the 22nd ACM international conference on Information & Knowledge Management (CIKM), pages 609-618, San Francisco, Cali- fornia, USA, October 2013. Association for Computing Machinery.
- Florian Lingenfelser, Johannes Wagner, Elisabeth André, Gary McKeown, and Will Curran. An event driven fusion approach for enjoyment recognition in real-time. In Proceedings of the 22nd ACM International Conference on Multimedia (MM), pages 377-386, Orlando, Florida, USA, November 2014. Association for Computing Machinery.
- Christian S. Loh. Information Trails: In-process assessment of game-based learning. In Dirk Ifenthaler, Deniz Eseryel, and Xun Ge, editors, Assessment in Game-Based Learning: Foundations, Innovations, and Perspectives, pages 123-144. Springer, New York, NY, 2012.
- Christian Sebastian Loh and Yanyan Sheng. Maximum Similarity Index (MSI): A metric to differ- entiate the performance of novices vs. multiple-experts in serious games. Computers in Human Behavior, 39:322-330, October 2014.
- Christian Sebastian Loh and Yanyan Sheng. Measuring expert performance for serious games analytics: From data to insights. In Christian Sebastian Loh, Yanyan Sheng, and Dirk Ifenthaler, editors, Serious Games Analytics: Methodologies for Performance Measurement, Assessment, and Improvement, pages 101-134. Springer International Publishing, Cham, 2015.
- Christian Sebastian Loh, Yanyan Sheng, and Dirk Ifenthaler. Serious games analytics: Theoretical framework. In Christian Sebastian Loh, Yanyan Sheng, and Dirk Ifenthaler, editors, Serious Games Analytics: Methodologies for Performance Measurement, Assessment, and Improvement, pages 3-29. Springer International Publishing, Cham, 2015.
- Hong Lu, Jun Yang, Zhigang Liu, Nicholas D. Lane, Tanzeem Choudhury, and Andrew T. Camp- bell. The Jigsaw continuous sensing engine for mobile phone applications. In Proceedings of the 8th ACM Conference on Embedded Networked Sensor Systems (SenSys), pages 71-84, Zürich, Switzerland, November 2010. Association for Computing Machinery.
- Hong Lu, Denise Frauendorfer, Mashfiqui Rabbi, Marianne Schmid Mast, Gokul T. Chittaranjan, Andrew T. Campbell, Daniel Gatica-Perez, and Tanzeem Choudhury. StressSense: Detecting stress in unconstrained acoustic environments using smartphones. In Proceedings of the 2012 ACM Con- ference on Ubiquitous Computing (UbiComp), pages 351-360, Pittsburgh, Pennsylvania, September 2012. Association for Computing Machinery.
- Weiquan Lu, Chenchen Sun, Timo Bleeker, Yingdan You, Shintaro Kitazawa, and Ellen Yi-Luen Do. Sensorendipity: a real-time web-enabled smartphone sensor platform for idea generation and prototyping. In Proceedings of the Second International Symposium of Chinese CHI, pages 11-18, Toronto, Ontario, Canada, April 2014. Association for Computing Machinery.
- Simon Lucey, Ahmed Bilal Ashraf, and Jeffrey F. Cohn. Investigating spontaneous facial action recognition through AAM representations of the face. In Kresimir Delac and Mislav Grgic, editors, Face Recognition, chapter 14, pages 275-286. IntechOpen, July 2007.
- Leah P. Macfadyen and Shane Dawson. Mining LMS data to develop an early warning system for educators: A proof of concept. Computers & Education, 54(2):588-599, February 2010.
- Thomas W. Malone. What makes things fun to learn? Heuristics for designing instructional com- puter games. In Proceedings of the 3rd ACM SIGSMALL Symposium and the first SIGPC Symposium on Small Systems, pages 162-169, Palo Alto, California, USA, September 1980. Association for Computing Machinery.
- Marco Manca, Fabio Paternò, and Carmen Santoro. Adaptive mobile multimodal user interfaces for dynamic accessibility. In Proceedings of the ACM CHI Annual Conference on Human Factors in Computing Systems Extended Abstracts (CHI EA), ACM, April 2013. ACM.
- Regan L. Mandryk and M. Stella Atkins. A fuzzy physiological approach for continuously modeling emotion during interaction with play technologies. International Journal of Human-Computer Studies, 65(4):329-347, April 2007.
- Kunal Mankodiya, Vinod Sharma, Rolando Martins, Ishan Pande, Samay Jain, Neal Ryan, and Rajeev Gandhi. Understanding user's emotional engagement to the contents on a smartphone display: Psychiatric prospective. In Proceedings of the 2013 IEEE 10th International Conference on Ubiquitous Intelligence & Computing and 2013 IEEE 10th International Conference on Autonomic & Trusted Computing (UIC-ATC), pages 631-637, USA, December 2013. IEEE Computer Society.
- Raphael Marczak and Gareth R. Schott. Audio visual analysis of player experience: Feedback- based gameplay metrics. In Petri Lankoski and Staffan Bjoerk, editors, Game Research Methods, pages 207-230. ETC Press, Pittsburgh, PA, USA, January 2015.
- Raphael Marczak, Jasper van Vught, Gareth Schott, and Lennart E. Nacke. Feedback-based game- play metrics: Measuring player experience via automatic visual analysis. In Proceedings of The 8th Australasian Conference on Interactive Entertainment: Playing the System (IE), 6, pages 1-10, Auckland, New Zealand, July 2012. Association for Computing Machinery.
- Panos Markopoulos, Janet C. Read, Stuart MacFarlane, and Johanna Hoysniemi. Evaluating chil- dren's interactive products: Principles and practices for interaction designers. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, 2008.
- Bertrand Marne, John Wisdom, Benjamin Huynh-Kim-Bang, and Jean-Marc Labat. The six facets of serious game design: A methodology enhanced by our design pattern library. In Proceedings of the European Conference on Technology Enhanced Learning (ECTEL), pages 208-221, Berlin, Heidelberg, 2012. Springer.
- Nurul Hidayah Mat Zain, Fariza Hanis Abdul Razak, Azizah Jaafar, and Mohd Firdaus Zulkipli. Eye tracking in educational games environment: Evaluating user interface design through Eye tracking patterns. In Halimah Badioze Zaman, Peter Robinson, Maria Petrou, Patrick Olivier, Tim- othy K. Shih, Sergio Velastin, and Ingela Nyström, editors, Visual Informatics: Sustaining Research and Innovations, pages 64-73. Springer, Berlin, Heidelberg, 2011.
- J. Matias Kivikangas, Lennart Nacke, and Niklas Ravaja. Developing a triangulation system for dig- ital game events, observational video, and psychophysiological data to study emotional responses to a virtual character. Entertainment Computing, 2(1):11-16, January 2011.
- Igor Mayer. Towards a comprehensive methodology for the research and evaluation of serious games. Procedia Computer Science, 15:233-247, 2012.
- Igor Mayer, Geertje Bekebrede, Casper Harteveld, Harald Warmelink, Qiqi Zhou, Theo van Rui- jven, Julia Lo, Rens Kortmann, and Ivo Wenzler. The research and evaluation of serious games: Toward a comprehensive methodology. British Journal of Educational Technology, 45(3):502-507, June 2014.
- Daniel McDuff, Rana El Kaliouby, Thibaud Senechal, David Demirdjian, and Rosalind Picard. Automatic measurement of ad preferences from facial responses gathered over the internet. Image and Vision Computing, 32(10):630-640, October 2014.
- Lorna McKnight and Janet C. Read. PLU-E: a proposed framework for planning and conducting evaluation studies with children. In Proceedings of the 25th BCS Conference on Human-Computer Interaction (BCS-HCI), pages 126-131, Newcastle-upon-Tyne, United Kingdom, July 2011. BCS Learning & Development Ltd.
- Mark Mcmahon and Cesar Ojeda. A model of immersion to guide the design of serious games. In Proceedings of the World Conference on E-Learning in Corporate, Government, Healthcare, and Higher Education E-Learn, pages 1833-1842, Chesapeake, VA, January 2008. AACE.
- Donald McMillan, Alistair Morrison, and Matthew Chalmers. Categorised ethical guidelines for large scale mobile HCI. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI), pages 1853-1862, Paris, France, April 2013. Association for Computing Machinery.
- Ben Medler, Michael John, and Jeff Lane. Data Cracker: Developing a visual game analytic tool for analyzing online gameplay. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI), pages 2365-2374, Vancouver, BC, Canada, May 2011. Association for Computing Machinery.
- Florian Mehm, Viktor Wendel, Stefan Göbel, and Ralf Steinmetz. Bat Cave: A testing and evalu- ation platform for digital educational games. In Bente Meyer, editor, ECGBL2009-4th European Conference on Games-Based Learning: ECGBL 2009, pages 251-260. Academic Conferences Lim- ited, Reading, England, December 2010.
- Florian Mehm, Stefan Göbel, and Ralf Steinmetz. Authoring of serious adventure games in Story- Tec. In Stefan Göbel, Wolfgang Müller, Bodo Urban, and Josef Wiemeyer, editors, E-Learning and Games for Training, Education, Health and Sports, pages 144-154. Springer, Berlin, Heidelberg, 2012.
- Ricardo Javier Rademacher Mena. Game assesement using the E/E Grid. In Leonard Annetta and Stephen C. Bronack, editors, Serious Educational Game Assessment: Practical Methods and Models for Educational Games, Simulations and Virtual Worlds, pages 95-117. SensePublishers, Rotterdam, 2011.
- Debra K. Meyer and Julianne C. Turner. Re-conceptualizing emotion and motivation to learn in classroom contexts. Educational Psychology Review, 18(4):377-390, December 2006.
- Emiliano Miluzzo, Nicholas D. Lane, Shane B. Eisenman, and Andrew T. Campbell. CenceMe : Injecting sensing presence into social networking applications. In Proceedings of the European Conference on Smart Sensing and Context ( EuroSSC ), 1, pages 1-28, Berlin, Heidelberg, 2007. Springer.
- Miroslav Minović and Miloš Milovanović. Real-time learning analytics in educational games. In Proceedings of the First International Conference on Technological Ecosystem for Enhancing Multicul- turality (TEEM), pages 245-251, Salamanca, Spain, November 2013. Association for Computing Machinery.
- Pejman Mirza-Babaei, Günter Wallner, Graham McAllister, and Lennart E. Nacke. Unified visu- alization of quantitative and qualitative playtesting data. In Proceedings of the ACM CHI Annual Conference on Human Factors in Computing Systems Extended Abstracts (CHI EA), pages 1363- 1368. ACM, 2014.
- Robert J. Mislevy, Russell G. Almond, and Janice F. Lukas. A brief introduction to evidence- centered design. Technical Report Research Report 03-16, Educational Testing Service, Princeton, NJ, 2003.
- Konstantin Mitgutsch and Narda Alvarado. Purposeful by Design? a serious game design as- sessment framework. In Proceedings of the International Conference on the Foundations of Digital Games (FDG), pages 121-128, Raleigh, North Carolina, May 2012. Association for Computing Machinery.
- Antonija Mitrovic and Brent Martin. Evaluating the effect of open student models on self- assessment. International Journal of Artificial Intelligence in Education, 17(2):121-144, 2007.
- Pablo Moreno-Ger, Javier Torrente, Yichuan Grace Hsieh, and William T. Lester. Usability test- ing for serious games: Making informed design decisions with user data. Advances in Human- Computer Interaction (Hindawi), 2012:13-26, January 2012.
- Allistair Morrison, Paul Tennent, and Matthew Chalmers. Coordinated visualisation of video and system log data. In Proceedings of the Fourth International Conference on Coordinated & Multiple Views in Exploratory Visualization (CMV), pages 91-102, USA, July 2006. IEEE Computer Society.
- Allistair Morrison, Paul Tennent, and Matthew Chalmers. Coordinated visualisation of video and system log data. In Proceedings of the Fourth International Conference on Coordinated & Multiple Views in Exploratory Visualization (CMV), pages 91-102, USA, July 2006. IEEE Computer Society.
- Theano Moussouri and George Roussos. Mobile Sensing, BYOD and Big Data Analytics: New technologies for audience research in museums. Participations: Journal of Audience and Reception Studies, 11(1):270-285, 2014.
- Stephan Mueller, Barbara Solenthaler, Mubbasir Kapadia, Seth Frey, Severin Klingler, Richard P. Mann, Robert W. Sumner, and Markus Gross. HeapCraft: Interactive data exploration and vi- sualization tools for understanding and influencing player behavior in minecraft. In Proceedings of the 8th ACM SIGGRAPH Conference on Motion in Games (MIG), pages 237-241, Paris, France, November 2015. Association for Computing Machinery.
- Mary Muir and Cristina Conati. Understanding student attention to adaptive hints with eye- tracking. In Proceedings of the 23rd International Conference on User Modeling, Adaptation and Personalization (UMAP), pages 148-160, Berlin, Heidelberg, 2012. Springer.
- Catherine Mulwa, Seamus Lawless, Mary Sharp, Inmaculada Arnedillo-Sanchez, and Vincent Wade. Adaptive educational hypermedia systems in technology enhanced learning: a literature review. In Proceedings of the 2010 ACM Conference on Information Technology Education (SIGITE), pages 73-84, Midland, Michigan, USA, October 2010. Association for Computing Machinery.
- Lennart E. Nacke, Anders Drachen, and Stefan Göbel. Methods for evaluating gameplay experi- ence in a serious gaming context. International Journal of Computer Science in Sport, 9(2):1-12, 2010.
- Lennart E. Nacke, Sophie Stellmach, and Craig A. Lindley. Electroencephalographic assessment of player experience: A pilot study in affective ludology. Simulation & Gaming, 42(5):632-655, October 2011.
- Lennart E. Nacke, Sophie Stellmach, Dennis Sasse, Joerg Niesenhaus, and Raimund Dachselt. LAIF: A Logging and interaction framework for gaze-based interfaces in virtual entertainment environments. Entertainment Computing, 2(4):265-273, January 2011.
- Keith Nesbitt, Karen Blackmore, Geoffrey Hookham, Frances Kay-Lambkin, and Peter Walla. Using the Startle Eye-Blink to measure affect in players. In Christian Sebastian Loh, Yanyan Sheng, and Dirk Ifenthaler, editors, Serious Games Analytics: Methodologies for Performance Measurement, Assessment, and Improvement, pages 401-434. Springer International Publishing, Cham, 2015.
- Pedro A. Nogueira, Rúben Aguiar, Rui Rodrigues, Eugénio Oliveira, and Lennart E. Nacke. Fuzzy affective player models: a physiology-based hierarchical clustering method. In Proceedings of the Tenth AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment (AIIDE), pages 132-138, Raleigh, NC, USA, October 2014. AAAI Press.
- Pedro A. Nogueira, Vasco Torres, Rui Rodrigues, and Eugénio Oliveira. An annotation tool for automatically triangulating individuals' psychophysiological emotional reactions to digital media stimuli. Entertainment Computing, 9-10:19-27, June 2015.
- Elena Patricia Nunez Castellar, Kimmo Oksanen, and Jan Van Looy. Assessing game experience: Heart rate variability, in-game behavior and self-report measures. In Proceedings of The 6th Inter- national Workshop on Quality of Multimedia Experience (QoMEX), pages 292-296. IEEE, 2015.
- Kari Nurmela, Erno Lehtinen, and Tuire Palonen. Evaluating CSCL log files by social network analysis. In Proceedings of the 1999 Conference on Computer support for Collaborative Learning (CSCL), pages 54-es, Palo Alto, California, December 1999. International Society of the Learning Sciences.
- Hiroaki Ogata, Mengmeng Li, Bin Hou, Noriko Uosaki, Moushir El-Bishouty, and Yoneo Yano. SCROLL: Supporting to share and reuse ubiquitous learning log in the context of language learn- ing. Research and Practice in Technology Enhanced Learning, 6(2):69-82, January 2011.
- Paul A. O'Keefe, Susan M. Letourneau, Bruce D. Homer, Ruth N. Schwartz, and Jan L. Plass. Learn- ing from multiple representations: An examination of fixation patterns in a science simulation. Computers in Human Behavior, 35:234-242, June 2014.
- Tanner Olsen, Katelyn Procci, and Clint Bowers. Serious games usability testing: How to ensure proper usability, playability, and effectiveness. In Aaron Marcus, editor, Design, User Experience, and Usability. Theory, Methods, Tools and Practice, pages 625-634. Springer, Berlin, Heidelberg, 2011.
- Hasiah Mohamed Omar and Azizah Jaafar. Challenges in the evaluation of educational com- puter games. In Proceedings of the International Symposium on Information Technology (ITSim), volume 1, pages 11-17, Kuala Lumpur, Malaysia, June 2010. IEEE.
- Kerry O'Regan. Emotion and E-learning. Journal of Asynchronous Learning Network, 7(3):78-92, September 2003.
- Joseph C. Osborn and Michael Mateas. A game-independent play trace dissimilarity metric. In Proceedings of the 9th International Conference on the Foundations of Digital Games (FDG). Society for the Advancement of the Science of Digital Games, 2014.
- Sharon Oviatt. Ten myths of multimodal interaction. Communications of the ACM, 42(11):74-81, November 1999.
- Sharon Oviatt, Adrienne Cohen, and Nadir Weibel. Multimodal learning analytics: Description of math data corpus for ICMI grand challenge workshop. In Proceedings of the 15th ACM on Interna- tional conference on multimodal interaction (ICMI), pages 563-568, Sydney, Australia, 2013. ACM Press.
- Luc Paquette, Jonathan Rowe, Ryan Baker, Bradford Mott, James Lester, Jeanine DeFalco, Keith Brawner, Robert Sottilare, and Vasiliki Georgoulas. Sensor-free or sensor-full: A comparison of data modalities in multi-channel affect Detection. In Proceedings of the 8th International Confer- ence on Educational Data Mining (EDM), pages 93-100, Madrid, Spain, 2016. ERIC.
- Pablo Paredes, David Sun, and John Canny. Sensor-less sensing for affective computing and stress management technology. In Proceedings of the 7th International Conference on Pervasive Computing Technologies for Healthcare (PervasiveHealth), pages 459-463, Venice, Italy, May 2013. ICST (Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering).
- Georgios Patsis, Hichem Sahli, Werner Verhelst, and Olga De Troyer. Evaluation of attention lev- els in a Tetris game using a brain computer interface. In Sandra Carberry, Stephan Weibelzahl, Alessandro Micarelli, and Giovanni Semeraro, editors, User Modeling, Adaptation, and Personal- ization, pages 127-138. Springer, Berlin, Heidelberg, 2013.
- Reinhard Pekrun. The control-value theory of achievement emotions: Assumptions, corollaries, and implications for educational research and practice. Educational Psychology Review, 18(4): 315-341, December 2006.
- Reinhard Pekrun, Thomas Goetz, Wolfram Titz, and Raymond P. Perry. Academic emotions in students' self-regulated learning and achievement: A program of qualitative and quantitative research. Educational Psychologist, 37(2):91-105, June 2002.
- Reinhard Pekrun, Anne C. Frenzel, Thomas Goetz, and Raymond P. Perry. The control-value theory of achievement emotions: An integrative approach to emotions in education. In Paul A. Schutz and Reinhard Pekrun, editors, Emotion in Education, pages 13-36. Academic Press, Burlington, January 2007.
- Reinhard Pekrun, Thomas Goetz, Anne C. Frenzel, Petra Barchfeld, and Raymond P. Perry. Mea- suring emotions in students' learning and performance: The achievement emotions questionnaire (aeQ). Contemporary Educational Psychology, 36(1):36-48, January 2011.
- Gabriela Trindade Perry, Ci Costa Kulpa, Eluza Pinheiro, and Marcelo Leandro Eichler. Lessons from an educational game usability evaluation. International Journal of Interactive Mobile Tech- nologies (iJIM), 6(2):23-28, April 2012.
- Rosalind W. Picard and Shaundra Bryant Daily. Evaluating affective interactions: Alternatives to asking what users feel. In Proceedings of CHI Workshop on Evaluating Affective Interfaces: Innovative Approaches (CHI), pages 2119-2122. ACM, 2005.
- David Pinelle, Nelson Wong, and Tadeusz Stach. Heuristic evaluation for games: Usability prin- ciples for video game design. In Proceedings of the SIGCHI Conference on Human Factors in Com- puting Systems (CHI), pages 1453-1462, Florence, Italy, April 2008. Association for Computing Machinery.
- Jan Plass, Bruce Homer, and Charles Kinzer. Playful Learning: An integrated design framework. Technical Report 02/2014, Games for Learning Institute, December 2014.
- Jan L. Plass, Suzy M. Letourneau, and Bruce D. Homer. Assessment mechanics: Design of learn- ing activities that produce meaningful data. Learning Analytics and Educational Data Mining Workshop, April 2013. Published: New York University, CREATE Lab.
- Ming-Zher Poh, Daniel J. McDuff, and Rosalind W. Picard. Non-contact, automated cardiac pulse measurements using video imaging and blind source separation. Optics Express, 18(10):10762- 10774, May 2010.
- Kaśka Porayska-Pomsta, Manolis Mavrikis, Sidney D'Mello, Cristina Conati, and Ryan S. J. d. Baker. Knowledge elicitation methods for affect modelling in education. International Journal of Artificial Intelligence in Education, 22(3):107-140, July 2013.
- Marc Prensky. Digital game-based learning. Computers in Entertainment, 1(1):21, October 2003.
- Rajagopal Raghunathan and Yaacov Trope. Walking the Tightrope between feeling ood and being accurate: Mood as a resource in processing persuasive messages. Journal of Personality and Social Psychology, 83(3):510-25, October 2002.
- O.M. Ramirez and Clarence J. Dockweiler. Mathematics anxiety: A systematic review. Anxiety Stress & Coping: Advances in Test Anxiety Research, 5:157-175, 1987.
- Rajib Rana, John Reilly, Raja Jurdak, Wen Hu, Xue Li, and Jeffrey Soar. Affect sensing on smart- phone -Possibilities of understanding cognitive decline in aging population. ArXiv, 1407.5910 [cs], June 2015. arXiv: 1407.5910.
- Niklas Ravaja, Timo Saari, Mikko Salminen, Jari Laarni, and Kari Kallinen. Phasic emotional reactions to video game events: A psychophysiological investigation. Media Psychology, 8(4): 343-367, November 2006.
- C. Reuter, F. Mehm, Stefan Göbel, and R. Steinmetz. Evaluation of adaptive serious games using playtraces and aggregated play data. In Proceedings of the 7th European Conference on Games Based Learning (ECGBL), volume 2, pages 504-511, Porto, Portugal, January 2013. Academic Conferences Ltd.
- Verónica Rivera-Pelayo, Valentin Zacharias, Lars Müller, and Simone Braun. A framework for applying quantified self approaches to support reflective learning. Mobile Learning, 2012:123- 131, 2012.
- Ma. Mercedes T. Rodrigo, Ryan S. J. d. Baker, Sidney D'Mello, Ma. Celeste T. Gonzalez, Maria C. V. Lagud, Sheryl A. L. Lim, Alexis F. Macapanpan, Sheila A. M. S. Pascua, Jerry Q. Santillano, Jessica O. Sugay, Sinath Tep, and Norma J. B. Viehland. Comparing learners' affect while using an Intelligent Tutoring System and a simulation problem solving game. In Beverley P. Woolf, Esma Ameur, Roger Nkambou, and Susanne Lajoie, editors, Intelligent Tutoring Systems, pages 40-49.
- Springer, Berlin, Heidelberg, 2008.
- Elizabeth Rowe, Jodi Asbell-Clarke, and Ryan S. Baker. Serious Games Analytics to measure im- plicit science learning. In Christian Sebastian Loh, Yanyan Sheng, and Dirk Ifenthaler, editors, Serious Games Analytics: Methodologies for Performance Measurement, Assessment, and Improve- ment, pages 343-360. Springer International Publishing, Cham, 2015.
- Jonathan P. Rowe, Bradford W. Mott, and James C. Lester. Opportunities and challenges in gener- alizable sensor-based affect recognition for learning. In Proceedings of the Workshops at the 17th International Conference on Artificial Intelligence in Education (AIED), volume 6, pages 24-30, Raleigh, NC, 2015. Springer.
- Jennifer Sabourin, Jonathan P. Rowe, Bradford W. Mott, and James C. Lester. When off-task is on-task: The affective role of off-task behavior in narrative-centered learning environments. In Proceedings of the International Conference on Artificial Intelligence in Education (AIED), pages 534-536. Springer, Springer, 2011.
- Jennifer L. Sabourin and James C. Lester. Affect and engagement in game-based learning envi- ronments. IEEE Transactions on Affective Computing, 5(1):45-56, January 2014.
- Sergio Salmeron-Majadas, Olga C. Santos, and Jesus G. Boticario. Affective state detection in ed- ucational systems through mining multimodal data sources. In Proceedings of the 6th International Conference on Educational Data Mining (EDM), pages 348-349. ERIC, 2013.
- Maria Ofelia Z. San Pedro, Ryan S. J. d. Baker, and Ma. Mercedes T. Rodrigo. Carelessness and af- fect in an intelligent tutoring system for mathematics. International Journal of Artificial Intelligence in Education (IJAIED), 24(2):189-210, June 2014.
- Jesse Schell. The art of game design: A book of lenses. CRC Press, Boston, MA, August 2008.
- Stefan Scherer, Nadir Weibel, Louis-Philippe Morency, and Sharon Oviatt. Multimodal prediction of expertise and leadership in learning groups. In Proceedings of the 1st International Workshop on Multimodal Learning Analytics (MLA), 1, pages 1-8, Santa Monica, California, October 2012. Association for Computing Machinery.
- Magy Seif El-Nasr, Heather Desurvire, Lennart Nacke, Anders Drachen, Licia Calvi, Katherine Isbister, and Regina Bernhaupt. Game user research. In Proceedings of the ACM CHI Annual Conference on Human Factors in Computing Systems Extended Abstracts (CHI EA), pages 2679- 2682, Austin, Texas, USA, May 2012. Association for Computing Machinery.
- Ángel Serrano-Laguna, Eugenio J. Marchiori, Ángel del Blanco, Javier Torrente, and Baltasar Fernández-Manjón. A framework to improve evaluation in educational games. In Proceedings of the 2012 IEEE Global Engineering Education Conference (EDUCON), 148, pages 1-8, Marrakech, Morocco, April 2012. IEEE.
- Ángel Serrano-Laguna, Javier Torrente, Pablo Moreno-Ger, and Baltasar Fernández-Manjón. Ap- plication of learning analytics in educational videogames. Entertainment Computing, 5(4):313- 322, December 2014.
- Noor Shaker and Mohammad Shaker. Towards understanding the nonverbal signatures of engage- ment in Super Mario Bros. In Proceedings of the 23rd International Conference on User Modeling, Adaptation and Personalization (UMAP), pages 423-434, Cham, 2014. Springer International Pub- lishing.
- Noor Shaker, Stylianos Asteriadis, Georgios N. Yannakakis, and Kostas Karpouzis. Fusing visual and behavioral cues for modeling user experience in games. IEEE Transactions on Cybernetics, 43 (6):1519-1531, December 2013.
- David H. Shanabrook, Ivon Arroyo, and Beverly Park Woolf. Using touch as a predictor of ef- fort: What the iPad can tell us about user affective state. In Judith Masthoff, Bamshad Mobasher, Michel C. Desmarais, and Roger Nkambou, editors, User Modeling, Adaptation, and Personaliza- tion, pages 322-327. Springer, Berlin, Heidelberg, 2012.
- Liping Shen, Enrique Leon, Victor Callaghan, and Ruimin Shen. Exploratory research on an affective e-learning model. In Proceedings of the 2007 Workshop on Blended Learning (WBL), pages 267-278, Edinburgh, United Kingdom, January 2007. Academic Press. References
- Kent Shi and Kamal Ali. GetJar Mobile application recommendations with very sparse datasets. In Proceedings of the 18th ACM International Conference on Knowledge Discovery and Data Mining (SIGKDD), pages 204-212, Beijing, China, August 2012. Association for Computing Machinery.
- Ben Shneiderman. Designing for fun: How can we design user interfaces to be more fun? Inter- actions, 11(5):48-50, September 2004.
- Laila Shoukry and Stefan Göbel. Storyplay Multimodal: A research tool for the multimodal eval- uation of serious games. In Proceedings of the 11th European Conference on Games Based Learning (ECGBL), pages 577-584. Academic Conferences and Publishing International Limited, October 2017.
- Laila Shoukry and Stefan Göbel. Reasons and Responses: A multimodal serious games evaluation framework. IEEE Transactions on Emerging Topics in Computing, 8(1):245-255, January 2017.
- Laila Shoukry and Stefan Göbel. Realizing a mobile multimodal platform for serious games ana- lytics. International Journal of Serious Games (IJSG), 6(4):19-48, November 2019.
- Laila Shoukry, Stefan Göbel, and Ralf Steinmetz. Learning analytics and serious games: Trends and considerations. In Proceedings of the 2014 ACM International Workshop on Serious Games (SeriousGames), pages 21-26, Orlando, Florida, USA, November 2014. Association for Computing Machinery.
- Laila Shoukry, Stefan Göbel, and Ralf Steinmetz. Towards Mobile Multimodal Learning Analyt- ics. In Proceedings of the Learning Analytics for and in Serious Games Workshop at the European Conference of Technology Enhanced Learning (ECTEL), page 16, Darmstadt, Germany, 2014.
- Laila Shoukry, Christian Reuter, and Florian Mehm. StoryTec and StoryPlay as tools for adaptive game-based learning research. In Stefan Göbel and Josef Wiemeyer, editors, Games for Training, Education, Health and Sports, pages 195-198. Springer International Publishing, Cham, 2014.
- Laila Shoukry, Christian Sturm, Galal H. Galal-Edeen, and Stefan Göbel. Conducting evaluation studies of mobile games with preschoolers. In Proceedings of DeLFI Workshops 2014 co-located with 12th e-Learning Conference of the German Computer Society (DeLFI), volume 1227, pages 262-269, Freiburg, Germany, 2014. CEUR-WS.org.
- Laila Shoukry, Polona Caserman, Stefan Göbel, and Ralf Steinmetz. Blitzmerker: Learning idioms with a mobile game. In Stefan Göbel, Minhua Ma, Jannicke Baalsrud Hauge, Manuel Fradinho Oliveira, Josef Wiemeyer, and Viktor Wendel, editors, Serious Games, pages 77-88. Springer In- ternational Publishing, Cham, 2015.
- Laila Shoukry, Christian Sturm, and Galal H. Galal-Edeen. Pre-MEGa: A proposed framework for the design and evaluation of preschoolers' mobile educational games. In Tarek Sobh and Khaled Elleithy, editors, Innovations and Advances in Computing, Informatics, Systems Sciences, Networking and Engineering, pages 385-390. Springer International Publishing, Cham, 2015.
- Laila Shoukry, Johannes Konert, and Stefan Göbel. The evaluation of learner experience in serious games. In Imed Bouchrika, Nouzha Harrati, and Phu Vu, editors, Learner Experience and Usability in Online Education, pages 122-148. IGI Global, January 2017.
- Valerie J. Shute. Stealth assessment in computer-based games to support learning. Computer Games and Instruction, 55(2):503-524, 2011.
- Valerie J. Shute, Matthew Ventura, Malcolm Bauer, and Zapata-Rivera Diego. Melding the power of serious games and embedded assessment to monitor and foster learning: flow and grow. In
- Ute Ritterfeld, Michael Cody, and Peter Vorderer, editors, Serious Games: Mechanisms and Effects, pages 295-321. Routledge, New York, August 2009.
- George Siemens and Ryan S. J. d. Baker. Learning analytics and educational data mining: Towards communication and collaboration. In Proceedings of the 2nd International Conference on Learning Analytics and Knowledge (LAK), pages 252-254, Vancouver, British Columbia, Canada, April 2012. Association for Computing Machinery.
- George Siemens and Phil Long. Penetrating the fog: Analytics in learning and education. EDU- CAUSE Review, 46(5):30, 2011.
- Anna Skinner, Clementina Russo, Lisa Baraniecki, and Molly Maloof. Ubiquitous Augmented Cognition. In DylanD. Schmorrow and CaliM. Fidopiastis, editors, Foundations of Augmented Cognition. Advancing Human Performance and Decision-Making through Adaptive Systems, volume 8534, pages 67-77. Springer International Publishing, 2014.
- Debra J. Slone. A Methodology for measuring usability evaluation skills using the constructivist theory and the Second Life virtual world. Journal of Usability Studies, 4(4):178-188, August 2009.
- Jiqiang Song, Eugene Y. Tang, and Leibo Liu. User behavior pattern analysis and prediction based on mobile phone sensors. In Chen Ding, Zhiyuan Shao, and Ran Zheng, editors, Network and Parallel Computing, pages 177-189. Springer, Berlin, Heidelberg, 2010.
- Minzhu Song and Sujing Zhang. EFM: A model for educational game design. In Zhigeng Pan, Xiaopeng Zhang, Abdennour El Rhalibi, Woontack Woo, and Yi Li, editors, Technologies for E- Learning and Digital Entertainment, pages 509-517. Springer, Berlin, Heidelberg, 2008.
- Jacopo Staiano, María Menéndez, Alberto Battocchi, Antonella De Angeli, and Nicu Sebe. UX_Mate: from facial expressions to UX evaluation. In Proceedings of the Designing Interactive Systems Conference (DIS), pages 741-750, Newcastle Upon Tyne, United Kingdom, June 2012. Association for Computing Machinery.
- Christina Steiner, P. Hollins, Eric Kluijfhout, Mihai Dascalu, Alexander Nussbaumer, Dietrich Al- bert, and Wim Westera. Evaluation of serious games: A holistic approach. In Proceedings of the 8th International Conference of Education Research and Innovation (ICERI), pages 4334-4342, Sevilla, Spain, November 2015. IATED.
- Christina M. Steiner, Michael Kickmeier-Rust, Elke Mattheiss, Stefan Göbel, and Dietrich Albert. Balancing on a high wire: Adaptivity key factor of future learning games. In Dr Michael D. Kickmeier-Rust and Dr Dietrich Albert, editors, An Alien's Guide to Multi-Adaptive Educational Computer Games, pages 43-88. Informing Science Press, February 2012.
- Ralf Steinmetz. Synchronization properties in multimedia systems. IEEE Journal on Selected Areas in Communications, 8(3):401-412, 1990.
- Sophie Stellmach, Lennart Nacke, and Raimund Dachselt. Advanced gaze visualizations for three- dimensional virtual environments. In Proceedings of the 2010 Symposium on Eye-Tracking Research & Applications (ETRA), pages 109-112, Austin, Texas, March 2010. Association for Computing Machinery.
- Veronica Sundstedt. Gazing at Games: An introduction to eye tracking control. Morgan & Claypool Publishers, San Rafael, CA, March 2012.
- Sakari Tamminen, Antti Oulasvirta, Kalle Toiskallio, and Anu Kankainen. Understanding Mobile Contexts. Personal and Ubiquitous Computing, 8(2):135-143, May 2004.
- Chek Tien Tan, Sander Bakkes, and Yusuf Pisan. Inferring player experiences using facial expres- sions analysis. In Proceedings of the 2014 Conference on Interactive Entertainment (IE), 7, pages 1-8, Newcastle, NSW, Australia, December 2014. Association for Computing Machinery.
- Chek Tien Tan, Pejman Mirza-Babaei, Veronica Zammitto, Alessandro Canossa, Genevieve Conley, and Guenter Wallner. Tool Design Jam: Designing tools for Games User Research. In Proceedings of the 2015 Annual Symposium on Computer-Human Interaction in Play (CHI PLAY), pages 827-831, London, United Kingdom, October 2015. Association for Computing Machinery.
- Kikumi K. Tatsuoka. Architecture of knowledge structures and cognitive diagnosis: A statisti- cal pattern recognition and classification approach. In Paul D. Nichols, Susan F. Chipman, and Robert L. Brennan, editors, Cognitively Diagnostic Assessment, pages 327-359. Routledge, Hills- dale, N.J, March 1995.
- Edison Thomaz, Aman Parnami, Jonathan Bidwell, Irfan Essa, and Gregory D. Abowd. Technolog- ical approaches for addressing privacy concerns when recognizing eating behaviors with wearable cameras. In Proceedings of the 2013 ACM International Joint Conference on Pervasive and Ubiqui- tous Computing (UbiComp), pages 739-748, Zurich, Switzerland, September 2013. Association for Computing Machinery.
- Kate Thompson. Using micro-patterns of speech to predict the correctness of answers to Mathe- matics problems: An exercise in multimodal learning analytics. In Proceedings of the 15th ACM on International Conference on Multimodal Interaction (ICMI), pages 591-598, Sydney, Australia, December 2013. Association for Computing Machinery.
- Marko Tkalcic, Ante Odic, Andrej Kosir, and Jurij Tasic. Affective labeling in a content-based recommender system for images. IEEE Transactions on Multimedia, 15(2):391-400, February 2013.
- John Traxler. Defining, discussing and evaluating mobile learning: The moving finger writes and having writ . . . . The International Review of Research in Open and Distributed Learning, 8(2), June 2007.
- Jan-Paul van Staalduinen and Sara de Freitas. A game-based learning framework: Linking game design and learning outcomes. In Myint Swe Khine, editor, Learning to Play: Exploring the Future of Education with Video Games, pages 29-54. Peter Lang, New York, NY, 2011.
- Giasemi Vavoula and Mike Sharples. Meeting the challenges in evaluating mobile learning: A 3-level evaluation framework. International Journal of Mobile and Blended Learning, 1(2):54-75, 2009.
- George Veletsianos. Contextually relevant pedagogical agents: Visual appearance, stereotypes, and first impressions and their impact on learning. Computers & Education, 55(2):576-585, September 2010.
- Terry P. Vendlinski, Girlie C. Delacruz, Rebecca E. Buschang, Gregory K. W. K. Chung, and Eva L. Baker. Developing high-quality assessments that align with instructional video games. Technical Report CRESST Report 774, ERIC Clearinghouse, 2010. Google-Books-ID: SZT2vgEACAAJ.
- Adrian Vosskühler. OGAMA description (for version 2.5) : A software to record, analyze and visualize gaze and mouse movements in screen-based environments. Technical report, Freie Uni- versität Berlin, 2010. event-place: Berlin, Germany.
- Van Den W. M. Hoogen, W. A. IJsselsteijn, and De Y. A.W. Kort. Towards real-time behavioral indicators of player experiences : pressure patterns and postural responses. In Proceedings of the 6th International Conference on Methods and Techniques in Behavioral Research, pages 100-101, Maastricht, 2008. Noldus.
- G. Wallner and S. Kriglstein. Visualization-based analysis of gameplay data : A review of literature. Entertainment Computing, 4(3):143-155, August 2013.
- Althea Vail Wallop. Learning to predict disengagement in educational iPad application. Technical report, Althea Vail Wallop CS 229 Final Project Report, 2010.
- Ben George Weber, Michael John, Michael Mateas, and Arnav Jhala. Modeling player retention in Madden NFL 11. In Proceedings of the twenty-third Innovative Applications of Artificial Intelligence Conference (IAAI), San Francisco, CA, 2011. AAAI.
- Nadir Weibel, Adam Fouse, Colleen Emmenegger, Sara Kimmich, and Edwin Hutchins. Let's look at the cockpit: Exploring mobile eye-tracking for observational research on the flight deck. In Proceedings of the Symposium on Eye Tracking Research and Applications (ETRA), pages 107-114, Santa Barbara, California, March 2012. Association for Computing Machinery.
- Benjamin Weiss. Analyzing user behavior for quality assessment. In Proceedings of the International Conference Interfaces and Human Computer Interaction (IADIS ), pages 223-226. IHE, 2012.
- Grant Wiggins and Jay McTighe. The understanding by design guide to creating high-quality units. ASCD, Alexandria, Va, January 2011.
- Brian M. Winn. The design, play, and experience framework. In Richard E. Ferdig, editor, Hand- book of Research on Effective Electronic Gaming in Education Set of 3, pages 1010-1024. Informa- tion Science Reference, Hershey, PA, August 2008.
- Marcelo Worsley. Multimodal Learning Analytics as a tool for bridging learning theory and com- plex learning behaviors. In Proceedings of the 2014 ACM workshop on Multimodal Learning An- alytics Workshop and Grand Challenge (MLA), 2, pages 1-4, Istanbul, Turkey, November 2014. Association for Computing Machinery.
- Marcelo Worsley and Paulo Blikstein. Deciphering the practices and affordances of different rea- soning strategies through multimodal learning analytics. In Proceedings of the 2014 ACM workshop on Multimodal Learning Analytics Workshop and Grand Challenge (MLA), pages 21-27, Istanbul, Turkey, November 2014. Association for Computing Machinery.
- Jane Yin-Kim Yau and Mike Joy. An adaptive context-aware mobile learning framework based on the usability perspective. International Journal of Mobile Learning and Organisation, 4(4): 378-390, December 2010.
- Chungkuk Yoo, Jaemyung Shin, Inseok Hwang, and Junehwa Song. FaceLog: Capturing user's everyday face using mobile devices. In Proceedings of the 2013 ACM Conference on Pervasive and Ubiquitous Computing (UbiComp), pages 163-166, Zurich, Switzerland, September 2013. Association for Computing Machinery.
- Bieke Zaman and Vero Vanden Abeele. Towards a likeability framework that meets child-computer interaction & communication sciences. In Proceedings of the 6th International Conference on In- teraction Design and Children (IDC), 1, pages 1-8, Aalborg, Denmark, June 2007. Association for Computing Machinery.
- Veronica Zammitto, Pejman Mirza-Babaei, Ian Livingston, Marina Kobayashi, and Lennart E. Nacke. Player experience: mixed methods and reporting results. In Proceedings of the ACM CHI Annual Conference on Human Factors in Computing Systems Extended Abstracts (CHI EA), pages 147-150, Toronto, Ontario, Canada, April 2014. Association for Computing Machinery.
- Juan-Diego Zapata-Rivera and Jim E. Greer. Interacting with inspectable Bayesian student models. International Journal of Artificial Intelligence in Education (IJAIED), 14(2):127-163, April 2004.
- Panagiotis Zervas, Sergio Eduardo Gomez Ardila, Ramon Fabregat, and Demetrios G. Sampson. Tools for context-aware learning design and mobile delivery. In Proceedings of the 2011 IEEE 11th International Conference on Advanced Learning Technologies (ICALT), pages 534-535, USA, July 2011. IEEE Computer Society.
- Patrick H. Zimmerman, J. Elizabeth Bolhuis, Albert Willemsen, Erik S. Meyer, and Lucas P. J. J. Noldus. The Observer XT: A tool for the integration and synchronization of multimodal signals. Behavior Research Methods, 41(3):731-735, August 2009.
- 1 Model: Dimensions of Serious Games Evaluation. . . . . . . . . . . . . . . . . . . . . . . . . .
- 2 Model: Reasons and Responses. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
- R & R Model Applied on Learning Dimension. . . . . . . . . . . . . . . . . . . . . . . . . . . . .
- R & R Model Applied on Gaming Dimension. . . . . . . . . . . . . . . . . . . . . . . . . . . . .
- R & R Model Applied on Using Dimension. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
- 6 Possible Multimodal Data. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
- 7 From Event Logs to Multimodal Data. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
- 8 LeGUC Features for Evaluating Experience in Serious Games Testing Sessions. . . . . . . . .
- 1 StoryTec Architecture. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
- 2 Multimodal Evaluation Platform Design Architecture [237]. . . . . . . . . . . . . . . . . . . .
- 3 Mobile Capturer Component Screens before and after GamePlay. . . . . . . . . . . . . . . . .
- 4 StoryPlayMM Observer App Screens. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
- 5 StoryPlayMM Replayer Component. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
- 6 Examples of Log Statistics. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
- 7 Flagging Frames with degraded Video Quality based on Mobile Sensing Data. . . . . . . . .
- 8 Scene History, Narrative, Gaming and Learning Context Tabs in StoryPlay [165]. . . . . . .
- 9 Possible Decision Tree for Predicting Best Media Source. . . . . . . . . . . . . . . . . . . . . .
- 10 Differences in Calculating Average Time per Click. . . . . . . . . . . . . . . . . . . . . . . . . .
- 1 Evaluation Sessions at the Nursery. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
- 2 Mapping Evaluation Data to Framework Dimensions. . . . . . . . . . . . . . . . . . . . . . . .
- 3 Occurences of Congnitive, Behavioral and Context States during Playtesting Sessions. . . .
- 4 Timing of Affective and Cognitive States Related to Dimensions of In-Game Actions. . . . .
- 5 Some Screens of the Mobile Game created with StoryTec and tested with StoryPlayMM. .
- 6 Evaluation Session Setup. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
- 7 Extracting and Aligning Count Measures for Each Scene from Observer and Game Logs. .
- 8 Detecting Interesting Behavior in Logs and Investigating Corresponding Observer Data. . .
- 9 Different Relations between Game and Observer Count Measures for Different Scene Types.
- 10 Metric Significance (Distance from Average) of ClicksPerVisit (CPV) and TimePerVisit (TPV) Calculated for each Scene Type Separately (to Compare Scenes) and Correlated with Number of Observations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
- 11 Normalized Factors for ClicksPerVisit and TimePerVisit Significance Metric Correlated with ObservationCount. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
- 12 Participants in Student Evaluation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . List of Tables 3.1 A Summary of Serious Games Studies Using Multimodal Data. . . . . . . . . . . . . . . . . .
- 2 A Summary of Non-Gaming Learning Environments' Studies Using Multimodal Data. . . .
- 3 A Summary of Non-Educational Games' Studies Using Multimodal Data. . . . . . . . . . . .
- 4 A Summary of Studies Using Multimodal Data for the Evaluation of Multimedia. . . . . .
- 5 Theoretical Frameworks for Serious Games Evaluation. . . . . . . . . . . . . . . . . . . . . .
- 6 Evaluation Focus and Features of the Different Platforms. . . . . . . . . . . . . . . . . . . . .
- 7 Modalities Supported by the Different Platforms. . . . . . . . . . . . . . . . . . . . . . . . . .
- 1 Aspects to be Evaluated in a Serious Game. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
- 2 Evaluation States Identified in Literature to Occur in Interaction Environments. . . . . . .
- 1 Evaluation Participants in Different Sessions. . . . . . . . . . . . . . . . . . . . . . . . . . . . .
- 2 Devices and Software Used in Evaluation and Number of Log Files Generated. . . . . . . .
- A List of Papers Arising from this Thesis Laila Shoukry and Stefan Göbel. Reasons and Responses: A multimodal serious games evaluation frame- work. IEEE Transactions on Emerging Topics in Computing, 8(1):245-255, January 2017
- Laila Shoukry and Stefan Göbel. Realizing a mobile multimodal platform for serious games analyt- ics. International Journal of Serious Games (IJSG), 6(4):19-48, November 2019
- Laila Shoukry, Johannes Konert, and Stefan Göbel. The evaluation of learner experience in serious games. In Imed Bouchrika, Nouzha Harrati, and Phu Vu, editors, Learner Experience and Usability in Online Education, pages 122-148. IGI Global, January 2017
- Laila Shoukry and Stefan Göbel. Storyplay Multimodal: A research tool for the multimodal evalua- tion of serious games. In Proceedings of the 11th European Conference on Games Based Learning (ECGBL), pages 577-584. Academic Conferences and Publishing International Limited, October 2017
- Laila Shoukry, Stefan Göbel, and Ralf Steinmetz. Learning analytics and serious games: Trends and con- siderations. In Proceedings of the 2014 ACM International Workshop on Serious Games (SeriousGames), pages 21-26, Orlando, Florida, USA, November 2014. Association for Computing Machinery Laila Shoukry, Christian Sturm, Galal H. Galal-Edeen, and Stefan Göbel. Conducting evaluation stud- ies of mobile games with preschoolers. In Proceedings of DeLFI Workshops 2014 co-located with 12th e-Learning Conference of the German Computer Society (DeLFI), volume 1227, pages 262-269, Freiburg, Germany, 2014. CEUR-WS.org
- Laila Shoukry, Stefan Göbel, and Ralf Steinmetz. Towards Mobile Multimodal Learning Analytics. In Proceedings of the Learning Analytics for and in Serious Games Workshop at the European Conference of Technology Enhanced Learning (ECTEL), page 16, Darmstadt, Germany, 2014
- Laila Shoukry, Christian Reuter, and Florian Mehm. StoryTec and StoryPlay as tools for adaptive game- based learning research. In Stefan Göbel and Josef Wiemeyer, editors, Games for Training, Education, Health and Sports, pages 195-198. Springer International Publishing, Cham, 2014
- Laila Shoukry, Christian Sturm, and Galal H. Galal-Edeen. Pre-MEGa: A proposed framework for the design and evaluation of preschoolers' mobile educational games. In Tarek Sobh and Khaled Elleithy, ed- itors, Innovations and Advances in Computing, Informatics, Systems Sciences, Networking and Engineering, pages 385-390. Springer International Publishing, Cham, 2015
- Laila Shoukry, Polona Caserman, Stefan Göbel, and Ralf Steinmetz. Blitzmerker: Learning idioms with a mobile game. In Stefan Göbel, Minhua Ma, Jannicke Baalsrud Hauge, Manuel Fradinho Oliveira, Josef Wiemeyer, and Viktor Wendel, editors, Serious Games, pages 77-88. Springer International Publishing, Cham, 2015 the Diggers Club for Computer and Information at the German University in Cairo. 2008 | Scholarship from DAAD for writing Bachelor Thesis at Stuttgart University 2008 | Scholarship from DAAD for pursuing M.Sc. at Ulm University (not done due to child care) 2012 | Report for AINAC Conference Competition about Game Hamza (Master Thesis Project) awarded Second Place in Entrepreneurship Category 2017 | Participation at the Qomra Competition with the Short Movie "The Box" 2018 | Participation at the ALECSO Competition in Tunisia with the Mobile Game "Hamza" LANGUAGES Excellent command of Arabic both spoken and written (Mother Language) Excellent command of English both spoken and written. [16.11.2016] IELTS Score: 8: Listening:9, Reading:8, Writing:8, Speaking:7, Overall: 8. [22.06.2004] ELPET (The English Language Proficiency Entrance Test) at AUCairo, was eligible for ECLT 103/102. [06.07.2004] TOEFL computer-based Exam Score 273 (=643 paper-based) - 28/30 in Listening, 30/30 in Structure/Writing, 24/30 in Reading Excellent command of German both spoken and written. (Abitur Grade 12 in
- German) Very good command of French both spoken and written. (Abitur Grade 14 in French) Laila Shoukry, M.Sc.