Academia.eduAcademia.edu

Outline

Effect of video styles on learner engagement in MOOCs

2023, Technology, Pedagogy and Education

https://doi.org/10.1080/1475939X.2023.2246981

Abstract

Video lectures in massive open online courses (MOOCs) provide an opportunity to not only deliver instructional content but also engage learners. While there are many different styles of video lectures, it is not clear how video styles affect learner engagement. This study analysed and critiqued different typologies of video styles and classified MOOC video styles on a speaker-centric to media-centric spectrum. A total of 1372 survey responses were used for data analysis. The findings indicated that the ‘media-centric’ and ‘balanced’ video styles enhanced learner engagement to varying degrees in MOOCs of different study areas. In contrast, the ‘speaker-centric’ video style offered no advantages for promoting engagement in any MOOC study area. Effect sizes ranged from .03 to .07, indicating that video styles had a small to medium effect on engagement. These findings can provide new insights into the design of video lectures for different study areas in MOOCs.

References (80)

  1. Alario-Hoyos, C., Muñoz-Merino, P. J., Pérez-Sanagustín, M., Delgado Kloos, C., & Parada, G. A. H. (2016). Who are the top contributors in a MOOC? Relating participants' performance and contributions. Journal of Computer Assisted Learning, 32(3), 232-243. https://doi.org/10.1111/jcal.12127
  2. Alemayehu, L., & Chen, H.-L. (2021). Learner and instructor-related challenges for learners' engagement in MOOCs: A review of 2014-2020 publications in selected SSCI indexed journals. Interactive Learning Environments, 31(5), 3172-3194. Advance online publication. https://doi.org/10.1080/10494820.2021.1920430
  3. Alemdag, E. (2022). Effects of instructor-present videos on learning, cognitive load, motivation, and social presence: A meta-analysis. Education and Information Technologies, 27(9), 12713-12742. https://doi.org/10.1007/s10639-022- 11154-w Appleton, J. J., Christenson, S. L., Kim, D., & Reschly, A. L. (2006). Measuring cognitive and psychological engagement: Validation of the student engagement instrument. Journal of School Psychology, 44(5), 427-445. https://doi.org/10. 1016/j.jsp.2006.04.002
  4. Baker, R., Xu, D., Park, J., Yu, R., Li, Q., Cung, B., Fischer, C., Rodriguez, F., Warschauer, M., & Smyth, P. (2020). The benefits and caveats of using clickstream data to understand student self-regulatory behaviors: Opening the black box of learning processes. International Journal of Educational Technology in Higher Education, 17(1), 13. https://doi.org/10. 1186/s41239-020-00187-1
  5. Carmichael, M., Reid, A.-K., & Karpicke, J. (2018). Assessing the impact of educational video on student engagement, critical thinking and learning: The current state of play. SAGE.
  6. Chen, Y., Gao, Q., Yuan, Q., & Tang, Y. (2019). Facilitating students' interaction in MOOCs through timeline-anchored discussion. International Journal of Human-Computer Interaction, 35(19), 1781-1799. https://doi.org/10.1080/ 10447318.2019.1574056
  7. Chen, H.-T. M., & Thomas, M. (2020). Effects of lecture video styles on engagement and learning. Educational Technology Research & Development, 68(5), 2147-2164. https://doi.org/10.1007/s11423-020-09757-6
  8. Chen, C. J., Wong, V. S., Teh, C. S., & Chuah, K. M. (2017). MOOC videos-derived emotions. Journal of Telecommunication, Electronic and Computer Engineering, 9(2-9), 137-140. https://jtec.utem.edu.my/jtec/article/view/2688
  9. Chorianopoulos, K. (2018). A taxonomy of asynchronous instructional video styles. International Review of Research in Open & Distributed Learning, 19(1), 295-311. https://doi.org/10.19173/irrodl.v19i1.2920
  10. Cobos, R., & Ruiz-Garcia, J. C. (2021). Improving learner engagement in MOOCs using a learning intervention system: A research study in engineering education. Computer Applications in Engineering Education, 29(4), 733-749. https:// doi.org/10.1002/cae.22316
  11. Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Lawrence Erlbaum Associates.
  12. Comer, D. K., Baker, R., & Wang, Y. (2015). Negativity in massive online open courses: Impacts on learning and teaching and how instructional teams may be able to address it. InSight: A Journal of Scholarly Teaching, 10, 92-113. https:// doi.org/10.46504/10201508co
  13. Costley, J., & Lange, C. H. (2017). Video lectures in e-learning. Interactive Technology & Smart Education, 14(1), 14-30. https://doi.org/10.1108/ITSE-08-2016-0025
  14. Daniels, L. M., Adams, C., & McCaffrey, A. (2016). Emotional and social engagement in a massive open online course: An examination of Dino 101. In M. P. McCreery (Ed.), Emotions, technology, and learning (pp. 25-41). Academic Press.
  15. Dart, S. (2020, December 6-9). Khan-style video engagement in undergraduate engineering: Influence of video duration, content type and course. Proceedings of the 31st Annual Conference of the Australasian Association for Engineering Education, Sydney, Australia.
  16. Dart, S., & Gregg, A. (2021, December). Know your stuff, show enthusiasm, keep it on message: Factors influencing video engagement in two mechanical engineering courses. Proceedings of the Research in Engineering Education Symposium and Australasian Association for Engineering Education Annual Conference, Australia.
  17. Deng, R., & Benckendorff, P. (2021). What are the key themes associated with the positive learning experience in MOOCs? An empirical investigation of learners' ratings and reviews. International Journal of Educational Technology in Higher Education, 18(1), 9. https://doi.org/10.1186/s41239-021-00244-3
  18. Deng, R., Benckendorff, P., & Gannaway, D. (2019). Progress and new directions for teaching and learning in MOOCs. Computers & Education, 129, 48-60. https://doi.org/10.1016/j.compedu.2018.10.019
  19. Deng, R., Benckendorff, P., & Gannaway, D. (2020). Learner engagement in MOOCs: Scale development and validation. British Journal of Educational Technology, 51(1), 245-262. https://doi.org/10.1111/bjet.12810
  20. Deng, R., & Gao, Y. (2023a). Effects of embedded questions in pre-class videos on learner perceptions, video engage- ment, and learning performance in flipped classrooms. Active Learning in Higher Education, Advance online publica- tion. https://doi.org/10.1177/14697874231167098
  21. R. DENG
  22. Deng, R., & Gao, Y. (2023b). A review of eye tracking research on video-based learning. Education and Information Technologies, 28(6), 7671-7702. https://doi.org/10.1007/s10639-022-11486-7
  23. Deng, R., & Gao, Y. (2023c). Using learner reviews to inform instructional video design in MOOCs. Behavioral Sciences, 13 (4), 330. https://doi.org/10.3390/bs13040330
  24. Endres, T., Weyreter, S., Renkl, A., & Eitel, A. (2020). When and why does emotional design foster learning? Evidence for situational interest as a mediator of increased persistence. Journal of Computer Assisted Learning, 36(4), 514-525. https://doi.org/10.1111/jcal.12418
  25. Gallego-Romero, J. M., Alario-Hoyos, C., Estévez-Ayres, I., & Delgado Kloos, C. (2020). Analyzing learners' engagement and behavior in MOOCs on programming with the Codeboard IDE. Educational Technology Research & Development, 68(5), 2505-2528. https://doi.org/10.1007/s11423-020-09773-6
  26. Greene, J. A., Oswald, C. A., & Pomerantz, J. (2015). Predictors of retention and achievement in a massive open online course. American Educational Research Journal, 52(5), 925-955. https://doi.org/10.3102/0002831215584621
  27. Guo, P. J., Kim, J., & Rubin, R. (2014, March). How video production affects student engagement: An empirical study of MOOC videos. The first ACM conference on learning at scale, Atlanta, GA.
  28. Guzmán-Valenzuela, C., Gómez-González, C., Rojas-Murphy Tagle, A., & Lorca-Vyhmeister, A. (2021). Learning analytics in higher education: A preponderance of analytics but very little learning? International Journal of Educational Technology in Higher Education, 18(1), 23. https://doi.org/10.1186/s41239-021-00258-x
  29. Haensel, J. X., Smith, T. J., & Senju, A. (2022). Cultural differences in mutual gaze during face-to-face interactions: A dual head-mounted eye-tracking study. Visual Cognition, 30(1-2), 100-115. https://doi.org/10.1080/13506285.2021.1928354
  30. Hair, J. F., Black, W. C., Babin, B. J., & Anderson, R. E. (2014). Multivariate data analysis: Pearson new international edition. Pearson.
  31. Hansch, A., McConachie, K., Schmidt, P., Hillers, L., Newman, C., & Schildhauer, T. (2015). The role of video in online learning: Findings from the field and critical reflections. SSRN Electronic Journal. https://doi.org/10.2139/ssrn.2577882
  32. Hoi, V. N., & Le Hang, H. (2021). The structure of student engagement in online learning: A bi-factor exploratory structural equation modelling approach. Journal of Computer Assisted Learning, 37(4), 1141-1153. https://doi.org/10.1111/jcal.12551
  33. Hong, J., Pi, Z., & Yang, J. (2018). Learning declarative and procedural knowledge via video lectures: Cognitive load and learning effectiveness. Innovations in Education and Teaching International, 55(1), 74-81. https://doi.org/10.1080/ 14703297.2016.1237371
  34. Hughes, C., Costley, J., & Lange, C. (2019). The effects of multimedia video lectures on extraneous load. Distance Education, 40(1), 54-75. https://doi.org/10.1080/01587919.2018.1553559
  35. Hu, H., Zhang, G., Gao, W., & Wang, M. (2020). Big data analytics for MOOC video watching behavior based on Spark. Neural Computing and Applications, 32(11), 6481-6489. https://doi.org/10.1007/s00521-018-03983-z
  36. Jiang, Y., & Peng, J.-E. (2023). Exploring the relationships between learners' engagement, autonomy, and academic performance in an English language MOOC. Computer Assisted Language Learning, 1-26. Advance online publication. https://doi.org/10.1080/09588221.2022.2164777
  37. Jung, E., Kim, D., Yoon, M., Park, S., & Oakley, B. (2019). The influence of instructional design on learner control, sense of achievement, and perceived effectiveness in a supersize MOOC course. Computers & Education, 128, 377-388. https:// doi.org/10.1016/j.compedu.2018.10.001
  38. Jung, Y., & Lee, J. (2018). Learning engagement and persistence in Massive Open Online Courses (MOOCS). Computers & Education, 122, 9-22. https://doi.org/10.1016/j.compedu.2018.02.013
  39. Kay, R. H. (2012). Exploring the use of video podcasts in education: A comprehensive review of the literature. Computers in Human Behavior, 28(3), 820-831. https://doi.org/10.1016/j.chb.2012.01.011
  40. Kennedy, G., Coffrin, C., de Barba, P., & Corrin, L. (2015, March). Predicting success: How learners' prior knowledge, skills and activities predict MOOC performance. 5th International Conference on Learning Analytics and Knowledge, New York.
  41. Kim, J., Kwon, Y., & Cho, D. (2011). Investigating factors that influence social presence and learning outcomes in distance higher education. Computers & Education, 57(2), 1512-1520. https://doi.org/10.1016/j.compedu.2011.02.005
  42. Kuo, T. M., Tsai, C.-C., & Wang, J.-C. (2021). Linking web-based learning self-efficacy and learning engagement in MOOCs: The role of online academic hardiness. The Internet and Higher Education, 51, 100819. https://doi.org/10.1016/j. iheduc.2021.100819
  43. Lackmann, S., Léger, P.-M., Charland, P., Aubé, C., & Talbot, J. (2021). The influence of video format on engagement and performance in online learning. Brain Sciences, 11(2), 128. https://doi.org/10.3390/brainsci11020128
  44. Lallé, S., & , and Conati, C. (2020). A data-driven student model to provide adaptive support during video watching across MOOCs. In I. I. Bittencourt, M. Cukurova, K. Muldner, R. Luckin, & E. Millán, Artificial intelligence in education (pp. 282-295). Springer. https://doi.org/10.1007/978-3-030-52237-7_23
  45. Lange, C., & Costley, J. (2020). Improving online video lectures: Learning challenges created by media. International Journal of Educational Technology in Higher Education, 17(1), 16. https://doi.org/10.1186/s41239-020-00190-6
  46. Lan, M., & Hew, K. F. (2020). Examining learning engagement in MOOCs: A self-determination theoretical perspective using mixed method. International Journal of Educational Technology in Higher Education, 17(1), 7. https://doi.org/10. 1186/s41239-020-0179-5
  47. Lee, Y. (2018). Effect of uninterrupted time-on-task on students' success in Massive Open Online Courses (MOOCs). Computers in Human Behavior, 86, 174-180. https://doi.org/10.1016/j.chb.2018.04.043
  48. Lemay, D. J., & Doleck, T. (2022). Predicting completion of Massive Open Online Course (MOOC) assignments from video viewing behavior. Interactive Learning Environments, 30(10), 1782-1793. https://doi.org/10.1080/10494820.2020. 1746673
  49. Li, K. (2019). MOOC learners' demographics, self-regulated learning strategy, perceived learning and satisfaction: A structural equation modeling approach. Computers & Education, 132, 16-30. https://doi.org/10.1016/j.compedu. 2019.01.003
  50. Li, Q., & Baker, R. (2018). The different relationships between engagement and outcomes across participant subgroups in Massive Open Online Courses. Computers & Education, 127, 41-65. https://doi.org/10.1016/j.compedu.2018.08.005
  51. Majid, S., Khine, W. K. K., Oo, M. Z. C., & Lwin, Z. M. (2012 , November). An analysis of YouTube videos for teaching information literacy skills. Proceedings of the International Conference on Computers and Advanced Technology in Education (ICCAT 2001). Beijing, China.
  52. Manuel, F., Maur, A., Weiser, C., & Winkel, K. (2021). Pre-class video watching fosters achievement and knowledge retention in a flipped classroom. Computers & Education, 179, 104399. https://doi.org/10.1016/j.compedu.2021. 104399
  53. Maroco, J., Maroco, A. L., Campos, J. A. D. B., & Fredricks, J. A. (2016). University student's engagement: Development of the University Student Engagement Inventory (USEI). Psicologia: Reflexão e Crítica, 29(1), 1-12. https://doi.org/10. 1186/s41155-016-0042-8
  54. Mayer, R. E., & Moreno, R. (2002). Animation as an aid to multimedia learning. Educational Psychology Review, 14(1), 87-99. https://doi.org/10.1023/A:1013184611077
  55. Noetel, M., Griffith, S., Delaney, O., Sanders, T., Parker, P., Del Pozo Cruz, B., & Lonsdale, C. (2021). Video improves learning in higher education: A systematic review. Review of Educational Research, 91(2), 204-236. https://doi.org/10.3102/ 0034654321990713
  56. Perna, L. W., Ruby, A., Boruch, R. F., Wang, N., Scull, J., Ahmad, S., & Evans, C. (2014). Moving through MOOCs: Understanding the progression of users in massive open online courses. Educational Researcher, 43(9), 421-432. https://doi.org/10.3102/0013189x14562423
  57. Pi, Z., Chen, M., Zhu, F., Yang, J., & Hu, W. (2022). Modulation of instructor's eye gaze by facial expression in video lectures. Innovations in Education and Teaching International, 59(1), 15-23. https://doi.org/10.1080/14703297.2020. 1788410
  58. Pi, Z., Xu, K., Liu, C., & Yang, J. (2020). Instructor presence in video lectures: Eye gaze matters, but not body orientation. Computers & Education, 144, 103713. https://doi.org/10.1016/j.compedu.2019.103713
  59. Plass, J. L., & Kaplan, U. (2016). Emotional design in digital media for learning. In S. Tettegah & M. Gartmeier (Eds.), Emotions, technology, design, and learning (pp. 131-162). Academic Press.
  60. Pursel, B. K., Zhang, L., Jablokow, K. W., Choi, G. W., & Velegol, D. (2016). Understanding MOOC students: Motivations and behaviours indicative of MOOC completion. Journal of Computer Assisted Learning, 32, 202-217. https://doi.org/10.1111/ jcal.12131
  61. Romero-Rodríguez, L. M., Ramírez-Montoya, M. S., & González, J. R. V. (2019). Gamification in MOOCs: Engagement application test in energy sustainability courses. IEEE Access, 7, 32093-32101. https://doi.org/10.1109/access.2019. 2903230
  62. Ruipérez-Valiente, J. A., Staubitz, T., Jenner, M., Halawa, S., Zhang, J., Despujol, I., Maldonado-Mahauad, J., Montoro, G., Peffer, M., Rohloff, T., Lane, J., Turro, C., Li, X., Pérez-Sanagustín, M., & Reich, J. (2022). Large scale analytics of global and regional MOOC providers: Differences in learners' demographics, preferences, and perceptions. Computers & Education, 180, 104426. https://doi.org/10.1016/j.compedu.2021.104426
  63. Santos Espino, J. M., Afonso Suárez, M. D., & González-Henríquez, J. J. (2020). Video for teaching: Classroom use, instructor self-production and teachers' preferences in presentation format. Technology, Pedagogy & Education, 29 (2), 147-162. https://doi.org/10.1080/1475939x.2020.1726805
  64. Santos Espino, J. M., Afonso Suárez, M. D., & Guerra Artal, C. (2016). Speakers and boards: A survey of instructional video styles in MOOCs. Technical Communication, 63(2), 101-115. https://www.ingentaconnect.com/content/stc/tc/2016/ 00000063/00000002/art00004
  65. Sanz-Martínez, L., Er, E., Martínez-Monés, A., Dimitriadis, Y., & Bote-Lorenzo, M. L. (2019). Creating collaborative groups in a MOOC: A homogeneous engagement grouping approach. Behaviour & Information Technology, 38(11), 1107-1121. https://doi.org/10.1080/0144929x.2019.1571109
  66. Semenova, T. (2022). The role of learners' motivation in MOOC completion. Open Learning: The Journal of Open, Distance & E-Learning, 37(3), 273-287. https://doi.org/10.1080/02680513.2020.1766434
  67. Seo, K., Dodson, S., Harandi, N. M., Roberson, N., Fels, S., & Roll, I. (2021). Active learning with online video: The impact of learning context on engagement. Computers & Education, 165. https://doi.org/10.1016/j.compedu.2021.104132
  68. Shah, D. (2022). By the numbers: MOOCs in 2021. https://www.classcentral.com/report/mooc-stats-2021/ Smith, K. B. (2002). Typologies, taxonomies, and the benefits of policy classification. Policy Studies Journal, 30(3), 379-395. https://doi.org/10.1111/j.1541-0072.2002.tb02153.x
  69. Stöhr, C., Stathakarou, N., Mueller, F., Nifakos, S., & McGrath, C. (2019). Videos as learning objects in MOOCs: A study of specialist and non-specialist participants' video activity in MOOCs. British Journal of Educational Technology, 50(1), 166-176.
  70. Sun, Y., Guo, Y., & Zhao, Y. (2020). Understanding the determinants of learner engagement in MOOCs: An adaptive structuration perspective. Computers & Education, 157, 103963. https://doi.org/10.1016/j.compedu.2020.103963
  71. Tawfik, A. A., Reeves, T. D., Stich, A. E., Gill, A., Hong, C., McDade, J., Pillutla, V. S., Zhou, X., & Giabbanelli, P. J. (2017). The nature and level of learner-learner interaction in a chemistry Massive Open Online Course (MOOC). Journal of Computing in Higher Education, 29(3), 411-431. https://doi.org/10.1007/s12528-017-9135-3
  72. Thornhill, S., Asensio, M., & Young, C. (2002). Video streaming: A guide for educational development. JISC Click and Go Video Project.
  73. van Wermeskerken, M., Ravensbergen, S., & van Gog, T. (2018). Effects of instructor presence in video modeling examples on attention and learning. Computers in Human Behavior, 89, 430-438. https://doi.org/10.1016/j.chb. 2017.11.038
  74. Walji, S., Deacon, A., Small, J., & Czerniewicz, L. (2016). Learning through engagement: MOOCs as an emergent form of provision. Distance Education, 37(2), 208-223. https://doi.org/10.1080/01587919.2016.1184400
  75. Wang, J., Antonenko, P., & Dawson, K. (2020). Does visual attention to the instructor in online video affect learning and learner perceptions? An eye-tracking analysis. Computers & Education, 146, 1-16. https://doi.org/10.1016/j.compedu. 2019.103779
  76. Wang, W., Guo, L., He, L., & Wu, Y. J. (2019). Effects of social-interactive engagement on the dropout ratio in online learning: Insights from MOOC. Behaviour & Information Technology, 38(6), 621-636. https://doi.org/10.1080/ 0144929x.2018.1549595
  77. Watson, W. R., Kim, W., & Watson, S. L. (2016). Learning outcomes of a MOOC designed for attitudinal change: A case study of an Animal Behavior and Welfare MOOC. Computers & Education, 96, 83-93. https://doi.org/10.1016/j. compedu.2016.01.013
  78. Wei, X., Saab, N., & Admiraal, W. (2021). Assessment of cognitive, behavioral, and affective learning outcomes in massive open online courses: A systematic literature review. Computers & Education, 163, 104097. https://doi.org/10.1016/j. compedu.2020.104097
  79. Wilson, K. E., Martinez, M., Mills, C., D'Mello, S., Smilek, D., & Risko, E. F. (2018). Instructor presence effect: Liking does not always lead to learning. Computers & Education, 122, 205-220. https://doi.org/10.1016/j.compedu.2018.03.011
  80. Xu, X., Zhu, X., & Chan, F. M. (2023). System design of Pintrich's SRL in a supervised-PLE platform: A pilot test in higher education. Interactive Learning Environments, 31(2), 683-700. https://doi.org/10.1080/10494820.2020.1802296