What Are Cybersecurity Education Papers About?
2020, Proceedings of the 51st ACM Technical Symposium on Computer Science Education
https://doi.org/10.1145/3328778.3366816Abstract
Cybersecurity is now more important than ever, and so is education in this field. However, the cybersecurity domain encompasses an extensive set of concepts, which can be taught in different ways and contexts. To understand the state of the art of cybersecurity education and related research, we examine papers from the ACM SIGCSE and ACM ITiCSE conferences. From 2010 to 2019, a total of 1,748 papers were published at these conferences, and 71 of them focus on cybersecurity education. The papers discuss courses, tools, exercises, and teaching approaches. For each paper, we map the covered topics, teaching context, evaluation methods, impact, and the community of authors. We discovered that the technical topic areas are evenly covered (the most prominent being secure programming, network security, and offensive security), and human aspects, such as privacy and social engineering, are present as well. The interventions described in SIGCSE and ITiCSE papers predominantly focus on tertiary education in the USA. The subsequent evaluation mostly consists of collecting students' subjective perceptions via questionnaires. However, less than a third of the papers provide supplementary materials for other educators, and none of the authors published their dataset. Our results provide orientation in the area, a synthesis of trends, and implications for further research. Therefore, they are relevant for instructors, researchers, and anyone new in the field of cybersecurity education. The information we collected and synthesized from individual papers are organized in a publicly available dataset.
References (47)
- Brett A. Becker and Keith Quille. 2019. 50 Years of CS1 at SIGCSE: A Review of the Evolution of Introductory Programming Education Research. In Proceedings of the 50th ACM Technical Symposium on Computer Science Education (SIGCSE '19). ACM, New York, NY, USA, 338-344. https://doi.org/10.1145/3287324.3287432
- Tanner J. Burns, Samuel C. Rios, Thomas K. Jordan, Qijun Gu, and Trevor Un- derwood. 2017. Analysis and Exercises for Engaging Beginners in Online CTF Competitions for Security Education. In 2017 USENIX Workshop on Advances in Security Education (ASE 17). USENIX Association, Vancouver, BC, 9. https: //www.usenix.org/conference/ase17/workshop-program/presentation/burns
- Krzysztof Cabaj, Dulce Domingos, Zbigniew Kotulski, and Ana Respício. 2018. Cybersecurity education: Evolution of the discipline and analysis of master programs. Computers & Security 75 (2018), 24-35. https://doi.org/10.1016/j.cose. 2018.01.015
- Justin Cappos and Richard Weiss. 2014. Teaching the Security Mindset with Reference Monitors. In Proceedings of the 45th ACM Technical Symposium on Computer Science Education (SIGCSE '14). ACM, New York, NY, USA, 523-528. https://doi.org/10.1145/2538862.2538939
- Benedict Chukuka and Michael Locasto. 2016. A Survey of Ethical Agreements in Information Security Courses. In Proceedings of the 47th ACM Technical Sym- posium on Computing Science Education (SIGCSE '16). ACM, New York, NY, USA, 479-484. https://doi.org/10.1145/2839509.2844580
- Pranita Deshpande and Irfan Ahmed. 2019. Topological Scoring of Concept Maps for Cybersecurity Education. In Proceedings of the 50th ACM Technical Symposium on Computer Science Education (SIGCSE '19). ACM, New York, NY, USA, 731-737. https://doi.org/10.1145/3287324.3287495
- Michael H. Dunn and Laurence D. Merkle. 2018. Assessing the Impact of a National Cybersecurity Competition on Students' Career Interests. In Proceedings of the 49th ACM Technical Symposium on Computer Science Education (SIGCSE '18). ACM, New York, NY, USA, 62-67. https://doi.org/10.1145/3159450.3159462
- Serge Egelman, Julia Bernd, Gerald Friedland, and Dan Garcia. 2016. The Teaching Privacy Curriculum. In Proceedings of the 47th ACM Technical Symposium on Computing Science Education (SIGCSE '16). ACM, New York, NY, USA, 591-596. https://doi.org/10.1145/2839509.2844619
- Damjan Fujs, Anže Mihelič, and Simon L. R. Vrhovec. 2019. The Power of Interpretation: Qualitative Methods in Cybersecurity Research. In Proceedings of the 14th International Conference on Availability, Reliability and Security (ARES '19). ACM, New York, NY, USA, 92:1-92:10. https://doi.org/10.1145/3339252.3341479
- Binto George, Martha Klems, and Anna Valeva. 2013. A Method for Incorporating Usable Security into Computer Security Courses. In Proceeding of the 44th ACM Technical Symposium on Computer Science Education (SIGCSE '13). ACM, New York, NY, USA, 681-686. https://doi.org/10.1145/2445196.2445395
- Sara Hooshangi, Richard Weiss, and Justin Cappos. 2015. Can the Security Mindset Make Students Better Testers?. In Proceedings of the 46th ACM Technical Symposium on Computer Science Education (SIGCSE '15). ACM, New York, NY, USA, 404-409. https://doi.org/10.1145/2676723.2677268
- Petri Ihantola, Arto Vihavainen, Alireza Ahadi, Matthew Butler, Jürgen Börstler, Stephen H. Edwards, Essi Isohanni, Ari Korhonen, Andrew Petersen, Kelly Rivers, Miguel Ángel Rubio, Judy Sheard, Bronius Skupas, Jaime Spacco, Claudia Szabo, and Daniel Toll. 2015. Educational Data Mining and Learning Analytics in Programming: Literature Review and Case Studies. In Proceedings of the 2015 ITiCSE on Working Group Reports (ITICSE-WGR '15). ACM, New York, NY, USA, 41-63. https://doi.org/10.1145/2858796.2858798
- ISC) 2 . 2018. Cybersecurity Professionals Focus on Developing New Skills as Work- force Gap Widens. Technical Report. Cybersecurity Workforce Study.
- Ge Jin, Manghui Tu, Tae-Hoon Kim, Justin Heffron, and Jonathan White. 2018. Game Based Cybersecurity Training for High School Students. In Proceedings of the 49th ACM Technical Symposium on Computer Science Education (SIGCSE '18). ACM, New York, NY, USA, 68-73. https://doi.org/10.1145/3159450.3159591
- Association for Computing Machinery (ACM) Joint Task Force on Comput- ing Curricula and IEEE Computer Society. 2013. Computer Science Curricula 2013: Curriculum Guidelines for Undergraduate Degree Programs in Computer Science. ACM, New York, NY, USA. https://doi.org/10.1145/2534860
- Joint Task Force on Cybersecurity Education. 2017. Cybersecurity Curricular Guideline. Retrieved November 25, 2019 from http://cybered.acm.org/
- Keith S. Jones, Akbar Siami Namin, and Miriam E. Armstrong. 2018. The Core Cyber-Defense Knowledge, Skills, and Abilities That Cybersecurity Students Should Learn in School: Results from Interviews with Cybersecurity Professionals. ACM Trans. Comput. Educ. 18 (2018), 11:1-11:12. https://doi.org/10.1145/3152893
- Niakam Kazemi and Shiva Azadegan. 2010. IPsecLite: A Tool for Teaching Security Concepts. In Proceedings of the 41st ACM Technical Symposium on Computer Science Education (SIGCSE '10). ACM, New York, NY, USA, 138-142. https: //doi.org/10.1145/1734263.1734312
- Hieke Keuning, Johan Jeuring, and Bastiaan Heeren. 2018. A Systematic Literature Review of Automated Feedback Generation for Programming Exercises. ACM Trans. Comput. Educ. 19, 1 (Sept. 2018), 3:1-3:43. https://doi.org/10.1145/3231711
- Barbara Kitchenham and Stuart Charters. 2007. Guidelines for performing Sys- tematic Literature Reviews in Software Engineering. Technical Report. EBSE.
- J Richard Landis and Gary G Koch. 1977. The measurement of observer agreement for categorical data. Biometrics 33, 1 (1977), 159-174.
- Andrew Luxton-Reilly, Simon, Ibrahim Albluwi, Brett A. Becker, Michail Gian- nakos, Amruth N. Kumar, Linda Ott, James Paterson, Michael James Scott, Judy Sheard, and Claudia Szabo. 2018. Introductory Programming: A Systematic Liter- ature Review. In Proceedings Companion of the 23rd Annual ACM Conference on Innovation and Technology in Computer Science Education (ITiCSE 2018 Compan- ion). ACM, New York, NY, USA, 55-106. https://doi.org/10.1145/3293881.3295779
- Jun Ma, Jun Tao, Jean Mayo, Ching-Kuang Shene, Melissa Keranen, and Chaoli Wang. 2016. AESvisual: A Visualization Tool for the AES Cipher. In Proceedings of the 2016 ACM Conference on Innovation and Technology in Computer Science Education (ITiCSE '16). ACM, New York, NY, USA, 230-235. https://doi.org/10. 1145/2899415.2899425
- Naja A. Mack, Kevin Womack, Earl W. Huff Jr., Robert Cummings, Negus Dowling, and Kinnis Gosha. 2019. From Midshipmen to Cyber Pros: Training Minority Naval Reserve Officer Training Corp Students for Cybersecurity. In Proceedings of the 50th ACM Technical Symposium on Computer Science Education (SIGCSE '19). ACM, New York, NY, USA, 726-730. https://doi.org/10.1145/3287324.3287500
- Lauri Malmi. 2015. Can We Show an Impact? ACM Inroads 6, 1 (Feb. 2015), 30-31. https://doi.org/10.1145/2727129
- Monique Mezher and Ahmed Ibrahim. 2019. Introducing Practical SHA-1 Colli- sions to the Classroom. In Proceedings of the 50th ACM Technical Symposium on Computer Science Education (SIGCSE '19). ACM, New York, NY, USA, 879-884. https://doi.org/10.1145/3287324.3287446
- Allen Parrish, John Impagliazzo, Rajendra K. Raj, Henrique Santos, Muham- mad Rizwan Asghar, Audun Jøsang, Teresa Pereira, and Eliana Stavrou. 2018. Global Perspectives on Cybersecurity Education for 2030: A Case for a Meta- discipline. In Proceedings Companion of the 23rd Annual ACM Conference on Innovation and Technology in Computer Science Education (ITiCSE 2018 Compan- ion). ACM, New York, NY, USA, 36-54. https://doi.org/10.1145/3293881.3295778
- Kai Petersen, Robert Feldt, Shahid Mujtaba, and Michael Mattsson. 2008. Sys- tematic Mapping Studies in Software Engineering. In Proceedings of the 12th International Conference on Evaluation and Assessment in Software Engineer- ing (EASE'08). BCS Learning & Development Ltd., Swindon, UK, 68-77. http: //dl.acm.org/citation.cfm?id=2227115.2227123
- Kai Petersen, Sairam Vakkalanka, and Ludwik Kuzniarz. 2015. Guidelines for conducting systematic mapping studies in software engineering: An update. Information and Software Technology 64 (2015), 1 -18. https://doi.org/10.1016/j. infsof.2015.03.007
- Computing Research and Education Association of Australasia. 2016. CORE. Retrieved November 25, 2019 from http://www.core.edu.au/
- Khaled Salah. 2014. Harnessing the Cloud for Teaching Cybersecurity. In Proceed- ings of the 45th ACM Technical Symposium on Computer Science Education (SIGCSE '14). ACM, New York, NY, USA, 529-534. https://doi.org/10.1145/2538862.2538880
- Michael Skirpan, Jacqueline Cameron, and Tom Yeh. 2018. Quantified Self: An Interdisciplinary Immersive Theater Project Supporting a Collaborative Learning Environment for CS Ethics. In Proceedings of the 49th ACM Technical Symposium on Computer Science Education (SIGCSE '18). ACM, New York, NY, USA, 946-951. https://doi.org/10.1145/3159450.3159574
- Madiha Tabassum, Stacey Watson, Bill Chu, and Heather Richter Lipford. 2018. Evaluating Two Methods for Integrating Secure Programming Education. In Proceedings of the 49th ACM Technical Symposium on Computer Science Educa- tion (SIGCSE '18). ACM, New York, NY, USA, 390-395. https://doi.org/10.1145/ 3159450.3159511
- Blair Taylor and Siddharth Kaza. 2011. Security Injections: Modules to Help Students Remember, Understand, and Apply Secure Coding Techniques. In Proceedings of the 16th Annual Joint Conference on Innovation and Technology in Computer Science Education (ITiCSE '11). ACM, New York, NY, USA, 3-7. https://doi.org/10.1145/1999747.1999752
- Clark Taylor, Pablo Arias, Jim Klopchic, Celeste Matarazzo, and Evi Dube. 2017. CTF: State-of-the-Art and Building the Next Generation. In 2017 USENIX Work- shop on Advances in Security Education (ASE 17). USENIX Association, Van- couver, BC, 11. https://www.usenix.org/conference/ase17/workshop-program/ presentation/taylor
- Cynthia Taylor and Saheel Sakharkar. 2019. ');
- DROP TABLE Textbooks;-: An Argument for SQL Injection Coverage in Database Textbooks. In Proceedings of the 50th ACM Technical Symposium on Computer Science Education (SIGCSE '19). ACM, New York, NY, USA, 191-197. https://doi.org/10.1145/3287324.3287429
- Maxim Timchenko and David Starobinski. 2015. A Simple Laboratory Environ- ment for Real-World Offensive Security Education. In Proceedings of the 46th ACM Technical Symposium on Computer Science Education (SIGCSE '15). ACM, New York, NY, USA, 657-662. https://doi.org/10.1145/2676723.2677225
- Giovanni Vigna, Kevin Borgolte, Jacopo Corbetta, Adam Doupé, Yanick Fratan- tonio, Luca Invernizzi, Dhilung Kirat, and Yan Shoshitaishvili. 2014. Ten Years of iCTF: The Good, The Bad, and The Ugly. In 2014 USENIX Summit on Gam- ing, Games, and Gamification in Security Education (3GSE 14). USENIX Associ- ation, San Diego, CA, 7. https://www.usenix.org/conference/3gse14/summit- program/presentation/vigna
- James Walker, Man Wang, Steven Carr, Jean Mayo, and Ching-Kuang Shene. 2019. Teaching Integer Security Using Simple Visualizations. In Proceedings of the 2019 ACM Conference on Innovation and Technology in Computer Science Education (ITiCSE '19). ACM, New York, NY, USA, 513-519. https://doi.org/10. 1145/3304221.3319760
- Man Wang, Steve Carr, Jean Mayo, Ching-Kuang Shene, and Chaoli Wang. 2014. MLSvisual: A Visualization Tool for Teaching Access Control Using Multi-level Security. In Proceedings of the 2014 Conference on Innovation and Technology in Computer Science Education (ITiCSE '14). ACM, New York, NY, USA, 93-98. https://doi.org/10.1145/2591708.2591730
- Man Wang, Jean Mayo, Ching-Kuang Shene, Steve Carr, and Chaoli Wang. 2017. UNIXvisual: A Visualization Tool for Teaching UNIX Permissions. In Proceedings of the 2017 ACM Conference on Innovation and Technology in Computer Science Education (ITiCSE '17). ACM, New York, NY, USA, 194-199. https://doi.org/10. 1145/3059009.3059031
- Man Wang, Jean Mayo, Ching-Kuang Shene, Thomas Lake, Steve Carr, and Chaoli Wang. 2015. RBACvisual: A Visualization Tool for Teaching Access Control Using Role-based Access Control. In Proceedings of the 2015 ACM Conference on Innovation and Technology in Computer Science Education (ITiCSE '15). ACM, New York, NY, USA, 141-146. https://doi.org/10.1145/2729094.2742627
- Michael Whitney, Heather Lipford-Richter, Bill Chu, and Jun Zhu. 2015. Em- bedding Secure Coding Instruction into the IDE: A Field Study in an Ad- vanced CS Course. In Proceedings of the 46th ACM Technical Symposium on Computer Science Education (SIGCSE '15). ACM, New York, NY, USA, 60-65. https://doi.org/10.1145/2676723.2677280
- Le Xu, Dijiang Huang, and Wei-Tek Tsai. 2012. V-lab: A Cloud-based Virtual Laboratory Platform for Hands-on Networking Courses. In Proceedings of the 17th ACM Annual Conference on Innovation and Technology in Computer Science Education (ITiCSE '12). ACM, New York, NY, USA, 256-261. https://doi.org/10. 1145/2325296.2325357
- Maximilian Zinkus, Oliver Curry, Marina Moore, Zachary Peterson, and Zoë J. Wood. 2019. Fakesbook: A Social Networking Platform for Teaching Security and Privacy Concepts to Secondary School Students. In Proceedings of the 50th ACM Technical Symposium on Computer Science Education (SIGCSE '19). ACM, New York, NY, USA, 892-898. https://doi.org/10.1145/3287324.3287486
- Valdemar Švábenský, Jan Vykopal, and Pavel Čeleda. 2019. Dataset: What Are Cybersecurity Education Papers About? A Systematic Literature Review of SIGCSE and ITiCSE Conferences. Zenodo. https://doi.org/10.5281/zenodo.3506640