Review of Digital Badges in Computer Science Literature and Applications
Abstract
In the increasingly researched field of gamification, that is the application of the game design elements in no game contexts, one of the most used techniques is the use of digital badges. Through this paper, we conducted a literature review on the use and effectiveness of badges. We include other literature reviews, empirical case studies, and application reports. This analysis shows the widespread use, the different types and specialized nature of digital badges throughout the private and public sectors.
References
[1] S. Abramovich, C. Schunn, and R. M. Higashi. Are badges useful in education?: It depends upon the type of badge and expertise of learner. Educational Technology Research and Development, 61(2):217–232, 2013.
[2] S. S. Abrams and S. Walsh. Gamified vocabulary. Journal of Adolescent & Adult Literacy, 58(1):49–58, 2014.
[3] I. Ajzen. The theory of planned behavior. Organizational behavior and human decision processes, 50(2):179–211, 1991.
[4] D. Akbulut and G. Yıldırım. The usage of green gamification for public interest function of public relations. CTC 2019, 2019.
[5] R. S. Alsawaier. The effect of gamification on motivation and engagement. The International Journal of Information and Learning Technology, 35(1):56 – 79, 2018.
[6] A. Amriani, A. F. Aji, A. Y. Utomo, and K. M. Junus. An empirical study of gamification impact on e-learning environment. In Proceedings of 2013 3rd International Conference on Computer Science and Network Technology, pages 265 – 269. IEEE, 2013.
[7] A. Anderson, D. Huttenlocher, J. Kleinberg, and J. Leskovec. Steering user behavior with badges. In Proceedings of the 22nd International Conference on World Wide Web, pages 95–106. ACM, 2013.
[8] G. Barata, S. Gama, J. Jorge, and D. Gon¸calves. Engaging engineering students with gamification. In 2013 5th International Conference on Games and Virtual Worlds for Serious Applications (VS-GAMES), pages 1–8. IEEE, 2013.
[9] S. K. Bista, S. Nepal, N. Colineau, and C. Paris. Using gamification in an online community. In 8th International Conference on Collaborative Computing: Networking, Applications and Worksharing (CollaborateCom), pages 611–618. IEEE, 2012.
[10] H. Cavusoglu, Z. Li, and K.-W. Huang. Can gamification motivate volun-tary contributions?: the case of Stackoverflow Q&A community. In Proceedings of the 18th ACM Conference Companion on Computer Supported Cooperative Work & Social Computing, pages 171–174. ACM, 2015.
[11] L. De-Marcos, A. Dom´ınguez, J. Saenz-de Navarrete, and C. Pag´es. An empirical study comparing gamification and social networking on e-learning. Computers & Education, 75:82–91, 2014.
[12] S. Deterding, D. Dixon, R. Khaled, and L. Nacke. From game design elements to gamefulness: Defining gamification. In Proceedings of the 15th international academic MindTrek conference: Envisioning future media environments, volume 11, pages 9 – 15. ACM, 2011.
[13] B. J. DiSalvo and B. B. Morison. Khan Academy gamifies computer science. In Proc. 45th ACM Techn. Symp. Comput. Sci. Educ., pages 39 – 44, 2014.
[14] A. Dom´ınguez, J. Saenz-De-Navarrete, L. De-Marcos, L. Fern´andez-Sanz, C. Pag´es, and J.-J. Mart´ınez-Herr´aiz. Gamifying learning experiences: Practical implications and outcomes. Computers & Education, 63:380–392, 2013.
[15] D. Easley and A. Ghosh. Incentives, gamification, and game theory: An economic approach to badge design. ACM Trans. Econ. Comput., 4(3):16:1 – 16:26, 2016.
[16] L. Facey-Shaw, M. Specht, and J. Bartley-Bryan. Digital badges for motivating introductory programmers: Qualitative findings from focus groups. In 2018 IEEE Frontiers in Education Conference (FIE), pages 1–7, Oct 2018.
[17] L. Facey-Shaw, M. Specht, P. van Rosmalen, D. Brner, and J. Bartley-Bryan. Educational functions and design of badge systems: A conceptual literature review. IEEE Transactions on Learning Technologies, 11(4):536 – 544, 2018.
[18] L. Festinger. A theory of social comparison processes. Human relations, 7(2):117–140, 1954.
[19] Z. Fitz-Walter, D. Tjondronegoro, and P. Wyeth. Orientation passport: Using gamification to engage university students. In Proceedings of the 23rd Australian ComputerHuman Interaction Conference, OzCHI ’11, pages 122 – 125, New York, NY, USA, 2011. ACM.
[20] J. Fresno, H. Ortega-Arranz, A. Ortega-Arranz, A. Gonzalez-Escribano, and D. R. Llanos. Applying gamification in a parallel programming course. In Gamification in Education: Breakthroughs in Research and Practice, pages 278–302. IGI Global, 2018.
[21] J. Frith. Turning life into a game: Foursquare, gamification, and personal mobility. Mobile Media & Communication, 1(2):248–262, 2013.
[22] D. Gibson, N. Ostashewski, K. Flintoff, S. Grant, and E. Knight. Digital badges in education. Education and Information Technologies, 20(2):403 – 410, 2015.
[23] L. Haaranen, L. Hakulinen, P. Ihantola, and A. Korhonen. Software architectures for implementing achievement badges - practical experiences. In 2014 International Conference on Teaching and Learning in Computing and Engineering, pages 41–46, April 2014.
[24] L. Hakulinen, T. Auvinen, and A. Korhonen. Empirical study on the effect of achievement badges in TRAKLA2 online learning environment. In 2013 Learning and teaching in computing and engineering, pages 47–54. IEEE, 2013.
[25] A. M. Halavais. A genealogy of badges: Inherited meaning and monstrous moral hybrids. Information, Communication & Society, 15(3):354 – 373, 2012.
[26] J. Hamari. Transforming homo economicus into homo ludens: A field experiment on gamification in a utilitarian peer-to-peer trading service. Electronic Commerce Research and Applications, 12(4):236 – 245, 2013. Social Commerce- Part 2.
[27] J. Hamari. Do badges increase user activity? A field experiment on the effects of gamification. Computers in human behavior, 71:469 – 478, 2017.
[28] J. Hamari, J. Koivisto, H. Sarsa, et al. Does gamification work? - A literature review of empirical studies on gamification. In HICSS, volume 14, pages 3025–3034, 2014.
[29] K. Hilliard. Activision badges - the original gaming achievement (2013, october 23). Retrieved October, 2019, from Game Informer: https://www.gameinformer.com/b/features/archive/2013/10/26/activision-badgesthe-original-gaming-achievement.aspx.
[30] R. Hunicke, M. LeBlanc, and R. Zubek. Mda: A formal approach to game design and game research. In Proceedings of the AAAI Workshop on Challenges in Game AI, volume 4, page 1722, 2004.
[31] D. Huynh and H. Iida. An analysis of winning streak’s effects in language course of ”Duolingo”. Asia-Pacific Journal of Information Technology and Multimedia, 6(2), 2017.
[32] D. Huynh, L. Zuo, and H. Iida. Analyzing gamification of ”Duolingo” with focus on its course structure. In International Conference on Games and Learning Alliance, pages 268–277. Springer, 2016.
[33] D. Huynh, L. Zuo, and H. Iida. An Assessment of Game Elements in Language-Learning Platform Duolingo. In 2018 4th International Conference on Computer and Information Sciences (ICCOINS), pages 1–4. IEEE, 2018.
[34] M.-B. Ibanez, A. Di-Serio, and C. Delgado-Kloos. Gamification for engaging computer science students in learning activities: A case study. IEEE Transactions on learning technologies, 7(3):291–301, 2014.
[35] A. J¨arvinen. Games without frontiers: Theories and methods for game studies and design. Tampere University Press, 2008.
[36] D. Johnson, S. Deterding, K.-A. Kuhn, A. Staneva, S. Stoyanov, and L. Hides. Gamification for health and wellbeing: A systematic review of the literature. Internet interventions, 6:89–106, 2016.
[37] G. Kiryakova, N. Angelova, and L. Yordanova. Gamification in education. In Proceedings of 9th International Balkan Education and Science Conference, 2014.
[38] P. Kotler, H. Kartajaya, and I. Setiawan. Marketing 4.0: Moving from traditional to digital. John Wiley & Sons, 2016.
[39] K.-J. Majuri, Jenni and J. Hamari. Gamification of education and learning: A review of empirical literature. In Proceedings of the 2nd International GamiFIN Conference, GamiFIN 2018. CEUR-WS, 2018.
[40] R. McDaniel, R. Lindgren, and J. Friskics. Using badges for shaping interactions in online learning environments. In 2012 IEEE International Professional Communication Conference, pages 1–4, Oct 2012.
[41] J. A. Ruip´erez-Valiente, P. J. Mu˜noz-Merino, and C. Delgado Kloos. Detecting and clustering students by their gamification behavior with badges: A case study in engineering education. International Journal of Engineering Education, 33(2-B):816 – 830, 2017.
[42] L. Seixas, A. Gomes, and I. Melo Filho. Effectiveness of gamification in the engagement of students. Computers in Human Behavior, 58:48–63, 2016.
[43] M. Sicart. Defining game mechanics. Game Studies, 8(2):n, 2008.
[44] M. Sigala. The application and impact of gamification funware on trip planning and experiences: The case of tripadvisor´s funware. Electronic Markets, 25(3):189–209, 2015.
[45] R. St˚alnacke Larsson. Motivations in sports and fitness gamification: A study to understand what motivates the users of sports and fitness gamification services, 2013.
[46] A. Stott and C. Neustaedter. Analysis of gamification in education. Surrey, BC, Canada, 8:36, 2013.
[47] TechSci Research. Global gamification market by solution, by deployment, by organization size, by application, by end-user vertical, by region, competition, forecast & opportunities, 2024. TechSci Research, 2019.
[48] M. M. Ten´orio, F. A. F. Reinaldo, L. A. G´ois, R. P. Lopes, and G. dos Santos Junior. Elements of gamification in virtual learning environments. In International Conference on Interactive Collaborative Learning, pages 86–96. Springer, 2017.
[49] A. T´oth and E. L´og´o. The effect of gamification in sport applications. In ´ 2018 9th IEEE International Conference on Cognitive Infocommunications (CogInfoCom), pages 69–74. IEEE, 2018.
[50] R. van Roy, S. Deterding, and B. Zaman. Collecting Pok´emon or receiving rewards? How people functionalise badges in gamified online learning environments in the wild. International Journal of Human-Computer Studies, 127:62–80, 2019.
[2] S. S. Abrams and S. Walsh. Gamified vocabulary. Journal of Adolescent & Adult Literacy, 58(1):49–58, 2014.
[3] I. Ajzen. The theory of planned behavior. Organizational behavior and human decision processes, 50(2):179–211, 1991.
[4] D. Akbulut and G. Yıldırım. The usage of green gamification for public interest function of public relations. CTC 2019, 2019.
[5] R. S. Alsawaier. The effect of gamification on motivation and engagement. The International Journal of Information and Learning Technology, 35(1):56 – 79, 2018.
[6] A. Amriani, A. F. Aji, A. Y. Utomo, and K. M. Junus. An empirical study of gamification impact on e-learning environment. In Proceedings of 2013 3rd International Conference on Computer Science and Network Technology, pages 265 – 269. IEEE, 2013.
[7] A. Anderson, D. Huttenlocher, J. Kleinberg, and J. Leskovec. Steering user behavior with badges. In Proceedings of the 22nd International Conference on World Wide Web, pages 95–106. ACM, 2013.
[8] G. Barata, S. Gama, J. Jorge, and D. Gon¸calves. Engaging engineering students with gamification. In 2013 5th International Conference on Games and Virtual Worlds for Serious Applications (VS-GAMES), pages 1–8. IEEE, 2013.
[9] S. K. Bista, S. Nepal, N. Colineau, and C. Paris. Using gamification in an online community. In 8th International Conference on Collaborative Computing: Networking, Applications and Worksharing (CollaborateCom), pages 611–618. IEEE, 2012.
[10] H. Cavusoglu, Z. Li, and K.-W. Huang. Can gamification motivate volun-tary contributions?: the case of Stackoverflow Q&A community. In Proceedings of the 18th ACM Conference Companion on Computer Supported Cooperative Work & Social Computing, pages 171–174. ACM, 2015.
[11] L. De-Marcos, A. Dom´ınguez, J. Saenz-de Navarrete, and C. Pag´es. An empirical study comparing gamification and social networking on e-learning. Computers & Education, 75:82–91, 2014.
[12] S. Deterding, D. Dixon, R. Khaled, and L. Nacke. From game design elements to gamefulness: Defining gamification. In Proceedings of the 15th international academic MindTrek conference: Envisioning future media environments, volume 11, pages 9 – 15. ACM, 2011.
[13] B. J. DiSalvo and B. B. Morison. Khan Academy gamifies computer science. In Proc. 45th ACM Techn. Symp. Comput. Sci. Educ., pages 39 – 44, 2014.
[14] A. Dom´ınguez, J. Saenz-De-Navarrete, L. De-Marcos, L. Fern´andez-Sanz, C. Pag´es, and J.-J. Mart´ınez-Herr´aiz. Gamifying learning experiences: Practical implications and outcomes. Computers & Education, 63:380–392, 2013.
[15] D. Easley and A. Ghosh. Incentives, gamification, and game theory: An economic approach to badge design. ACM Trans. Econ. Comput., 4(3):16:1 – 16:26, 2016.
[16] L. Facey-Shaw, M. Specht, and J. Bartley-Bryan. Digital badges for motivating introductory programmers: Qualitative findings from focus groups. In 2018 IEEE Frontiers in Education Conference (FIE), pages 1–7, Oct 2018.
[17] L. Facey-Shaw, M. Specht, P. van Rosmalen, D. Brner, and J. Bartley-Bryan. Educational functions and design of badge systems: A conceptual literature review. IEEE Transactions on Learning Technologies, 11(4):536 – 544, 2018.
[18] L. Festinger. A theory of social comparison processes. Human relations, 7(2):117–140, 1954.
[19] Z. Fitz-Walter, D. Tjondronegoro, and P. Wyeth. Orientation passport: Using gamification to engage university students. In Proceedings of the 23rd Australian ComputerHuman Interaction Conference, OzCHI ’11, pages 122 – 125, New York, NY, USA, 2011. ACM.
[20] J. Fresno, H. Ortega-Arranz, A. Ortega-Arranz, A. Gonzalez-Escribano, and D. R. Llanos. Applying gamification in a parallel programming course. In Gamification in Education: Breakthroughs in Research and Practice, pages 278–302. IGI Global, 2018.
[21] J. Frith. Turning life into a game: Foursquare, gamification, and personal mobility. Mobile Media & Communication, 1(2):248–262, 2013.
[22] D. Gibson, N. Ostashewski, K. Flintoff, S. Grant, and E. Knight. Digital badges in education. Education and Information Technologies, 20(2):403 – 410, 2015.
[23] L. Haaranen, L. Hakulinen, P. Ihantola, and A. Korhonen. Software architectures for implementing achievement badges - practical experiences. In 2014 International Conference on Teaching and Learning in Computing and Engineering, pages 41–46, April 2014.
[24] L. Hakulinen, T. Auvinen, and A. Korhonen. Empirical study on the effect of achievement badges in TRAKLA2 online learning environment. In 2013 Learning and teaching in computing and engineering, pages 47–54. IEEE, 2013.
[25] A. M. Halavais. A genealogy of badges: Inherited meaning and monstrous moral hybrids. Information, Communication & Society, 15(3):354 – 373, 2012.
[26] J. Hamari. Transforming homo economicus into homo ludens: A field experiment on gamification in a utilitarian peer-to-peer trading service. Electronic Commerce Research and Applications, 12(4):236 – 245, 2013. Social Commerce- Part 2.
[27] J. Hamari. Do badges increase user activity? A field experiment on the effects of gamification. Computers in human behavior, 71:469 – 478, 2017.
[28] J. Hamari, J. Koivisto, H. Sarsa, et al. Does gamification work? - A literature review of empirical studies on gamification. In HICSS, volume 14, pages 3025–3034, 2014.
[29] K. Hilliard. Activision badges - the original gaming achievement (2013, october 23). Retrieved October, 2019, from Game Informer: https://www.gameinformer.com/b/features/archive/2013/10/26/activision-badgesthe-original-gaming-achievement.aspx.
[30] R. Hunicke, M. LeBlanc, and R. Zubek. Mda: A formal approach to game design and game research. In Proceedings of the AAAI Workshop on Challenges in Game AI, volume 4, page 1722, 2004.
[31] D. Huynh and H. Iida. An analysis of winning streak’s effects in language course of ”Duolingo”. Asia-Pacific Journal of Information Technology and Multimedia, 6(2), 2017.
[32] D. Huynh, L. Zuo, and H. Iida. Analyzing gamification of ”Duolingo” with focus on its course structure. In International Conference on Games and Learning Alliance, pages 268–277. Springer, 2016.
[33] D. Huynh, L. Zuo, and H. Iida. An Assessment of Game Elements in Language-Learning Platform Duolingo. In 2018 4th International Conference on Computer and Information Sciences (ICCOINS), pages 1–4. IEEE, 2018.
[34] M.-B. Ibanez, A. Di-Serio, and C. Delgado-Kloos. Gamification for engaging computer science students in learning activities: A case study. IEEE Transactions on learning technologies, 7(3):291–301, 2014.
[35] A. J¨arvinen. Games without frontiers: Theories and methods for game studies and design. Tampere University Press, 2008.
[36] D. Johnson, S. Deterding, K.-A. Kuhn, A. Staneva, S. Stoyanov, and L. Hides. Gamification for health and wellbeing: A systematic review of the literature. Internet interventions, 6:89–106, 2016.
[37] G. Kiryakova, N. Angelova, and L. Yordanova. Gamification in education. In Proceedings of 9th International Balkan Education and Science Conference, 2014.
[38] P. Kotler, H. Kartajaya, and I. Setiawan. Marketing 4.0: Moving from traditional to digital. John Wiley & Sons, 2016.
[39] K.-J. Majuri, Jenni and J. Hamari. Gamification of education and learning: A review of empirical literature. In Proceedings of the 2nd International GamiFIN Conference, GamiFIN 2018. CEUR-WS, 2018.
[40] R. McDaniel, R. Lindgren, and J. Friskics. Using badges for shaping interactions in online learning environments. In 2012 IEEE International Professional Communication Conference, pages 1–4, Oct 2012.
[41] J. A. Ruip´erez-Valiente, P. J. Mu˜noz-Merino, and C. Delgado Kloos. Detecting and clustering students by their gamification behavior with badges: A case study in engineering education. International Journal of Engineering Education, 33(2-B):816 – 830, 2017.
[42] L. Seixas, A. Gomes, and I. Melo Filho. Effectiveness of gamification in the engagement of students. Computers in Human Behavior, 58:48–63, 2016.
[43] M. Sicart. Defining game mechanics. Game Studies, 8(2):n, 2008.
[44] M. Sigala. The application and impact of gamification funware on trip planning and experiences: The case of tripadvisor´s funware. Electronic Markets, 25(3):189–209, 2015.
[45] R. St˚alnacke Larsson. Motivations in sports and fitness gamification: A study to understand what motivates the users of sports and fitness gamification services, 2013.
[46] A. Stott and C. Neustaedter. Analysis of gamification in education. Surrey, BC, Canada, 8:36, 2013.
[47] TechSci Research. Global gamification market by solution, by deployment, by organization size, by application, by end-user vertical, by region, competition, forecast & opportunities, 2024. TechSci Research, 2019.
[48] M. M. Ten´orio, F. A. F. Reinaldo, L. A. G´ois, R. P. Lopes, and G. dos Santos Junior. Elements of gamification in virtual learning environments. In International Conference on Interactive Collaborative Learning, pages 86–96. Springer, 2017.
[49] A. T´oth and E. L´og´o. The effect of gamification in sport applications. In ´ 2018 9th IEEE International Conference on Cognitive Infocommunications (CogInfoCom), pages 69–74. IEEE, 2018.
[50] R. van Roy, S. Deterding, and B. Zaman. Collecting Pok´emon or receiving rewards? How people functionalise badges in gamified online learning environments in the wild. International Journal of Human-Computer Studies, 127:62–80, 2019.
Published
2020-04-30
How to Cite
ZSIGMOND, I.; LORINCZ, B.; MOLNAR, A.E..
Review of Digital Badges in Computer Science Literature and Applications.
Studia Universitatis Babeș-Bolyai Informatica, [S.l.], v. 65, n. 1, p. 17-32, apr. 2020.
ISSN 2065-9601.
Available at: <https://www.cs.ubbcluj.ro/~studia-i/journal/journal/article/view/48>. Date accessed: 25 apr. 2024.
doi: https://doi.org/10.24193/subbi.2020.1.02.
Issue
Section
Articles
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
When the article is accepted for publication, I, as the author and representative of the coauthors, hereby agree to transfer to Studia Universitatis Babes-Bolyai, Series Informatica, all rights, including those pertaining to electronic forms and transmissions, under existing copyright laws, except for the following, which the author specifically retain: the right to make further copies of all or part of the published article for my use in classroom teaching; the right to reuse all or part of this material in a review or in a textbook of which I am the author; the right to make copies of the published work for internal distribution within the institution that employs me.
