Field Experiment of the Memory Retention of Programmers Regarding Source Code

A. Fekete; Z. Porkoláb

doi:10.24193/subbi.2023.1.05

A. Fekete Eötvös Loránd University, Faculty of Informatics, Egyetem tér 1-3., 1053 Budapest, Hungary
Z. Porkoláb Eötvös Loránd University, Faculty of Informatics, Egyetem tér 1-3., 1053 Budapest, Hungary

DOI: https://doi.org/10.24193/subbi.2023.1.05

Abstract

Program comprehension is a continuously important topic in computer science since the spread of personal computers, and several program comprehension models have been identified as possible directions of active code comprehension. There has been little research on how much programmers remember the code they have once written. We conducted two experiments with a group of Computer Science MSc students. In the first experiment, we examined the code comprehension strategies of the participants. The students were given a task to implement a minor feature in a relatively small C++ project. In the second experiment, we asked the students 2 months later to complete the same task again. Before starting the clock, we asked the students to fill a questionnaire which aimed to measure program code-related memory retention: we inquired about how much the students remembered the code, down to the smallest relevant details, e.g. the name of functions and variables they had to find to complete the task.

After the second experiment, we could compare the solution times of those students who participated in both parts. As one result, we could see that these students could solve the task in shorter time than they did in the first experiment. We also looked at the results of the questionnaire: the vast majority of students could not precisely remember more than two or three identifiers from the original code. In this paper, we will show how this result compares to the forgetting curve.

References

1. Magdalena Andrzejewska and Agnieszka Skawinska, Examining students’ intrinsic cognitive load during program comprehension–an eye tracking approach, International Conference on Artificial Intelligence in Education, Springer, 2020, pp. 25–30.
2. Lee Averell and Andrew Heathcote, The form of the forgetting curve and the fate of memories, Journal of mathematical psychology 55 (2011), no. 1, 25–35.
3. Bas Cornelissen, Andy Zaidman, Arie Van Deursen, and Bart Van Rompaey, Trace visualization for program comprehension: A controlled experiment, 2009 IEEE 17th International Conference on Program Comprehension, IEEE, 2009, pp. 100–109. 4. Hermann Ebbinghaus, Uber das gedachtnis, 1885.
5. Anett Fekete and Zoltan Porkolab, Report on a Field Experiment of the Comprehension Strategies of Computer Science MSc Students, 2022 IEEE 16th International Scientific Conference on Informatics - Proceedings, IEEE, 2022, pp. 73–81.
6. Anett Fekete and Zoltan Porkolab, A comprehensive review on software comprehension models, Annales Mathematicae et Informaticae, vol. 51, Lıceum University Press, 2020, pp. 103–111.
7. Keumseok Kang and Jungpil Hahn, Learning and forgetting curves in software development: Does type of knowledge matter?, ICIS 2009 Proceedings (2009), 194.
8. Philipp Kather and Jan Vahrenhold, Is algorithm comprehension different from program comprehension?, 2021 IEEE/ACM 29th International Conference on Program Comprehension (ICPC), IEEE, 2021, pp. 455–466.
9. Jurgen Koenemann and Scott P Robertson, Expert problem solving strategies for program comprehension, Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 1991, pp. 125–130.
10. Jacob Kr¨uger, G¨ul C¸alıklı, Thorsten Berger, Thomas Leich, and Gunter Saake, Effects of explicit feature traceability on program comprehension, Proceedings of the 2019 27th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, 2019, pp. 338–349.
11. Jacob Kr¨uger and Regina Hebig, What developers (care to) recall: An interview survey on smaller systems, 2020 IEEE International Conference on Software Maintenance and Evolution (ICSME), IEEE, 2020, pp. 46–57.
12. Jacob Kruger, Jens Wiemann, Wolfram Fenske, Gunter Saake, and Thomas Leich, Do you remember this source code?, Proceedings of the 40th International Conference on Software Engineering, 2018, pp. 764–775.
13. Jaap MJ Murre and Joeri Dros, Replication and analysis of ebbinghaus’ forgetting curve, PloS one 10 (2015), no. 7.
14. Takao Nakagawa, Yasutaka Kamei, Hidetake Uwano, Akito Monden, Kenichi Matsumoto, and Daniel M German, Quantifying programmers’ mental workload during program comprehension based on cerebral blood flow measurement: a controlled experiment, Companion proceedings of the 36th international conference on software engineering, 2014, pp. 448–451.
15. Zoltan Porkolab, Tibor Brunner, Daniel Krupp, and Marton Csordas, Codecompass: an open software comprehension framework for industrial usage, Proceedings of the 26th Conference on Program Comprehension, 2018, pp. 361–369.
16. Dong Gue Roe, Seongchan Kim, Yoon Young Choi, Hwije Woo, Moon Sung Kang, Young Jae Song, Jong-Hyun Ahn, Yoonmyung Lee, and Jeong Ho Cho, Biologically plausible artificial synaptic array: Replicating ebbinghaus’ memory curve with selective attention, Advanced Materials 33 (2021), no. 14, 2007782.
17. Utku Unal, Eray Tuzun, Tamer Gezici, and Ausaf Ahmed Farooqui, Investigating the impact of forgetting in software development, arXiv preprint arXiv:2204.07669 (2022).
18. Weifeng Xu, Dianxiang Xu, and Lin Deng, Measurement of source code readability using word concreteness and memory retention of variable names, 2017 IEEE 41st Annual Computer Software and Applications Conference (COMPSAC), vol. 1, IEEE, 2017, pp. 33–38.