Developers Build Mental Maps to Systematically Track Down Software Errors

Debugging represents a fundamental challenge within software engineering. Haolin Li and Michael Coblenz, both from the University of California San Diego, alongside their colleagues, present new insights into this crucial process. Their research addresses a significant gap in understanding how experienced developers approach bug fixing in substantial, practical codebases. Through qualitative observation of seven professional developers and five professional live-streamers tackling seventeen debugging tasks within their own working environments, the team theorises debugging as a systematic, iterative diagnostic process. They demonstrate that programmers refine their understanding of a system to direct information collection, alternating between code navigation and execution, and adapting their reasoning based on context, complexity, and familiarity. This grounded theory highlights the human-centred aspects of professional debugging, offering valuable implications for the design of improved tools and more effective software engineering education.

Scientists are beginning to unravel the complex cognitive processes underpinning debugging, a fundamental activity in software engineering. While prior research has catalogued debugging tools and strategies, a comprehensive theory explaining how experienced developers tackle bugs in large, real-world codebases has remained elusive. This work details a grounded theory of professional debugging, emerging from detailed observation of developers confronting genuine software issues in their own projects. The study focused on seven professional developers and five live-coding streamers, observing them across seventeen distinct debugging tasks. Researchers discovered that debugging isn’t simply about applying pre-defined techniques, but rather a continuous refinement of a programmer’s mental model of the system. This mental model, a working representation of how the code functions, is iteratively updated as developers gather information, guiding their subsequent actions. Crucially, developers don’t necessarily strive for complete system understanding; instead, they extend their existing knowledge just enough to resolve the immediate issue, a strategy termed ‘knowledge avoidance’. Observations revealed a dynamic interplay between navigation and execution strategies, with developers seamlessly switching between exploring the codebase and running the program to observe its behaviour. They employ both forward and backward tracing, reasoning about cause and effect in either direction, adapting their approach based on the complexity and familiarity of the code. Furthermore, the research highlights the significant role of external resources, such as colleagues and version control history, complementing code-based evidence. This establishes a human-centred perspective on debugging, offering valuable insights for the design of more effective debugging tools and improved software engineering education. The analysis of recordings from five additional live-streamed debugging sessions further validated these findings, confirming that mental model development constitutes the bulk of the debugging effort. Developers were observed formulating vague explanations and iteratively refining them based on gathered information, rather than strictly adhering to a scientific hypothesis-testing method. Across the 17 debugging tasks observed, developers dedicated an average of 12% of their time to initial bug reproduction. A substantial 57% of effort was then directed towards mental model development, representing the most time-consuming stage of the process. Conversely, constructing a fix consumed 13% of the total time, while validating that fix accounted for the remaining 18%. This distribution highlights the cognitive emphasis of professional debugging, with significantly more time invested in understanding the problem than in implementing or confirming a solution. The research revealed an average of 1.65 fix-validate-fix cycles per task, indicating iterative refinement is common. In nine of the seventeen tasks, developers returned to the mental model update stage after an initial fix failed verification, demonstrating the importance of continuous refinement of understanding. Furthermore, the four incomplete debugging sessions all concluded with developers still actively updating their mental models, suggesting this stage often represents the limiting factor in task completion. Combined data from seven professional developers and five live-streamers totalled 11 hours of debugging activity. Participants tackled 11 debugging tasks for an average duration of 7.8 hours, while streamers completed six tasks over 3.1 hours. Task times varied considerably, ranging from 10 minutes for a simple syntax error (S1TA) to 1 hour and 54 minutes for a complex icon triggering issue (S5TA). Detailed breakdowns for each task, including resolution status, are provided in Table 4, illustrating the diversity of challenges encountered. A constructivist grounded theory methodology underpinned this work, enabling a detailed analysis of professional debugging practices. Initially, researchers conducted direct observations of seven professional developers (P1, P7) as they tackled 17 debugging tasks within their own established codebases. This approach prioritised understanding how developers actually reason about bugs in realistic, complex software projects, rather than relying on self-reported strategies. Data collection involved meticulous observation of participants’ work, capturing the diverse contexts surrounding their debugging efforts. Following the initial observational phase, the research team analysed recordings from five professional live-streamers (S1, S5) engaged in six additional debugging sessions. This secondary analysis served to validate and refine the theoretical categories emerging from the first stage, providing a crucial check for consistency and broadening the scope of the findings. The selection of live-streamed sessions allowed access to debugging work occurring in a more natural, unprompted setting, complementing the controlled observations. Participants were recruited based on stringent criteria, including a minimum of three years of full-time software development experience. The study deliberately avoided constraints on programming languages, task types, or difficulty levels to maximise the external validity of the findings. Participants were sourced through social media, developer events, and established industry partnerships, resulting in a cohort of seven developers, five men and two women, with an average of 6.5 years of professional experience, ranging from 3 to 15 years. This dual-phase approach, combining direct observation and analysis of live-streamed work, facilitated theoretical saturation and ensured the robustness of the grounded theory. For years, the subtle art of debugging has remained surprisingly opaque to systematic study. The researchers demonstrate that experienced developers don’t just react to bugs; they actively construct and refine a mental model of the system, guiding their investigation with a blend of forward and backward reasoning. It’s a crucial step because simply providing better tools won’t solve the problem if we don’t first understand the cognitive processes they’re meant to augment. This isn’t merely about tracing code, but about building a coherent picture of how things should work, and then systematically comparing that expectation to reality. The emphasis on integrating external resources, documentation, online forums, even tacit knowledge, highlights the inherently social nature of modern software development. However, the study’s reliance on observation presents limitations. Capturing the full richness of a developer’s thought process remains a challenge, and the relatively small sample size necessitates caution when generalising findings. Future work could benefit from combining observational studies with techniques like concurrent think-aloud protocols, or even neuroimaging, to gain deeper insights into the cognitive mechanisms at play. Ultimately, this research isn’t just about debugging; it’s about understanding how humans reason about complex systems, a skill increasingly vital in a world saturated with technology.