Proactive AI Achieves 52% Developer Acceptance in Five-Day IDE Field Study

The challenge of seamlessly integrating artificial intelligence into everyday coding practices remains a significant hurdle for software development. Nadine Kuo of JetBrains Amsterdam, Agnia Sergeyuk of JetBrains Research Belgrade, Valerie Chen of Carnegie Mellon University, and Maliheh Izadi of Delft University of Technology investigated developer responses to proactive assistance within a professional Integrated Development Environment. Their five-day field study, observing 15 developers, examined 229 AI interventions and over 5,700 interaction points to determine how and when developers best receive automated suggestions. The research demonstrates that carefully timed proactive suggestions , particularly at natural workflow boundaries , significantly reduce cognitive load and increase engagement compared to reactive prompts. These findings offer crucial insights for designing future coding assistants that enhance, rather than disrupt, developer workflows and balance automation with user agency.

Researchers engineered ProAIDE to deliver code quality suggestions, moving beyond reactive tools to anticipate developer needs and address issues before they escalate. The system was developed through an iterative, human-centered design process encompassing four phases, beginning with a prototype and culminating in a deployed, fully functional feature within a widely used IDE. This approach enabled evaluation of proactive support within authentic, real-world development workflows spanning 12 programming languages. Scientists harnessed telemetry logs capturing 5,732 interaction points and 229 interventions to meticulously track developer responses to ProAIDE’s suggestions.

The experimental setup examined how proactive suggestions were received across different workflow stages, specifically contrasting interventions occurring at workflow boundaries, such as post-commit actions, with those delivered mid-task, like during declined edits. Data collection extended beyond simple acceptance or dismissal of suggestions, incorporating structured daily surveys and a comprehensive post-study questionnaire to gauge developer experience and perceived impact. This mixed-methods approach allowed for a nuanced understanding of both behavioral interactions and subjective perceptions. The research team quantified the impact of well-timed suggestions, revealing that interventions at workflow boundaries achieved a 52% engagement rate, significantly higher than the 62% dismissal rate observed for mid-task interventions.

Furthermore, analysis demonstrated that accepting well-timed proactive suggestions required significantly less interpretation time, an average of 45.4 seconds, compared to responding to reactive suggestions, which demanded 101.4 seconds (W = 109.00, r = 0.533, p = 0.0016). This finding highlights the enhanced cognitive efficiency achieved through proactive assistance, demonstrating how carefully timed interventions can streamline developer workflows and reduce mental fatigue. This methodological innovation, integrating proactive AI directly into a production IDE and employing a mixed-methods evaluation, enabled the identification of systematic patterns in human receptivity to assistance. The study’s findings provide actionable insights for designing future proactive coding assistants, informing decisions about intervention timing, contextual alignment, and the crucial balance between AI agency and user control within professional development environments. The detailed analysis of developer interactions and experiences directly supports the development of more effective and less disruptive AI-powered tools.

Workflow Boundaries Yield High Engagement Rates

Scientists achieved a 52% engagement rate for proactive assistance interventions timed at workflow boundaries, specifically after code commits. The five-day field study involved 15 developers interacting with a proactive feature integrated into a production-grade IDE, designed to offer code quality suggestions based on in-IDE activity. Researchers examined 229 interventions across a total of 5,732 interaction points to understand developer receptivity to these suggestions throughout their workflows and assess perceived impact. Data shows that interventions occurring mid-task, such as those following a declined edit, were dismissed 62% of the time, highlighting the importance of timing.

Experiments revealed a significant reduction in interpretation time for well-timed proactive suggestions compared to reactive suggestions; developers required an average of 45.4 seconds to process proactive suggestions, versus 101.4 seconds for reactive ones (W = 109.00, r = 0.533, p = 0.0016). This difference indicates enhanced cognitive efficiency when assistance is proactively offered at appropriate moments. The study utilized a mixed-methods approach, combining telemetry logs with daily and post-study surveys, allowing for analysis of both interaction patterns and developer experiences. Developers rated the ProAIDE system with a score of 72.8 out of 100 on the System Usability Scale (SUS), with a 95% confidence interval of [64.1, 81.5], demonstrating ease of use and appreciation for IDE integration.

While developers valued alignment between suggestions and task intent, the utility of the AI diminished when it lacked understanding of lower-level contextual details within the source code, including technical design choices and domain-specific patterns. Across the 5,732 interaction points, the research team quantitatively demonstrated that suggestions triggered at natural workflow boundaries, particularly after commits, were significantly more likely to be accepted by developers. This work provides empirically grounded implications for designing proactive coding assistants, focusing on intervention timing, contextual relevance, and balancing agency with user control within production IDEs. The study material, including recruitment texts and survey instruments, is available as a replication package.

Timing and Workflow Drive AI Acceptance

This research investigated interactions between professional developers and proactive artificial intelligence assistance within a production-grade integrated development environment. Through a five-day field study involving fifteen developers, the work demonstrated a clear link between the timing of suggestions and their acceptance, with interventions presented at natural workflow boundaries, such as immediately after code submission, achieving significantly higher engagement than those delivered mid-task. The study also revealed that well-timed proactive suggestions required less cognitive effort for developers, being interpreted and applied much faster than reactive suggestions. These findings contribute to a growing understanding of how to effectively integrate AI into everyday software engineering practices.

The research highlights the importance of designing proactive coding assistants that are sensitive to developer context and workflow stages, balancing automated support with user agency. While acknowledging limitations related to sample size and study duration, the authors suggest that longer-term investigations are necessary to fully assess the sustained impact of proactive assistance on developer productivity and satisfaction. The work offers empirical evidence and design insights to bridge the gap between conceptual AI prototypes and practical IDE integration, paving the way for adaptive and user-configurable AI support in software development.