RDEL #120: What skills become most valuable when developers work with AI agents?
Developers who plan before acting accept agent code more often, revealing that abstraction and specification skills matter more than typing speed.
Welcome back to Research-Driven Engineering Leadership. Each week, we pose an interesting topic in engineering leadership and apply the latest research in the field to drive to an answer.
AI coding agents represent a fundamentally different way of building software—instead of typing code line by line, developers now describe what they want in natural language and evaluate AI-generated solutions. As these tools rapidly spread across engineering organizations, leaders face critical decisions about adoption, training, and team structure. This week we ask: what skills become most valuable when developers work with AI agents?
The context
For decades, software development has followed a consistent pattern: developers translate high-level requirements into concrete code by manually typing sequences of instructions. This process requires deep technical knowledge—understanding syntax, libraries, design patterns, and the specific architecture of each codebase. Junior developers typically need years to build this expertise and understand how to turn abstract ideas into working software.
AI coding agents fundamentally alter this production model. Rather than typing code directly, developers now instruct agents in natural language about what they want to accomplish. The agent then generates code by combining language models with external tools that can search codebases, browse documentation, and execute commands. This abstraction raises important questions: If agents handle the syntactic work of writing code, what cognitive skills become most important? Do experienced developers benefit more than juniors, or does AI level the playing field? And critically for engineering leaders—does this actually improve productivity, or just change how developers spend their time?
The research
Researchers analyzed adoption and productivity data from 1,000 organizations using Cursor, an AI-assisted programming platform that released a coding agent feature in November 2024. The study combined usage patterns from 86,665 developers, organizational-level code merge data using a natural experiment around the agent’s release, detailed metrics from one company, and analysis of 77,024 developer messages to understand how people interact with agents.
Key findings include:
Organizational output increased substantially after agent release: Using a difference-in-differences analysis comparing 24 organizations that had platform access versus 8 that didn’t, weekly code merges increased 39% after the agent became the default generation mode, with no significant changes in code revert rates in the following six months.
Experienced developers accepted agent code significantly more often: One standard deviation higher work experience (6.6 years) corresponded to 6% higher accept rates. This positive experience gradient persisted even after controlling for role, sector, and seniority, suggesting the advantage comes from skills related to experience itself, not just job title or industry.
The experience advantage with agents reverses the pattern from older AI tools: Less experienced workers are more likely to accept suggestions from AI autocompletion features, which assist during task execution. In contrast, experienced developers excel with agents that require task specification—suggesting that organizational context and the ability to clearly define delegated work are the differentiating skills.
Experienced developers approached agents more strategically: For every standard deviation increase in work experience, developers were 1.6 percentage points more likely to send planning messages before implementation and 0.8 percentage points less likely to ask agents to explain code. Plan-first users also had higher accept rates overall, suggesting experienced developers’ advantage comes from better alignment between their intent and agent output.
Distribution of message intent Agent adoption enabled non-engineers to code: While 76% of agent users were software engineers, designers submitted 74.7 agent messages per week and accepted generated code 14% more often than engineers—suggesting agents allow people in traditionally low-code roles to perform programming tasks.
The application
This research reveals that AI agents create a new production model, but the skills that become most valuable reveal why experienced engineers gain more from these tools.. The skills that become most valuable are abstraction (understanding what agents can do and how to decompose tasks), clarity (specifying intent precisely in natural language), and evaluation (critically assessing generated code before accepting it). Experienced developers excel because they have more organizational context for task specification and more judgment for quality evaluation, not because they type faster or know more syntax.
To put this into practice, consider the following:
Invest in context-building for your team: The experience advantage stems partly from codebase knowledge that improves task specification. Create documentation, maintain architectural decision records, and ensure regular knowledge-sharing sessions so developers have the context needed to give agents effective instructions. Consider pairing junior developers with seniors specifically around agent usage patterns.
Develop new competencies around agent interaction: Create training specifically for how to work with agents—teach developers to plan before implementing (which correlates with higher accept rates), write clear natural language specifications, and critically evaluate generated code. Consider having experienced developers share their agent usage patterns, especially their planning and evaluation processes, with the rest of the team.
Measure scope and quality, not just velocity: Short-term quality metrics won’t capture the impact of long term decisions. Track not just how much code ships, but its maintainability, duplication patterns, and architectural coherence.
—
Happy Research Tuesday, and have a wonderful thanksgiving week.
Lizzie
PS - thanks to one of our readers, Matt, for recommending this study to us!