RDEL #27: The frequency and impact of destructive code reviews

This week we look at how destructive code reviews can impact the engineering team, how often it happens, and whether gender plays a role in the code review comment experience.

Welcome back to Research-Driven Engineering Leadership. Each week, we pose an interesting topic in engineering leadership, and apply the latest research in the field to drive to an answer.

This week, we look at the cornerstone of software development - the code review. This process of giving feedback on a teammate’s code can impact the productivity, trust, and psychological safety on an engineering team. Criticism is common, and can either be constructive or destructive in nature. So this week we ask: what is the frequency and impact of destructive code reviews?

The context

The code review process is an important tool on software teams to improve code quality, share knowledge, and provide learning opportunities. Prior research (Bosu et. al) shows that code reviews also impact impression formation between developers, including how the team trusts and relies on one another.

Code review comments are often aimed at improving code quality, but their delivery can come forward as either constructive or destructive. Destructive feedback can be defined as negative feedback that is both inconsiderate and nonspecific. Previous research has found that destructive criticism in broader contexts can provoke negative reactions in the recipient, such as reduced productivity and motivation. In this study, we’ll look at the specific frequency and impact in software engineering teams.

It’s important to note that this research is motivated by the impact of the software industry’s lack of diversity. In FAANG companies like Google and Facebook, women represent less than 25% of the tech workforce. Developers report that code review can also create toxic environments and contribute to diversity challenges, but this is the first study to examine those challenges in such detail.

The research

A team of researchers at Google and University of Auckland teamed up to explore the frequency, impact, and perceptions of destructive code reviews. Using data collected from series of surveys on hypothetical scenarios and past experiences, the team built a set of regression models to determine correlations and trends in the data.

The results found that:

• 22% of respondents who receive code review comments in their work reported receiving inconsiderate feedback at least once a year. 55% of respondents reported receiving nonspecific negative feedback at least once a year.

  • Interestingly, only 1 respondent believed they gave inconsiderate feedback, and 27% of respondents believe they gave nonspecific feedback in the last year.

• Destructive feedback was perceived to be less appropriate, valid, and likely to improve code quality compared to constructive criticism. Compared to men, women were more likely to perceive destructive criticism as appropriate.

  • Note: in open-ended questions, opinions were mixed on whether destructive criticism would be acceptable if the outcome was code quality. Some participants had no tolerance, while others found it to be acceptable if it was more “direct”.

• Compared to constructive criticism, destructive criticism negatively impacted motivation to continue working and the mood of participants. Women were less motivated to continue working with the developer who gave the feedback.

  

The application

This paper demonstrates how the nature of code review comments can have negative consequences on the teams cohesion, and the mechanism for doing so. If gone unnoticed, it can reduce trust and psychological safety on the engineering team, which can impact the productivity of the team and introduce turnover risk.

Some ways that engineering managers can mitigate against destructive code review comments are:

• Clarify the intention of the code review process as a team. While code reviews are valuable for improving code quality, they have equal or higher value as a tool for knowledge sharing and learning. Encouraging this culture will impact the tone and intention of engineers when they review a teammates code.

• Acknowledge unconscious bias. While a significant percentage of respondents received destructive criticism, a much smaller percentage believed that they delivered destructive criticism. It likely means that destructive feedback is often unintentional. This can be improved through transparent conversations in high-trust environments with teammates.

• Recognize if there are pressures on the code review process. Some teams that focus on frequent or continuous delivery might place undue pressure on the code review process. This can be especially true if teams are reporting productivity solely based on velocity (which the research recommends against!). When this happens, teammates become stressed and are prone to giving more negative feedback.

—

We hope this paper helps your team strengthen the collaboration process in your code reviews. Have a wonderful rest of the week, and happy Research Monday!

Lizzie

From the Quotient team