AI is not a panacea for software development

How much does developer productivity increase when using AI-enabled coding tools? Recently, there has been a lot of speculation that AI makes developers 2, 3, and even 5 times more productive. One report predicts a tenfold increase in developer productivity by 2030.

The irony, however, is that the engineering community has largely failed to agree on a universal way to measure labor productivity. Some have even abandoned the idea, arguing that most metrics are flawed or incomplete. Most claims that AI improves labor productivity today are qualitative in nature - based on surveys and anecdotes rather than quantitative data.

How can we judge artificial intelligence without first agreeing on how to measure performance? If we've learned anything from the remote work experiment, it's that we flounder without data to inform our decisions and move from one office, remote, and hybrid strategy to another based on dogma and ideology rather than data and measurement.

We are on a journey to replicate the experience of using artificial intelligence. To move forward, we must first understand and quantify its impact.

Backlog risk

The current hype around artificial intelligence may make some of us think twice - because of the unknown impact on quality, the potential risk of plagiarism and other factors. The most cautious companies have taken a wait-and-see attitude, waiting to see how it all turns out.

However, for technology-enabled businesses, the risk of falling behind is existential. AI is a dual gas pedal, influencing what and how companies create. Companies that invest in AI today can reap the double benefit of not only bringing new AI-powered products to market, but also bringing them to market faster and cheaper.

Most companies are focused on what to do, but AI can be the driving force behind how to do it, creating a 10x or even 100x engineering team. Companies that understand how to quickly bridge the chasm - by optimizing AI tools in the most efficient and effective way - and reach a performance plateau faster will have an advantage for years to come. The risk of doing nothing is too high.

Understanding trade-offs

To someone with a hammer, everything looks like a nail. It's the same with artificial intelligence.

According to a recent GitHub report, the main benefit of AI-enabled coding tools was cited by developers as an increase in coding language proficiency. Another key benefit is the automation of repetitive tasks such as writing boilerplate code. A recent experiment conducted by Codecov showed that ChatGPT is great at writing simple tests for trivial functions and relatively simple code paths.