Your AI Adoption Is High. Is Your Engineering Delivery Better?

Gaurav Singal is Chief Technology Officer at ConstructConnect, a Roper Technologies company.

Most technology leaders have oversight into how many engineers are using AI, but the ones I talk to still can’t answer what that AI usage has changed. Having faced the same challenge as we rolled out agentic development at ConstructConnect, I understand why this question is so difficult to answer.

After we built an AI adoption dashboard, our production feature count rose 5x to 10x in the equivalent quarter a year later. But features vary in size, and an estimate is an imperfect baseline. While the figures showed progress, they couldn’t establish how much came from AI or what customers gained.

It’s easy to leave this question out of an engineering review, but it’s also essential for shaping the investment. That’s why I want to share some strategies that have helped our team understand AI’s impact.

Follow the work beyond the release.

In our case, we wanted to know whether we’re building sound software, delivering it reliably and making something better for the customer.

Each question requires different evidence.

Test coverage, defects found after release and unresolved security findings can help assess software quality. Technical debt matters, too, because today’s shortcuts can slow the next release. Finally, high coverage is only reassuring when the tests catch the failures that matter.

For delivery, a team shipping more often while customers experience more disruption has work to do. To get a picture of this issue, track deployment frequency and lead time from commit to production. These metrics should be supplemented by change failure rate, the share of deployments requiring a fix or rollback and the time needed to restore service after a failed deployment. Availability completes the picture.

Faster work needs clearer decisions.

That makes the original request more important. An agent can implement a vague ticket quickly, but it cannot resolve a product decision you haven’t made.