Back to Devoured - June 26, 2026
DEVOURED
AI Has Outpaced How Engineering Organizations Measure Developer Productivity

AI Has Outpaced How Engineering Organizations Measure Developer Productivity

DevOps DevOps Digest
Engineering organizations are reporting AI-driven productivity gains while simultaneously losing visibility into the real costs of code validation and quality maintenance.
What: The 'State of Engineering Excellence 2026' report from Harness found that while 89% of leaders report gains, 31% of developer time is now spent on 'invisible' work like reviewing AI-generated code, with 54% of developers fearing performance surveillance.
Why it matters: Traditional metrics like velocity are failing because they don't capture the increased cognitive load and validation overhead introduced by AI-generated output, creating a disconnect between perceived and actual developer performance.
Takeaway: Update your metrics to include validation time, code quality, and burnout indicators rather than relying solely on legacy DORA or cycle-time frameworks.
Deep dive
  • AI usage is now ubiquitous, with 88% of leaders noting improved satisfaction.
  • 81% of developers report spending more time on code reviews.
  • 31% of developer time is estimated as 'invisible' work (context switching, bug fixing).
  • 94% of leaders admit existing metrics miss key factors like tech debt and validation time.
  • Developers are resisting top-down measurement due to privacy concerns.
  • Recommended: Separate 'improvement' data from 'performance evaluation' data.
Original article

AI coding tools have transformed the day-to-day work of software developers faster than the industry's measurement frameworks can keep up, according to The State of Engineering Excellence 2026, a new report from Harness.

The result is a growing visibility gap: engineering organizations are reporting record productivity gains while simultaneously acknowledging they no longer have the right instruments to tell whether those gains are real — or what they're costing.

The AI Productivity Paradox

The report tells a complicated story. AI adoption is now the default in engineering organizations, and self-reported impact is overwhelmingly positive — but the cost is accumulating in places organizations aren't watching.

  • Leaders are reporting big gains from AI. 89% of engineering leaders say developer productivity has improved since adopting AI coding tools, and 88% say developer satisfaction has improved.
  • Yet developers are spending more of their day on manual work. 81% say developers spend more time in code review since adopting AI coding tools, with 28% reporting a significant increase of more than 30%.
  • And nearly a third of that work isn't tracked anywhere. Organizations estimate approximately 31% of developer time is now consumed by invisible work like reviewing AI-generated code, fixing bugs, and context switching between tools.

Metrics That Don't Match the Work

The clearest sign that legacy frameworks aren't keeping pace is the contradiction in the data:

  • Leaders trust metrics that miss the basics. 89% say their current metrics accurately reflect AI's impact, yet 94% say key factors, including tech debt, validation time, and developer burnout, are missing from those same metrics. And only 6% believe the frameworks they have today can fix it.
  • The biggest AI challenge is measurement itself. When asked to name the single biggest challenge, the top answers are all visibility problems: measuring true productivity impact (26%), maintaining code quality with AI (24%), and proving ROI to leadership (18%).

Developers Don't Trust How AI Metrics Will Be Used

Even as productivity dashboards show green, developers are uneasy about how that data will be used. Part of the problem is structural: measurement systems are most often built top-down by leadership, without structured input from the practitioners being measured. When frameworks reflect only the leadership view, they systematically undercount the pressures developers are actually experiencing.

  • The perception gap is wide. Managers are nearly four times more likely than practitioners to report no concerns about how AI productivity data might be used to evaluate them (15% vs. 4%).
  • Fear of surveillance is widespread. 54% fear individual performance evaluations based on AI data. In addition, 46% of respondents cite struggling with pressure to work faster than is sustainable, and the same share report privacy or surveillance concerns.
  • Developers want a say in how they're measured. 55% want a clear separation between improvement data and performance evaluation, 50% want transparency about what's being measured, and 49% want to be involved in defining the metrics themselves.

What Engineering Leaders Should Measure Now

The frameworks engineering organizations rely on — velocity, DORA, cycle time, developer experience surveys — still work. They just weren't designed for what AI has changed about the work itself.

To capture AI's benefits without missing its costs, Harness recommends that engineering organizations:

  • Start measuring the new unit of work. Add code quality, validation time, cognitive load, and burnout indicators alongside the frameworks built around velocity and cycle time.
  • Treat AI performance as its own discipline. Track AI agent accuracy, acceptance, and cost separately from human developer output, with a shared definition of "good" across the organization.
  • Separate improvement data from performance evaluation. Build the measurement system with developers. Be explicit about how the data will be used, and involve developers in defining the metrics.

Methodology: This report is based on a survey of 700 software engineering practitioners and managers from large enterprises, commissioned by Harness and conducted by independent research firm Sapio Research in April 2026. The sample included 300 respondents from the United States and 100 each from the United Kingdom, India, France, and Germany.

Explore all 53 articles in today's digest

Fresh dev news, AI-digested daily. Never miss a trend at devoured.fyi