Your engineering org adopted an AI coding assistant some time in the last two years, probably without a formal evaluation, and someone has now asked whether it is working. The honest answer is that most organisations have no idea, because they are measuring the half of the pipeline that got faster.
There is real data on this now, and it is more interesting than either the vendor claims or the backlash.
What the evidence actually shows
DORA's 2025 survey of roughly 5,000 technology practitioners found something that reads as contradictory until you sit with it. Higher AI adoption was associated with more delivery throughput — a reversal of the prior year's finding — and also with more delivery instability.
Both at once. Teams ship more and break more.
DORA's own follow-up analysis frames AI as an amplifier rather than an accelerator, and that framing does most of the explanatory work. The assistant is genuinely fast at the first eighty percent: scaffolding, boilerplate, a plausible first implementation, test stubs. It is much less help at the remaining work, which is integration with systems that already exist, the edge cases nobody wrote down, and the production hardening that turns something that runs into something you can page someone about at 3am.
So the gain lands in one place and the cost lands in another. If you measure the first and not the second, the tool looks transformative. If you measure the second and not the first, it looks like a liability. Neither is the whole picture.
The adoption figure is worth noting too: around 90% of surveyed practitioners report using AI at work. Whatever you decide, you are not deciding whether to adopt. You are deciding what to put around something your team is already using.
The metrics that will mislead you
Most AI tooling dashboards report the things that are easy to instrument, which are almost exactly the things that do not matter.
Suggestion acceptance rate measures how often a developer pressed tab. It is not a productivity signal; a high acceptance rate is as consistent with uncritical acceptance as with good suggestions, and those have opposite consequences.
Lines of code generated is worse. More generated code is a cost as well as an output — it has to be reviewed, maintained, understood by someone in eighteen months, and carried forward. A team that generated fewer lines to achieve the same outcome did better, and this metric scores them lower.
Self-reported time saved is the one that gets quoted in business cases and it is the least reliable of the three. People are poor at estimating counterfactual time, and there is a well-documented gap between how fast AI-assisted work feels and how long it takes. Ask the question by all means, but do not build a number on the answer.
Per-developer AI usage is the one to refuse outright. It will be gamed within a sprint of anyone learning it exists, and it punishes the engineer who uses judgement about when the assistant helps — which is precisely the behaviour you want. This is productivity theatre with a new input.
What to measure instead
The useful frame is that AI assistance is a change to your delivery system, so you measure it the way you would measure any other change to that system: at the system level, before and after, including the parts you did not expect to move.
Four delivery signals, which you may already have from DORA and SPACE:
Throughput, because it should go up and if it has not you have a different problem. Change failure rate, because this is where the cost shows up. Time to restore, for the same reason. And lead time for changes, which catches the case where code is written faster but takes just as long to reach production because review and integration absorbed the gain.
Then two that are specific to this change:
Review cycle time and review load. If generated code arrives faster than it can be reviewed, review becomes your constraint, and the failure mode is that teams quietly lower the bar to keep up. Watching review queue depth is the earliest warning that this is happening.
Rework rate — how much of what shipped gets materially rewritten within some window. This is the clearest single measure of whether velocity was real or borrowed.
The team-level questions worth asking
Delivery metrics tell you what happened. They do not tell you why, and the why is where the intervention lives.
Has the work changed shape? AI assistance tends to compress the time spent writing and expand the time spent reading, reviewing and reconciling. If your engineers' days have shifted from authoring to reviewing, that is a real change in the job, and it has consequences for how draining the work is that no delivery metric captures.
Where is the assistant not helping? The honest answer from most teams is: the hard parts. Debugging a race condition in a system with ten years of history, deciding whether a feature should exist, untangling a data model that encodes three abandoned product directions. If your senior engineers' throughput has not moved, that is not a failure of adoption — it is a signal about what the tool does and does not do.
Is anyone slower? Occasionally yes, and it is usually someone working in an unusual codebase, an unusual language, or at a level of abstraction where plausible-but-wrong suggestions cost more to evaluate than to write from scratch. This is worth knowing and it is invisible in an aggregate.
Activity data helps with the first and third of these — it shows where time actually goes across tools, and whether the balance of authoring to reviewing has shifted. It cannot tell you the second; that needs asking people. Our note on why your best engineer looks unproductive covers why the aggregate is the wrong unit for the senior case specifically.
Worth a glance at the other well-documented intervention that claims to raise output per hour: what the four-day week data actually supports, where the measured revenue effect turns out to be a fraction of the figure usually quoted.
What to do with the answer
If throughput is up and stability is flat, you are in the good case. Invest in keeping it there, which per DORA's own recommendation means automated testing, continuous integration and small batches — the same practices that were a good idea before any of this.
If throughput is up and stability is down, you have not gained capacity, you have moved work downstream to review, QA and on-call. That is sometimes a reasonable trade and sometimes an expensive one, but it should be a decision rather than a surprise. The lever is usually review capacity and test coverage, not the assistant.
If nothing moved, check whether the tool is actually being used before concluding it does not work, and check whether your constraint was ever code-writing speed. For a lot of teams it was meeting load or context switching, and no coding assistant touches either.
Where ProdView fits
ProdView measures activity and application metadata, so it can show how the shape of engineering time has changed — authoring versus reviewing, focus time, fragmentation, how much of the week sits in the editor versus the review tool versus everything else. That is the "has the work changed shape" question, measured at team level rather than per person.
When not to pick us: you want DORA delivery metrics. Those come from your CI, deploy and incident systems, not from desktop activity, and a tool claiming to derive change failure rate from activity data is selling you something that does not work. Use both; they answer different halves of this.
Try before you commit
Three seats are free forever, which is enough to see whether the authoring-to-reviewing balance on your team has actually shifted. The ROI calculator is free and needs no signup.
DORA findings as summarised from its 2025 State of AI-assisted Software Development survey and subsequent analysis; check dora.dev directly for exact figures before citing them in a business case.