Contents
The Question Every CTO Gets Asked
Sooner or later, a board member or CEO asks: "How productive is engineering?" Usually it's a fair question with an anxious subtext. Engineering is a large cost, its output is hard to see, and other functions have numbers.
The wrong responses are defensiveness ("you can't measure engineering") and false precision (a dashboard of commits per engineer). The right response is a small set of measures that reflect outcomes, a clear line between what goes to the board and what stays in teams, and an honest explanation of what the numbers can't tell you.
The Frameworks
DORA's four keys
The DORA research programme gives the best-established delivery metrics:
- Deployment frequency: how often you release to production.
- Lead time for changes: from commit to running in production.
- Change failure rate: the share of deployments that cause a failure needing remediation.
- Time to restore service: how quickly you recover (recent reports call this failed deployment recovery time).
Their strength is that throughput and stability are measured together, and years of research show high performers do well on both. Their weakness is scope: they describe the delivery pipeline, not whether you built the right thing or whether engineers are drowning.
SPACE
The SPACE framework (Forsgren and colleagues, 2021) argues that productivity is multidimensional: Satisfaction and well-being, Performance, Activity, Communication and collaboration, Efficiency and flow. Its key contribution is a warning: never rely on a single dimension, and combine system data with what people tell you. It's a way of thinking rather than a metric set, which makes it hard to operationalise on its own.
DX Core 4
DX Core 4, published by DX in December 2024, tries to unify DORA, SPACE and developer experience research into a practical set of four dimensions, each with a key metric:
| Dimension | Key metric | What it tells you |
|---|---|---|
| Speed | Diffs (pull requests) per engineer, at team or organisation level | Throughput of the delivery system |
| Effectiveness | Developer Experience Index (survey-based) | Friction engineers face day to day |
| Quality | Change failure rate | Whether speed is costing stability |
| Impact | Percentage of time spent on new capabilities | How much effort goes to new value versus maintenance and toil |
It's too new to have a track record, but I like that it's explicitly designed to be reported upwards, balances speed against quality and experience, and includes an impact measure a CFO understands. The "diffs per engineer" metric is contentious; DX's own guidance is to look at it in aggregate, never to target it for individuals.
Developer experience surveys
Whatever framework you use, a regular survey is the cheapest high-value instrument you have. A short quarterly survey asking about build times, flaky tests, clarity of priorities, deployment confidence and focus time will find bottlenecks that no system metric reveals. Keep it short, report results back to teams quickly and visibly act on something, or response rates collapse.
What Goes to the Board vs What Stays in Teams
The audience determines the metric. Boards need trends and business relevance; teams need diagnostic detail.
| Audience | What they should see | What they shouldn't |
|---|---|---|
| Board | Quarterly trends: delivery speed, change failure rate, share of effort on new capability versus maintenance, key reliability events, developer experience trend | Team-level comparisons, individual data, raw activity counts |
| Leadership team | Monthly view by product area, investment allocation, major delivery risks | Individual data |
| Engineering teams | Full DORA metrics, survey detail, cycle time breakdown (coding, review, deploy), flaky test and build data | Anything used to rank people |
For the board I pair a single page of trend lines with narrative: what changed, why, and what we're doing about it. At Freddie's Flowers, the moments that matter commercially are peaks such as Valentine's Day and Mother's Day, so reliability and change safety around those periods get explicit mention. Numbers without narrative invite misinterpretation.
The Pitfalls
Lines of code. More code is not more value; often the best change deletes code. Anyone who has managed a replatforming away from a legacy monolith knows the most valuable weeks can have negative line counts.
Story points. Points are a team-internal estimation tool. The moment velocity is reported upwards or compared between teams, points inflate and the tool stops working for planning. Goodhart's law applies in full.
Individual metrics. Individual commit counts, pull request counts or review counts measure activity, not contribution. The engineer who unblocks three others, reviews carefully and fixes the build pipeline looks unproductive on these numbers. Individual metrics also destroy trust in the whole measurement effort. Performance is assessed through managers who understand the work, not dashboards.
Measuring without acting. A metric nobody acts on is overhead. Every measure should have an owner and a question it helps answer.
How AI Assistants Complicate Measurement
AI coding assistants have made the measurement problem harder in three ways.
Activity metrics inflate. If assistants make it easier to produce code and pull requests, activity-based measures rise whether or not outcomes improve. Diffs per engineer going up after an AI rollout is not, by itself, evidence of success.
Costs move downstream. Faster generation can mean larger changes, heavier review load and more rework. The 2024 DORA report found that greater AI adoption was associated with a small drop in delivery throughput and a larger drop in delivery stability, even though individuals reported feeling more productive. That gap between perceived and system-level productivity is exactly what you need to measure.
Vendor metrics are tempting. Acceptance rates and "percentage of code written by AI" are easy to get and easy to present. They measure tool usage, not engineering outcomes.
My approach is to measure AI adoption as an input and judge it on the same outcome metrics as everything else: lead time, change failure rate, review time, rework and developer experience. Compare trends before and after rollout, or between groups, and add survey questions about where assistants help and where they create extra work.
Advice by Stage
- Startup: skip frameworks. Track deployment frequency and incidents, talk to engineers weekly, and run a simple quarterly survey.
- Scaleup: adopt DORA metrics from your pipeline, add a developer experience survey and start reporting a small, stable set of trends to the board. DX Core 4 is a sensible template.
- Enterprise: invest in consistent instrumentation across teams, a dedicated developer productivity function and investment allocation reporting, while guarding hard against metrics being used for individual performance.
The Takeaway
There is no single number for engineering productivity, and anyone selling one is selling a proxy. Use DORA for delivery, surveys for experience and an impact measure for business value, report trends with narrative to the board and keep diagnostic detail within teams. Be especially sceptical of activity metrics in an era of AI assistants, and judge the tools on outcomes, not on how much code they produce.