Over the past year, I’ve noticed a question showing up in almost every conversation about AI.
“How much more productive are we now?”
It’s a reasonable question.
Companies are investing heavily in AI. Leaders want to justify those investments. Product teams want to demonstrate impact.
So we reach for the easiest number we can find.
Tasks completed.
Hours saved.
Lines of code generated.
Documents summarized.
Support tickets resolved.
The problem is that productivity is often the easiest metric to measure and the hardest one to interpret.
Faster Doesn’t Always Mean Better
Imagine a team using an AI assistant to write meeting summaries.
Previously, the task took 30 minutes.
Now it takes five.
On paper, productivity has increased by 83%.
That’s impressive.
But what happened to those 25 minutes?
Were they invested in customer research?
Product strategy?
Better product decisions?
Or were they simply filled with more meetings and more work?
Measuring time saved tells us very little about whether more value was actually created.
AI Often Changes the Nature of Work
One thing I’ve observed is that AI doesn’t simply make people work faster.
It changes what people spend their time doing.
A software engineer may write less boilerplate code and spend more time reviewing architecture.
A Product Manager may spend less time documenting requirements and more time validating customer problems.
A designer may generate concepts faster but spend more time evaluating trade-offs.
If the work itself changes, measuring only output becomes an incomplete picture.
Quantity Is Easier to Measure Than Quality
This is where many AI discussions become misleading.
Suppose a customer support team resolves twice as many tickets after introducing AI.
That sounds like a success.
But did customer satisfaction improve?
Were problems actually solved?
Did customers need to contact support again because the responses lacked context?
Similarly, an AI coding assistant might help developers produce more code.
But if defects increase or maintainability suffers, was the team truly more productive?
Productivity metrics often reward volume because quality is much harder to measure.
The Biggest Value of AI Is Often Invisible
Some of AI’s most valuable contributions never appear on a productivity dashboard.
It helps people explore more ideas before choosing one.
It reduces the fear of starting from a blank page.
It allows teams to test multiple approaches quickly.
It makes specialized knowledge more accessible.
None of these benefits are easy to express as “hours saved.”
Yet they often have a much greater impact on long-term outcomes.
Productivity Can Create the Wrong Incentives
One concern I have is that productivity metrics can unintentionally change behavior.
If success is measured by documents produced, people will produce more documents.
If it’s measured by code written, developers may write more code.
If it’s measured by tickets closed, support teams may optimize for speed instead of resolution.
We’ve seen similar problems long before AI.
Goodhart’s Law captures it well:
“When a measure becomes a target, it ceases to be a good measure.”
The same risk applies to AI productivity.
Measure Outcomes Instead
As Product Managers, I think we should ask different questions.
Instead of:
“How much faster did AI make this process?”
Consider asking:
- Did customers achieve their goals more easily?
- Did decision quality improve?
- Did we reduce costly mistakes?
- Did teams learn faster?
- Did customer satisfaction increase?
- Did the business create more value?
Those questions are harder to answer.
They’re also much closer to what actually matters.
AI Is an Amplifier, Not the Outcome
One lesson I’ve taken away from working with AI is that it amplifies existing systems.
A well-designed process often becomes more effective.
A poorly designed process may simply produce poor results more quickly.
That’s why measuring AI itself can be misleading.
The technology isn’t the outcome.
It’s one contributor to it.
What ultimately matters is whether customers, employees, and businesses achieve better results.
Final Thought
I don’t think productivity is a useless metric.
It can highlight efficiency gains and reveal opportunities for improvement.
The problem begins when we treat it as the primary measure of AI’s success.
AI isn’t valuable because it helps us do more work.
It’s valuable because it gives us the opportunity to do better work.
As Product Managers, that’s the distinction I believe we should focus on.
Because customers rarely care how quickly we produced something.
They care whether what we built actually solved their problem.
Leave a Reply