Early in my career, I treated every product experiment like an exam.
If the experiment improved the metric we were targeting, it was a success.
If it didn’t, I quietly moved on and looked for the next idea.
Looking back, I realize I was focusing on the wrong outcome.
The purpose of an experiment isn’t to prove you’re right.
It’s to reduce uncertainty.
Some of the most valuable lessons I’ve learned as a Product Manager have come from experiments that didn’t produce the results I expected.
A Failed Experiment Isn’t Always a Failed Decision
It’s easy to celebrate winning experiments.
They give us positive results, confidence, and a clear direction.
Failed experiments are different.
They can feel disappointing, especially after weeks of research, design, development, and analysis.
But I’ve come to believe that an experiment only truly fails when it leaves you with no new understanding.
If you now know that a customer assumption was wrong, that’s progress.
If you’ve ruled out an approach that looked promising, you’ve reduced future risk.
That’s valuable knowledge.
Every Experiment Tests an Assumption
One thing that changed my mindset was realizing that we’re rarely testing features.
We’re testing assumptions.
We might assume that simplifying onboarding will increase activation.
Or that adding recommendations will improve engagement.
Or that a new pricing model will increase conversions.
When an experiment doesn’t produce the expected outcome, it doesn’t necessarily mean the feature was bad.
It simply means one of our assumptions didn’t hold true.
That’s an important distinction.
Because assumptions can be challenged.
Egos shouldn’t be.
Sometimes the Metric Isn’t the Problem
I remember working on an initiative where we were confident a particular improvement would increase feature adoption.
The experiment ran successfully.
The implementation worked exactly as intended.
But the adoption numbers barely changed.
At first, it felt like the experiment had failed.
Later, after speaking with customers, we discovered something we hadn’t considered.
The issue wasn’t that users couldn’t find the feature.
They simply didn’t need it as often as we had assumed.
The experiment answered a question we didn’t even know we were asking.
That’s the kind of insight that changes future product decisions.
Negative Results Narrow the Path Forward
When people hear that an experiment didn’t work, they often think nothing was achieved.
I see it differently.
Every negative result removes one possible path.
That may not sound exciting, but product management is often about eliminating uncertainty.
The fewer assumptions you’re carrying into future decisions, the stronger those decisions become.
Sometimes knowing what doesn’t work is just as valuable as knowing what does.
Share the Experiments That Didn’t Work
One thing I’ve noticed is that teams love sharing successful experiments.
The unsuccessful ones quietly disappear.
That’s a missed opportunity.
Failed experiments often prevent other teams from repeating the same mistakes.
They create organizational knowledge.
I’ve worked with teams where documenting unsuccessful experiments became just as valuable as documenting successful ones.
It encouraged curiosity instead of blame.
And over time, people became more willing to test bold ideas because failure wasn’t treated as wasted effort.
Be Careful About Calling Something a Failure
Not every experiment that misses its target is actually unsuccessful.
Sometimes the timing is wrong.
Sometimes the audience is too small.
Sometimes the metric you’re measuring isn’t the best indicator of success.
I’ve learned to spend as much time understanding the result as I spend looking at the number itself.
The data tells you what happened.
The conversations usually explain why.
Build a Culture That Rewards Learning
The best product teams I’ve worked with weren’t obsessed with being right.
They were obsessed with learning quickly.
That doesn’t mean celebrating every failed idea.
It means recognizing that thoughtful experiments reduce uncertainty, regardless of the outcome.
When teams become afraid of failure, they naturally stop experimenting.
And when experimentation slows down, learning slows down too.
Final Thought
Over time, I’ve realized that successful Product Managers aren’t defined by how many winning experiments they run.
They’re defined by how well they learn from the ones that don’t.
Every experiment leaves you with a choice.
You can see it as evidence that your idea failed.
Or you can see it as evidence that you’ve become a little less uncertain than you were yesterday.
For me, that’s what experimentation has always been about.
Not proving that we’re right.
But learning enough to make the next decision better than the last.
Leave a Reply