Experimentation is often treated as a numbers game.

Run more experiments.

Ship more variants.

Increase the percentage of the product covered by testing.

But there is another question product teams should ask:

Are our experiments actually worth the investment?

Running an experiment has a cost. Product, design, engineering, analytics, and research time all go into designing, launching, monitoring, and interpreting it.

That makes Experiment ROI worth measuring.

Start With the Value of the Decision

The ROI of an experiment isn’t necessarily the revenue generated by the winning variant.

Sometimes the biggest value comes from avoiding a bad product decision.

Imagine your team is considering a major change to an important workflow.

Building and rolling it out would take three months.

A two-week experiment shows that the proposed change doesn’t improve the target outcome.

You may have generated zero incremental revenue.

But you potentially avoided three months of engineering work and the cost of introducing a worse experience to thousands of users.

That’s still valuable.

Separate Experiment Cost From Product Impact

Start by estimating what the experiment required.

This could include:

• Engineering time
• Product and design time
• Analytics and research
• Infrastructure costs
• Opportunity cost of delaying other work
• Time spent analyzing and communicating results

Then look at what the experiment produced.

Did it generate incremental revenue?

Improve conversion?

Reduce churn?

Increase adoption?

Improve efficiency?

Or prevent the team from making a poor decision?

The comparison doesn’t have to be perfectly financial to be useful.

Don’t Only Count Winning Experiments

If you calculate ROI only from experiments that produced positive results, experimentation will look artificially successful.

A failed experiment can create substantial value.

Suppose a team expects a new feature to increase activation by 10%.

The experiment shows no meaningful improvement.

That result prevents the team from investing further in the feature and redirects resources elsewhere.

The experiment didn’t “fail.”

The hypothesis failed.

The organization learned something valuable at relatively low cost.

Consider the Size of the Decision

Not every experiment deserves the same level of investment.

A small button change might require a few hours of work.

A major pricing experiment could involve weeks of preparation and significant risk.

The potential value of the decision should influence how much you invest in learning about it.

This is where expected value becomes useful.

If a decision could materially affect revenue, retention, or customer experience, spending more to reduce uncertainty may be justified.

Measure Learning Velocity

Not every experiment produces an immediate business outcome.

Some experiments answer important product questions.

For example:

Will customers understand this workflow?

Does this onboarding approach help users reach value faster?

Is this problem important enough for customers to change their behaviour?

These answers can shape future decisions.

One useful way to think about this is learning per unit of effort.

How much uncertainty did the experiment remove relative to the time and resources invested?

This can be particularly valuable for early-stage products where learning is often more important than short-term optimization.

Look at the Opportunity Cost

Experimentation itself competes for resources.

If your engineering team spends two weeks building an experiment, they aren’t spending those two weeks on something else.

That doesn’t mean experimentation should be minimized.

It means teams should be selective.

Ask whether the question is important enough to test.

If the result wouldn’t change your decision, the experiment probably isn’t worth running.

Track the Portfolio, Not Just Individual Experiments

A single experiment can be misleading.

Look across your experimentation program.

How many experiments produce meaningful product changes?

How often do experiments invalidate existing assumptions?

How much engineering effort goes into low-value tests?

How quickly do teams move from result to decision?

Over time, this gives you a better picture of whether experimentation is becoming a productive part of the product development process.

Final Thought

The ROI of experimentation isn’t simply the revenue generated by successful tests.

It’s the value of making better decisions with less uncertainty.

Sometimes that means discovering what works.

Sometimes it means discovering what doesn’t.

And sometimes the most valuable experiment is the one that stops your team from spending six months building something customers don’t actually need.