Conversion ROI stays blank until the cost is whole and the uplift is causal

Do not calculate return from a raw conversion-rate change. ROI needs reviewed whole-life economic cost and a defensible estimate of incremental contribution caused by the intervention.

What to take away

Define the commercial outcome and returns rule

State legal seller, product, eligible adult customer population, England geography rule, channel, period and timezone. Track order placed, payment authorised, captured, fulfilled, retained, cancelled, returned, refunded and disputed. The government's returns guidance shows why relevant cancellation and refund work can continue after an online sale, subject to exceptions.

Use retained net contribution, not gross order value. The blank formula is:

Retained net contribution = [settled cash from eligible retained orders] - [refunds] - [disputes] - [product and fulfilment variable cost] - [other approved variable costs].

Build the full denominator of whole-life economic cost

An accountant must define revenue recognition and cost treatment. A VAT adviser handles the transaction's actual basis. HMRC's VAT records guidance establishes the record boundary, not the answer.

Whole-life economic cost has four groups. Build and design covers product and claim work, research, design and development. Experimentation and analytics covers test build, analytics setup and reporting. Assurance covers privacy and PECR, consumer review, accessibility, product safety, security and QA. Rollout and operations covers payments and fraud, monitoring, fulfilment or support effects, refunds, correction, rollback and exit.

Typical effort for a single A/B test is 20 to 40 hours of engineering and analyst time, plus analytics and review. For illustration, a £1,000 supplier invoice plus VAT at 20% gives a £1,200 cash cost before any recovery. That is a typical range anchor, not a quote for every project.

Add supplier invoices and buyer labour separately. Use a finance-approved loaded labour method. Keep cash timing, accounting entries and opportunity cost in distinct views. Do not treat free software or an existing employee as costless.

Pull inputs from named sources. Use the finance ledger (for example Xero or Sage) for settled cash, refunds, disputes and supplier invoices.

Use the ecommerce platform and payment processor for order, payment and refund states. Use the experimentation platform (for example Optimizely or VWO) for assignment, exposure and variant logs. Use analytics (for example Google Analytics 4) for on-site behaviour and conversion events.

Establish causal incrementality with a counterfactual

Record the observation window and late-state rule. A purchase retained at the first report may later be returned or disputed. Do not silently change the window between the intervention and counterfactual groups. If post-order data are incomplete, show that uncertainty and defer the financial conclusion.

Observed orders answer what happened. Platform attribution allocates credit by a stated rule. Neither alone answers what the change caused. Stock, product mix, promotion, season, outages, payment acceptance and fulfilment may move at the same time.

HM Treasury's May 2026 Magenta Book is central-government evaluation guidance, not a retail benchmark. Its quality in impact evaluation page distinguishes monitored outcomes from attribution and explains the role of a counterfactual.

Before exposure, predeclare eligible units, allocation, comparison, intervention, primary outcome, post-order window, exclusions, missingness, uncertainty, validity risks and stopping rule. A statistician and causal evaluator decide whether the design supports an effect estimate.

Preserve allocation and actual exposure separately. A customer assigned to a variant may not load it, and a returning person may cross devices or channels. Record contamination, outages, stock changes, promotion and fulfilment disruption. The analysis plan states whether and how these issues are handled before results are seen.

Causal incremental contribution = [reviewed intervention contribution] - [reviewed counterfactual contribution], with only approved adjustments.

Keep the calculation blank until admissible

ROI = ([causal incremental contribution] - [whole-life economic cost]) / [whole-life economic cost].

Worked example. A test group has 1,000 eligible retained orders. Settled cash from those orders is £50,000. Refunds are £2,000, disputes £500, product and fulfilment variable cost £30,000, and payment processing fees £1,500. Retained net contribution is £50,000 - £2,000 - £500 - £30,000 - £1,500 = £16,000. The reviewed counterfactual gives £13,500.

Causal incremental contribution is £16,000 - £13,500 = £2,500. Whole-life economic cost is £1,000 in this example. ROI is (£2,500 - £1,000) / £1,000 = 150%. If cost uncertainty gives a £1,000 to £1,500 range, the ROI range is 67% to 150%.

Do not calculate if the cost denominator is zero, partly omitted or measured for a different period. Report a range or uncertainty interval when the method supports one, plus sensitivity to each material input. Do not present an attributed value as the upper bound of a causal result.

Keep downside and stopping decisions outside the headline ratio. Report consumer complaints, accessibility defects, safety holds, payment failures, data incidents and refund disruption under their own definitions. Any one may stop the change even when the estimated contribution is positive.

Customer protections remain stop conditions. The CMA's price transparency guidance applies independently to mandatory fees and taxes. A favourable calculation cannot compensate for unclear pricing, inaccessible service, unsafe goods, unapproved tracking, payment failure or broken refunds.

The decision can continue, revise, stop or collect more evidence. This method does not promise a return and is not personal financial, accounting or tax advice.

Before you act

Common questions

What inputs does ROI require before it can be calculated?

ROI needs a reviewed whole-life economic cost and a defensible estimate of incremental contribution caused by the intervention. The cost denominator must be reviewed and measured for the same period as the contribution.

Why is platform attribution not enough to show what a change caused?

Observed orders show what happened in the test group, but they do not isolate the counterfactual. You need a predeclared comparison, clean allocation and recorded exposure to estimate what the change caused. Stock, product mix, promotion, season, outages, payment acceptance and fulfilment may all move at the same time as the intervention.

When should the financial conclusion be deferred?

If post-order data are incomplete, defer the financial conclusion until returns and disputes are resolved. A purchase retained at the first report may later be returned or disputed.