Conversion reporting you can reproduce, from order-state ledger to causal boundary

Ecommerce conversion measurement should answer one merchant decision with reproducible definitions and reconciled customer states. A report is not ready merely because it contains a conversion rate. The reader must be able to see who and what was counted, which records disagree, what data are missing, how uncertainty was handled and whether the result is observed, attributed or causal.

The operating case is an England online merchant serving adults. UK and Great Britain official sources retain their stated geography. Research closed on 6 September 2026, and the guide supplies no benchmark, rate, uplift or merchant outcome.

What to take away

Begin with the seller and decision

Write the legal seller, product or service, customer population, England geography rule, channel and decision period. Use Europe/London as the reporting timezone and document daylight-saving handling. Name the current journey, proposed change, decision owner and safe alternative.

State the question in operational terms. For example, the merchant may need to know whether a revised product-information route changes retained orders without increasing unclear prices, accessibility failures, tracking without the required permission, payment errors or customer remedies. That is narrower than asking whether conversion improved.

Define the identity unit before counting. A browser, device, account, person, basket, order and payment transaction are different units. Cross-device matching may be incomplete or impermissible. Record the identifier, join logic, known collisions, missingness and change history rather than pretending to count unique people.

Build a measurement specification

Every measure needs the event or state, eligible population, numerator, denominator or unit, period, timezone, source systems, query version, exclusions, late-data rule, missing-data treatment, duplicate handling, bot and staff treatment, owner, uncertainty and recheck trigger. Keep those fields beside the figure in the report.

The UK Statistics Authority's Code of Practice for Statistics formally governs official statistics, but its quality principles can inform other analysis. It stresses suitable data, rigorous methods, coherence, comparability and openness about quality. Applying those principles voluntarily does not make a merchant dashboard an official statistic.

Version the business definition separately from the query. A stable query can become wrong when the checkout, payment integration, product mix or return window changes. A revised query can break a time series even when the label is unchanged. Record both changes and explain their effect.

Preserve the customer-state ledger

Keep order placed, payment authorised, payment captured, order confirmed, fulfilled and retained as separate states. Add cancelled, returned, refunded and disputed. The government's online-selling guidance locates customer information, total price, payment obligation and confirmation within the transaction. Its returns page explains why later remedies matter, subject to exceptions.

A checkout completion belongs to the store system. Authorisation and capture belong to payment records. Settlement belongs to provider or bank evidence. Revenue and contribution belong to accounting records under approved policies. None is a synonym for another.

Reconcile transitions. Count orders that remain authorised but uncaptured, captured but unconfirmed, fulfilled but later returned, or refunded but not yet reflected in the cash record. Give each disagreement an owner and ageing rule. Do not delete exceptions to make totals agree.

Retained order must have a stated post-order window. The same order may appear retained at an early cut and refunded later. A report should show whether that window is complete, which records arrived after the cut and when the result will be restated.

Separate observation from explanation

Observation describes recorded events under stated rules, and reconciliation compares systems meant to describe the same commercial state. Descriptive association examines patterns without claiming causation.

Platform attribution assigns credit by a documented rule, while marketing mix modelling estimates relationships from aggregated series and assumptions.

Causal studies estimate the difference between the observed intervention and a defensible account of what would otherwise have occurred.

These outputs should occupy different report fields. A platform-attributed order is not causal incrementality. A model coefficient is not a directly observed customer event. An observed rise is not an effect unless the design supports that conclusion.

HM Treasury's May 2026 Magenta Book is central-government evaluation guidance rather than a retail benchmark. Its impact-evaluation quality guidance distinguishes outcome monitoring from attribution and explains the counterfactual. The merchant's statistician should choose a proportionate design and state its limits.

Specify an experiment before exposure

For an A/B or other controlled comparison, predeclare the eligible population, allocation unit, assignment method, variants, actual-exposure definition, primary outcome, post-order window, exclusions, missingness, contamination, guardrails, stopping rule, uncertainty method and analysis owner. Preserve assignment and exposure separately. A customer may be assigned but never load the variant.

Record concurrent changes such as stock, price, promotion, fulfilment, payment availability and outages. The May 2026 Magenta Book analytical-methods annex discusses experimental and quasi-experimental approaches and their limits. It does not provide a universal store sample size or stopping threshold.

Qualified statistical review must occur before exposure and again before interpretation. Early stopping after a favourable fluctuation, changing the primary outcome or excluding inconvenient states weakens the claim. Operational stop conditions for consumer harm, safety, accessibility, privacy, payment or security remain independent of the statistical plan.

Keep permission separate from validity

Inventory cookies, pixels, scripts, SDKs, server events and identity joins. Record purpose, data, device operation, recipient, permission route, retention and deletion. The ICO's final April 2026 storage and access technologies guidance supplies the current PECR and related data-protection boundary.

A permitted operation may still produce biased or incomplete data. A statistically useful event may still lack the necessary privacy basis. The privacy reviewer decides permission; the statistician decides analytical fitness. Neither approval substitutes for the other.

The ICO's data-minimisation guidance requires processing to be adequate, relevant and limited to what is necessary, while noting current DUAA-related context. Do not collect extra identifiers simply to make joins easier. Recheck the page and the actual purpose before publication or configuration change.

Report customer-protection guardrails

Measurement cannot compensate for a defective journey. Show price-integrity, optional-choice, accessibility, product-safety, payment and security exceptions beside the commercial states.

The CMA's price-transparency guidance addresses mandatory fees, taxes and charges. Its additional-charge guidance covers active agreement for optional payments. A design that records more orders after obscuring a charge is not an acceptable success.

For goods, current OPSS product-safety guidance supports separate traceability and corrective-action work. The dashboard should expose a safety hold or withdrawal rather than blend it into a conversion denominator.

Accessibility reporting needs affected task, environment, method, evidence and reviewer. Cover information, checkout, authentication, payment, cancellation and support. A raw automated issue count is neither a usability conclusion nor a legal verdict.

Reconcile payment and money measures

Identify the provider entity and source behind authorisation, capture, settlement, refund and dispute. The FCA's payment-provider guidance explains register and protection boundaries; regulatory status does not validate a dashboard or settle an accounting question.

Gross order value, settled cash, recognised revenue and net contribution need distinct definitions. Finance records VAT and accounting basis, cost period, refunds and reversals. Analysts should not invent a contribution figure by subtracting partial costs from attributed order value.

If ROI is in scope, its numerator requires defensible causal incremental contribution. Its denominator requires reviewed whole-life economic cost. Keep both blank when evidence is incomplete. An observed conversion change or platform-attributed revenue is not an admissible substitute.

Design the dashboard around decisions and exceptions

The first panel states the decision, population, period, timezone and definition version. The next reconciles the state ledger. A third shows data quality: missing, late, duplicate, bot, staff, unmatched and contradictory records. Guardrail queues show unresolved customer, accessibility, safety, privacy, payment and security issues.

Only then show descriptive or causal results, labelled by evidence class. Add uncertainty and limitations on the same page. Buyer-set thresholds remain blank until owners approve them. There is no universal ecommerce conversion target for England.

The NCSC's logging guidance supports deciding which records can answer incident questions and how they are retrieved and protected. Measurement and security logs may overlap, but their purposes, access and retention still need review.

Set correction and recheck rules

Record who can correct a definition, restate a period, withdraw a result or stop a report. Preserve the original figure, reason, affected decisions and replacement version. A silent dashboard edit damages the evidence trail.

Reopen review when the seller, product, customer population, journey, timezone logic, event schema, identity rule, permission route, payment provider, return window, query or evaluation design changes. Also recheck source guidance before publication. The next useful task is to create one blank measurement specification and reconcile a single reporting period before anyone discusses performance.

Before you act

Common questions

What should a measurement specification include?

Each measure needs a defined event or state, an eligible population, a numerator and a unit. It also needs a period, timezone, source systems and query version. Add exclusions, late-data and missing-data rules, duplicate handling, owner, uncertainty and recheck trigger. Keep those fields beside the figure in the report.

How should a retained order be defined?

It needs a stated post-order window, since an order kept at an early cut may be refunded later. The report should say if that window is complete and when the figure will be restated.

Why keep permission separate from validity?

A permitted operation can still yield biased or incomplete data, and a useful event may lack a privacy basis. The privacy reviewer judges permission and the statistician judges analytical fitness; neither approval replaces the other.