Measurement
Part of Conversion reporting you can reproduce, from order-state ledger to causal boundary
Ecommerce conversion reporting dashboard: specification
Specify an ecommerce conversion reporting dashboard with a defined conversion rate, reconciled order states, data-quality queues and correction controls.
An ecommerce conversion reporting dashboard needs definitions, reconciliation and exception queues before charts. Build it for one legal seller, product and customer population, England geography rule and Europe/London reporting period. Leave all performance thresholds blank until their owners approve them from evidence.
What to take away
- Build the dashboard for one legal seller, product, customer population, geography rule and reporting period before adding any charts.
- Define conversion as an eligible store session containing at least one retained order; report the conversion rate as those sessions divided by all eligible store sessions in the same reporting cohort, with exclusions and the retained-order window.
- Leave performance thresholds blank until their owners approve them from evidence.
- Reconcile placed, authorised, captured, confirmed, fulfilled and retained counts separately rather than forcing totals to match.
- Give data quality its own panel covering refused tracking, missing events, late arrivals, duplicates, bots and unmatched identities.
- Keep observed events, reconciled states, attribution, modelled estimates and causal effects in separate labelled layers.
Put the specification above the result
The header should show decision, current journey, intervention if any, channel, identity unit, period, timezone, data cut, restatement date, definition version, event-schema version, query version and accountable owner. Add links to the source records and change log.
Dashboard header specification
- Decision, journey, intervention, channel
- Identity unit, period, timezone, data cut
- Restatement date, definition version
- Event-schema version, query version
- Accountable owner and change log
Expose for each displayed value: event or state; eligible population; numerator; denominator or unit; exclusions. Also expose late and missing data; duplicates; bot and staff handling; uncertainty; recheck trigger.
A merchant conversion dashboard is for operational and experimental decisions, not official statistics. Keep observed rates descriptive: a change in conversion rate alone does not establish that a particular intervention caused it.
Reconcile the order strip
Show separate counts for placed, authorised, captured, confirmed, fulfilled and retained. Put cancelled, returned, refunded and disputed beside them. A state strip makes gaps visible: captured but unconfirmed, confirmed but unfulfilled, or refunded but not reflected in settlement.
The Consumer Contracts (Information, Cancellation and Additional Charges) Regulations 2013 support keeping payment obligation and confirmation distinct. The Consumer Rights Act 2015 sets consumer rights for goods, digital content and services; the GOV.UK announcement, “New rights for consumers when buying digital content” concerns digital-content remedies, not a general returns rule. Keep cancellation, returns, refunds and disputes separate. State the retained-order window and whether it is complete.
Add a reconciliation queue with order reference, disagreeing systems, first-seen time, current owner, customer protection applied and next action. The dashboard must not force store, payment, fulfilment and accounting totals to match by deleting unresolved cases.
Give data quality its own panel
Display refused or unavailable tracking, missing expected events, late arrivals, duplicated records, bots, staff journeys, unmatched identities and query failures. Use counts and defined shares only where the denominator exists.
Storage of or access to information on a user's device is governed by the Privacy and Electronic Communications (EC Directive) Regulations 2003 (PECR); where personal data is processed, UK GDPR and the Data Protection Act 2018 also apply. The ICO's storage and access technologies guidance sets out the permission boundary for cookies and similar operations. Permission does not prove measurement validity, and analytical usefulness does not provide permission.
Keep interpretation in labelled layers
Separate observed events, reconciled commercial states, descriptive association, platform attribution, modelled estimates and causal effects. Show the method, assumptions and uncertainty beside any derived figure. An attributed order or observed conversion change must never appear in a causal-effect field.
HM Treasury's The Magenta Book: Central Government guidance on evaluation distinguishes monitoring from attribution and explains the counterfactual. Keep dashboard labels honest: an attributed order is not a causal effect unless the evaluation design supports that claim.
Use a guardrail queue for each non-compensating gate
Create distinct queues for unclear price or optional choices, accessibility defects, product-safety concerns, privacy issues, payment mismatches and security events. Each row needs severity method, evidence, owner, status and action. Buyer-set triggers stay [blank]. A commercial panel cannot turn a safety or consumer HOLD into a pass.
End with the decision record: continue, revise, pause, withdraw or gather more evidence. Record who decided, which version they considered and what would reopen the report.
Control access and corrections
Give viewers only the data they need for their role. Separate access to identifiable journey records from access to aggregate reporting. Log definition, query, filter and permission changes so an investigator can reconstruct what a decision maker saw.
Add a correction queue with detected date, affected period, faulty source or logic, owner, customer impact, replacement version and decision consequence. Do not overwrite a reported value silently. If a late refund or dispute changes the retained-order view, show both the original cut and the restatement.
The dashboard owner reviews empty panels as carefully as high values. An absent feed, expired query or missing denominator should produce UNKNOWN or HOLD rather than a reassuring zero.
Before you act
- Define the decision, journey, channel, identity unit, period and accountable owner.
- Expose eligible population, numerator, denominator, exclusions and uncertainty for every value.
- Show separate counts for placed, authorised, captured, confirmed, fulfilled and retained orders.
- Record the retained-order window and whether it is complete.
- Label each derived figure with its method, assumptions and uncertainty.
- Log definition, query, filter and permission changes for later reconstruction.
Common questions
What should the dashboard header show?
It should show the decision, current journey, any intervention, channel, identity unit, period and timezone. It should also show the data cut, restatement date, definition version, event-schema version, query version and accountable owner, with links to source records and the change log.
How should unresolved reconciliation cases be handled?
The dashboard must not force store, payment, fulfilment and accounting totals to match by deleting unresolved cases. Instead, add a reconciliation queue with order reference, disagreeing systems, first-seen time, current owner, customer protection applied and next action.
What happens when a late refund or dispute changes the retained-order view?
Do not overwrite a reported value silently. Show both the original cut and the restatement. Record the detected date, affected period, faulty source or logic, owner, customer impact, replacement version and decision consequence in a correction queue.
