Strategy
Part of A store conversion plan built on a defined journey and predeclared stop rules
Conversion planning template: a change log to complete before reading the result
A conversion planning template for an England store: log stock, price, promotion, outage and release changes before anyone reads an observed rate.
Two reporting periods are rarely alike. Before an England store reads an observed rate, it needs a record of what changed between them: stock, price, promotions, channel mix, fulfilment, outages and releases. Copy this blank change log for one customer-journey decision and fill it with merchant-owned evidence.
What to take away
- The observed rate is eligible completions divided by eligible starts, and it must not be labelled customer benefit or causal impact.
- Complete the change log before anyone reads the primary outcome.
- Fill every entry with merchant-owned evidence, not a supplier case study or another merchant's rate.
- Choose one claim the design permitsobserved, attributed, causal or none.
- Thresholds stay blank until the merchant has approved them for this decision.
Freeze the baseline before the period starts
The baseline fields come first, because a log means little if the rate itself is loosely defined. Fix the numerator, denominator, unit, identity rule, period and source query before the period opens. Changing them afterwards is a change to log, not a correction to hide.
Baseline definition fields
- Eligible person, session, journey or order
- Start event and completion event
- Numerator, denominator, unit, identity rule
- Reporting period, timezone, source version
- Late events, duplicates, bots, missing records
- Authorisation, capture, refund, dispute treatment
- Approved decision threshold and evidence
Illustrative worked example: an England store records 1,240 completed checkouts as the numerator and 31,000 eligible checkout starts as the denominator over 28 days, giving an observed rate of 4.0 percent. This example is not a benchmark. The merchant's own eligibility rule and source query determine the real figures.
The analyst owns the reproducible query, and the statistician approves interpretation. Neither signs the log on behalf of the other, so each field has a named owner before the first entry is made.
Complete the change log
Each row records something that could make this period unlike the last one. Leave a row blank until evidence exists, and write "none" only when someone has checked. A blank entry that is later filled is itself worth dating.
Entry / What to record
- Customer observations
- Stated preferences, kept apart from recorded behaviour: [ ]
- Trading changes
- Stock, price, promotion, channel mix and fulfilment changes: [ ]
- Technical changes
- Outage, release and supplier-change records: [ ]
- Gaps
- Missing populations, inaccessible routes, blocked optional tracking: [ ]
- Expiry
- Evidence expiry, contradiction and correction trigger: [ ]
The decision owner should reconcile this log before reading the primary outcome. A changed product range or a payment outage can make two periods unlike, while an absent customer group can make the visible denominator misleading.
Do not fill a gap with a supplier case study or another merchant's rate. An empty row that stays visibly empty is more useful to the next reviewer than a borrowed figure that looks complete.
Choose the claim the design permits
The last field in the log is the claim. Tick one of four: observed, attributed, causal or none. A before-and-after movement with no comparison supports only the first, and the wording of every summary must stay inside the tick.
HM Treasury's quality guidance for impact evaluation is primarily concerned with the issue of attribution. Monitoring a change is not enough to say the change caused the result, so keep "caused" out of the report unless the design allows it.
The experimentation specialist approves a proportionate comparison route and a counterfactual before any causal wording is used. Record the allocation unit, exposure record, primary outcome, guardrails and uncertainty method beside the tick.
Merchant and journey fields that frame the log
Every log belongs to one customer-journey decision, so name the decision first. The merchant and journey fields below tie each entry to a seller, a population and a stretch of the journey.
- Legal seller and trading identity
- [ ]
- Product or service, exclusions and safety class
- [ ]
- Customer population and observable eligibility
- [ ]
- England geography rule, channel and field dates
- [ ]
- Journey owner and decision to be made
- [ ]
The observed rate is only as clear as these fields. If the geography rule or the eligible population shifts during the period, that shift is a log entry, not a footnote.
Gates that sit outside the log
A change log records what happened. It does not approve the journey. Customer and assurance gates stay separate, each with named evidence and an owner, and no completed log substitutes for one.
Customer and assurance gates
- Seller, price, delivery and order evidence
- Optional charge and free choice evidence
- Cancellation, return, refund and support route
- Accessibility scope, defects and owner
- Product substantiation and withdrawal route
- UK GDPR and PECR purpose and consent
- Payment-state reconciliation and exception owner
- Security access, logs and incident response
Check the proposed journey against the Consumer Contracts (Information, Cancellation and Additional Charges) Regulations 2013 and the CMA's online and distance-selling guidance. A qualified consumer lawyer must decide how they apply to the specific product and contract.
Run a separate UK GDPR and PECR review for purpose, consent or exception, minimisation and deletion. Before live use, agree stop signals, rollback steps and a recovery test with an authorised decision maker.
Before you act
- Freeze numerator, denominator, unit, period and source query before the period opens.
- Complete every change-log row, or leave it visibly blank.
- Reconcile the log before anyone reads the primary outcome.
- Tick one claimobserved, attributed, causal or none.
- Keep thresholds blank until the merchant approves them.
Common questions
Why complete the change log before reading the result?
Because a changed product range, price or payment outage can make two periods unlike. If the log is filled after the rate is seen, the explanation tends to be written to fit the number. Reconcile first, then read the primary outcome.
What does the observed rate measure, and what must it not be called?
It is eligible completions divided by eligible starts. It must not be labelled customer benefit or causal impact, so the number alone does not show that a change helped customers.
Can a supplier case study fill a gap in the log?
No. Fill every entry with merchant-owned evidence. A gap left visibly empty is honest, while another merchant's rate or a supplier case study can make two unlike periods look comparable.
