Rules and ethics
How to tell a real store conversion problem from ordinary weekly variation
How to separate real store conversion problems from ordinary weekly variation by fixing your order state and eligible population, then checking UK trading patterns.
What to take away
- A one-week drop in orders is ordinary variation until it exceeds the spread your own quiet weeks already show.
- Choose one order state, such as paid orders, and use it for every comparison you keep.
- Choose one eligible population, and remove sessions that could never have bought.
- Genuine problems repeat across weeks, channels and devices, and survive a clean rebuild of the data.
- When a symptom repeats, investigate it with a fixed diagnostic rather than a redesign guess.
Choose the order state before you compare anything
Analytics platforms count several different things and label them similarly. A completed checkout, an authorised payment, a paid order, and a paid order that was not later refunded are four separate numbers.
Pick one definition, write it down, and apply it to every chart you keep. Otherwise a change in refund handling will look like a change in demand.
Order state rules
- Count paid orders, not payment attempts
- Remove cancelled and refunded orders
- Hold fraud-review orders until they clear
- Keep subscription renewals separate
- Report net revenue beside order count
Choose the order state
- Count paid orders rather than payment attempts.
- Remove cancelled and refunded orders from the numerator.
- Hold fraud-review orders aside until they clear.
- Keep subscription renewals in a separate series.
- Report net revenue beside order count, not instead of it.
Define the eligible population
Conversion rate is a fraction. The denominator should describe sessions that had a fair chance to buy. The practical detail is set out in 4 UK shopping peaks that set.
| Traffic segment | Include in eligible population? | Reason |
|---|---|---|
| Returning logged-in customers | Yes | They can complete checkout. |
| Staff and agency IP ranges | No | Internal test activity skews rates. |
| Sessions landing on out-of-stock pages | No | No purchase was available. |
| Known bot traffic | No | Non-human sessions never purchase. |
| Sessions during a gateway outage | No | Checkout was unavailable at the time. |
A rate built from every session on the site will fall whenever a discontinued product page gets a burst of traffic. Cleaning the denominator often removes a false alarm before you touch the design.
Signals that outlive ordinary noise
Conversion rate optimisation depends on comparing a measured rate with a controlled baseline, so an unfiltered week rarely proves anything on its own.
Symptoms worth investigating usually share three traits. They persist for two or more weeks, they appear in more than one channel, and they survive a second data pull using the same order state and population rules.
A fall confined to one ad campaign is usually a media problem, not a store problem. A fall across paid search, email and direct traffic points at the store itself.
Example: a dip that turned out to be normal
Paid orders fell from 96 to 78 in one week, then returned to 101, 94 and 99.
That range sat inside the store's own eight-week spread, so the pattern was noise. Paid orders of this size move by double-digit percentages between ordinary weeks. The shape of the series mattered more than the single number.
Where UK trading patterns add noise
UK retail carries calendar effects. Black Friday, the January sales, bank holidays and payday weekends all shift traffic and buying intent within days. The ONS retail industry statistics hub publishes monthly retail sales trends that put a single week into context.
A conversion fall in the first week of a sale is often a mix shift, because discount hunters behave differently from full-price buyers. Compare similar weeks with similar promotions.
Checkout friction and legal duties worth ruling out
Shorter forms reduce friction. The ICO UK GDPR guidance hub sets the limits on what you may collect, how long you keep it and what you tell shoppers.
Failed payments, expired card rules and mandatory pre-contract disclosures can all cut completion without any design change. Check gateway logs and your checkout copy before rewriting whole pages.
From symptoms to a diagnosis
When a symptom survives the order-state check, the population check and a second week, treat it as a real case. The six store symptoms diagnostic lays out six unranked ecommerce demand signals, each with current UK controls and explicit measurement limits.
If you would rather have the numbers reviewed with you, our ecommerce conversion optimisation service opens with the same order-state and population audit.
Common questions
How many weeks of data do I need?
Two to three comparable weeks, plus the same weeks a year earlier if your trading is seasonal. If your weekly order count is small, the natural range is wider and the wait is longer.
Can a UK seasonal event cause a false alarm?
Yes. Sale weeks, bank holidays and Black Friday change the mix of visitors. A fall measured against a sale week is not evidence on its own.
What if only mobile conversion has fallen?
Check the release timeline, form errors and payment step first. A device-specific fall appearing after a deployment is more likely to be a bug than a demand shift.
Should I log changes while I watch?
Yes. Keep a short record of deployments, price edits, stock changes and campaign launches. Without it you cannot separate your own actions from ordinary variation.
