Ecommerce conversion software selection: the acceptance test

What to take away

Test one candidate against one job and one journey state, with one named decision owner.
Predeclare the cases and the expected transitions before the trial starts.
A tidy dashboard is not acceptance evidence.
Rehearse export, restore, deletion, supplier outage and rollback during the test, not after signing.
End with one of four verdictsadmit, request evidence, retain the manual route or hold.

Pick one job and one journey state

An acceptance test checks a single claim. Take an England store selling homeware online, with the payment step as the journey state and the head of ecommerce as decision owner. The test asks whether an analytics add-on gives reliable payment-step visibility.

Keep the question that narrow. A tool that records checkout events should not quietly become the system for personalisation, accessibility assurance or payment decisions, and a wider claim needs its own test.

Candidates in this area include Google Analytics 4 for event data, Hotjar or Contentsquare for session evidence, and Optimizely or VWO for split tests. A hosted Shopify checkout exposes fewer editable steps than a self-hosted Adobe Commerce build, which changes what can be tested.

Predeclare the cases

Write the cases down before any data arrives. Otherwise the trial drifts toward whatever the tool happens to show well. The five steps below turn the question into a record that someone else can check.

Bounded acceptance test

Predeclare 200 eligible orders across 14 days (illustrative figures only).
List the expected transitionsbasket to payment, then authorisation and order confirmation.
Name the access roles allowed to view or export each record.
Rehearse export, restore, deletion, supplier outage and rollback.
Record the verdict as admit, request evidence, retain the manual route or hold.

The manual route is a candidate in the same test. Record how an authorised person reconciles evidence, investigates a mismatch and allocates a test on that route. Do not invent its cost or accuracy.

Check what the test touches

Before any live data flows, map cookies, scripts, device access and customer records. A trial that reads the payment step may also read more than the question needs.

The ICO's guidance explains how PECR and data protection law apply to technologies that store information, or access information stored on a device. An analytics category is not permission, so the map comes before the switch-on.

Ask the supplier for governance, incident, network, data and independent assurance evidence. The NCSC publishes questions to ask your suppliers for this. A certificate needs an issuer, subject, scope, date and exclusions, and it is not approval.

Test the configured journey with expert- and user-informed accessibility checks across product pages, consent, payment, errors and support.

Rehearse the exit during the test

The exit is part of the evidence. A candidate that records payment steps well but cannot return its data has not passed. Run the rehearsal on the trial configuration while the store can still walk away.

Work through export and restore, then deletion and a supplier outage, and finish with a rollback to the last approved state. Each step needs a witness and a note of what was actually recovered.

Where a black-box score replaces inspectable evidence, log "score not reproducible", attach the vendor's model note and set a review date. A score nobody can reproduce cannot carry an admit verdict on its own.

Record the verdict and the blocks

The head of ecommerce signs the record and the finance lead confirms the exit cost. The verdict is one of four: admit, request evidence, retain the manual route or hold. Nothing else goes in the box.

Mark a gap unknown rather than assuming it away, and do not treat a sales demonstration as proof of the contracted service. Admission evidence for the supplier's identity and edition belongs in the candidate dossier and in supplier vetting.

Any candidate with unknown data roles, untested accessibility on the configured journey, an unsafe customer effect or a failed exit rehearsal is held, whatever the claimed capability.

Before signing, work through the supplier contract questions on data return, notice periods and service credits.

Before you act

Name the job, the journey state and the decision owner.
Predeclare cases and expected transitions before the trial.
Map cookies, scripts, device access and customer records.
Rehearse export, restore, deletion, outage and rollback.
Record one of the four verdicts and have it signed.

Common questions

What does an acceptance test need before it starts?

One job, one journey state and a named decision owner, plus predeclared cases, expected transitions and the access roles allowed to view or export each record.

Why is a tidy dashboard not enough?

A dashboard shows what the tool displays, not whether it matches the real payment states. Acceptance needs predeclared cases checked against source records.

What holds a candidate back?

Unknown data roles, untested accessibility on the configured journey, an unsafe customer effect or a failed exit rehearsal. The verdict is then hold, whatever the claimed capability.

Can the manual route win?

Yes. It is a candidate with its own documented cost and accuracy, and retaining it is one of the four verdicts.