Ecommerce Conversion Rate Optimization: Do You Have The Traffic To Test?

Ecommerce conversion rate optimization has a reputation problem, and it earned it. The genre is full of listicles about button colours, and button colours have never moved a conversion rate in a way that survived being measured properly.

The reason most CRO fails is not bad ideas. It is that the maths of testing does not work at the volume most stores have, and nobody says so.

The uncomfortable arithmetic

An A/B test needs enough conversions in each variant to distinguish a real difference from noise. How many depends on your baseline rate and the size of the effect you are hoping to find, and the smaller the effect, the more you need.

Detecting a small relative improvement typically needs thousands of conversions per variant. Not sessions. Conversions.

Run the calculation for your own store before you plan a testing programme. A site with a few hundred orders a month cannot resolve a five percent improvement in any reasonable timeframe, and running the test anyway produces a number that looks like a result and is not one.

This is not an argument against improving your store. It is an argument that below a certain volume, the honest method is different.

What to do instead at low volume

Stop testing and start finding things that are broken. Broken is not a matter of opinion and does not need statistics.

Watch session recordings. Twenty sessions that reached checkout and did not complete will teach you more in an hour than a month of testing. You will see the field somebody typed into three times, the error message that appears above the fold they are not looking at, the phone layout where the button sits under the keyboard.

Complete a purchase on your own store, on a real phone, on mobile data. Not the office wifi. Buy something with a real card. Most people who own a store have never done this and are surprised by it.

Read your support tickets. Every question about shipping cost, delivery time or returns is a question your product page failed to answer, and for every person who asked there are more who left.

Check your speed on a mid-range Android over 4G. Not your laptop. Your laptop is a lie.

The things that reliably matter

Across most stores, the same handful of things account for most of the loss. None of them are subtle.

Shipping cost revealed late. The most common reason people abandon a cart is a cost they did not expect. The fix is not cheaper shipping, it is earlier shipping. Say the number on the product page.

Forced account creation. A guest checkout option, offered before the customer has invested effort, not after.

Delivery date, not delivery speed. Standard shipping means nothing. Arrives Thursday means something.

Mobile forms. Correct input types so the numeric keypad appears for a card number, address autofill that works, a viewport that does not zoom when a field is focused.

The returns policy, where the decision happens. On the product page, in a sentence, not linked in the footer.

Errors that survive. A validation failure that clears the form is a lost order. Every time.

These are not tests. They are defects. Fix them without measuring, because the cost of fixing them is low and the argument for keeping them is nonexistent.

When you do have the volume

Test big changes, not small ones. A different page structure, a removed step, a different value proposition. Big changes produce big effects, and big effects need less traffic to detect, which means you get an answer this month.

Decide the sample size and the duration before you start, and do not look early and stop when it is winning. Peeking at a running test and stopping on a good day is the single most common way to get a false result, and it is why so many teams have a folder of wins that never showed up in revenue.

Run for whole weeks. Weekday and weekend traffic behave differently.

And measure revenue per session rather than conversion rate. A change that lifts conversion by pushing everyone to the cheapest item can raise the rate and lower the takings, and if you are watching the rate you will call it a success.

The part CRO advice usually skips

Site speed is a conversion feature and it is usually a technical problem rather than a design one. So is a checkout that fails under load, a rate quote that takes four seconds because two carriers are being queried one after the other, and a stock number that was accurate ten minutes ago.

These do not show up in a heatmap. They show up as people leaving, and no amount of copy testing recovers them.

Where we fit, and where we do not

The parts of this we affect are the infrastructure ones: rate quotes fast enough to show shipping cost early, stock accurate at the moment of the sale so nobody buys something you cannot ship, and a checkout your own code controls, so fixing a mobile form is a change you make rather than a ticket with a theme vendor.

Where we are the wrong answer: we do not do testing, heatmaps or session recording, and dedicated tools for those are good and inexpensive. If your storefront is on a hosted platform and converting fine, none of this is a reason to move. And a fast backend does not rescue a confusing product page, which is the part you own either way.

The short version

Work out whether you have the traffic to test. If you do not, hunt defects instead and fix them without measuring. If you do, test large changes, decide the duration up front, and watch revenue per session.

Either way, buy something from your own store on your phone first. It is free and it is usually the most informative hour of the whole programme.