Testing Traps – How to Get the Most from Your Testing

- 9 Tips for Better Business Decisions

online testing

Too many bad decisions are being made as the result of testing.

The proliferation of online testing tools have made it easy for anyone to test different elements of their websites and marketing campaigns.

However that doesn't mean most of them know what they are doing.

As a result plenty of bad decisions are being made that are costing businesses lost revenue and leads.

While tools like Google Experiments, Optimizely, Monetate and many others bring testing to the masses, in the hands of amateurs these tools can be downright dangerous.

Here are nine tips and tricks to make you a better tester:

1: Don't DIY

Testing is as much art as it is science and you need professional advice to get the most from your testing efforts. You need both people who understand the numbers, and experts who know when things smell a funny colour.

That is, people who know what to look for and know if the statistical results don't conform to previous tests and other industry research.

You may not need them to do all your testing, but at least get them to give a professional opinion of your results.

2: Too Small a Sample Size

Offline market researchers and pollsters often say that 200 is a good sample size, but online I wouldn't be happy with anything less than a thousand, and even that may be too small a sample.

If we take the average conversion rate of 2%, then a sample size of 1,000 is only going to give you 20 final conversions.

Personally I wouldn't base any decision on only 20 results.

Ten thousand would be a better starting sample size to give you 200 conversions, but for a small business site that number may simply be out of the question.

This brings up the issue of whether or not a small business site should even be testing if it can't get a big enough sample size.

3: Understand that Google Experiments is a Blunt Tool

Google Experiments is probably the testing tool that most online business owners and webmasters are familiar with. However it is a very blunt and flawed tool. In no way should you use it to make final business decisions.

It's useful for an overall view, or to get started, but if you want detailed information on your test results you will need to at least develop custom tags and variables within Google Analytics.

In other words cross check your results with another measurement tool. To really crunch the numbers, you should probably export the results to a spreadsheet and run your own calculations and charts.

In fact, whatever tool you use, you should make sure you understand its limitations.

4: Know What You Are Really Testing

This may sound obvious, but quite often people aren't really testing what they think they are testing. This is again where you need professional advice.

In many initial testing cases the results are confusing and inconclusive. In which case you will need to conduct further tests to find out what is really going on.

For instance if your site is a little slow to load, then adding a video and testing its effectiveness will be problematic. The video may have no effect on the results simply because people can;t be bothered waiting for it to load.

If however, you have a fast site, then a video may have a dramatic effect on you conversion rates.

So what were you really testing? The speed of the site or the effect of video? You can't tell, and you will need to conduct further tests to find out.

5: Know What to Measure

For many people, conversion rate is the ultimate measure; however it is not the only thing that needs to be taken into account. There are several other key metrics that all need to be taken into consideration together with the conversion rate.

    • Bounce Rate: How many people came to the page and left the site from that page. This indicates that the page is not giving them what they want, or that there is some other problem preventing a user from engaging with you.

    • Time on Page: Related to the above, how much time a prospective customer spends on your page or site could be a measure of how engaged they are or how hard they are having to work to find what they want. This metric especially needs to be used in conjunction with the others listed here.

    • Pages Viewed: Again, related to the above and for the same reasons. It could mean they are highly engaged or having difficulty finding what they want.

    • Pages Shared: If they share it, then presumably they like it and were engaged enough to take this action.

  • Average Order Value: While the conversion rate for one variation may be higher, the other option might have a higher order value and therefore be making you more money overall.

There are many other metrics you can use, and again the key is to use a combination of measures to give you a complete picture.

6: Understand the Subtleties and Tone

As I've said in numerous other posts, the types of words, images and layout you use can have tremendous impact on your conversion rate. Simply changing the order of words or colour of a button can have significant effect.

For example if you are testing whether video works at improving your results, you need to take into account the tone, style and quality of the video. EG: Does a professionally made video work better than an amateur one?

Just adding a video to a page isn't enough.

The same goes for testing long or short copy. Just because it's short or long isn't enough, you need to test well written, professional copywriting against supplier supplied copy, your development agency's copy and your own writing efforts.

You can't just test long vs short copy. If neither of them were prepared by an experienced copy-writer, then your test results are meaningless.

The same goes for design elements, if they weren't done by a professional designer.

7: Test Quickly and Often

As mentioned above, one test is rarely enough. To get reliable results you will have to refine and retest your hypotheses several times before you get a meaningful result. So don't think testing is a once only solution.

8: Test in the Real World

Real world testing always shows up more unforeseen results than can ever be discovered from focus groups, lab testing and limited beta testing groups.

While there's always the possibility you can end up with egg on your face, you will learn valuable lessons. The disastrous meltdown of Click Frenzy is the classic example. Hugely embarrassing for the owners, but they learned heaps from it.

9: Test Early

All too often testing is done at the end of a process, when things have already been built, resources committed and budgets spent.

If testing shows up problems at this stage there is often resistance to making any changes as people have already invested time and effort to get to this stage.

Testing often makes people feel invalidated, and they take it personally and will try and defend their original decisions.

It is always better to test early and discover any problems before too much has been invested in any production.

Need Content marketing and copywriting that gets results.

We've helped people sell everything from gold to live fish – fluffy slippers to booty boosting underwear.

If you want tangible, bankable results from your copy or content, give us a call - 0411 353 693.


How to Write Effective About Pages

Almost everybody makes the same basic mistake when writing About pages, bios or social media profiles - It’s Not About You. Learn how to create an About page that gets leads and sales.


How To Talk About Yourself Successfully

Tips to Successfully Promote Yourself Some practical tips on how to write about yourself without spoiling your pants. This post will show how to avoid “We we” and its confederates Aye aye, Mimi, Hehe and Chi chi as well as the dreaded third person affectation.


Why Being the Best is Really Bad for Your Business

One of the quickest ways to make your prospective customers throw up and leave is to say you are: “Australia’s leading expert in...” Whether you’re in real estate, marketing, training, or social media: Make the above statement and you lose all credibility with customers.


Profile Building

We all have an online profile, but are you really in control of yours? Digital media is now the dominant form of media in Australia. It is crucial for you to take responsibility for your online profile


To Market Online You Must Market Offline

Traditional offline marketing is one of the most effective ways of driving qualified customers to your online business. However, it is often totally overlooked by many in the ecommece and online marketing arena.


Social Media Conversion without a Word of English

This viral YouTube sensation shows how to communicate and engage with an audience without using English. Interestingly it also helps demonstrates some of the issues facing business and marketing today.


"Added value beyond my expectations," George McCullough, Snowden Mining, Perth, Western Australia.
"One of the pioneers of e-commerce and internet marketing in Western Australia," Jim Poignand, Vista Energy, Perth, Western Australia.
"I speak to them, I get a result," Wayne MacKenzie Brown, Australian Coral Farms, Karratha and Perth, Western Australia.

Contact us

  • Perth 6000
  • Western Australia
  • 0411 353 693
  • contact@hotly.com.au