Testing Traps – How to Get the Most from Your Testing
- 9 Tips for Better Business Decisions
Too many bad decisions are being made as the result of testing.
The proliferation of online testing tools have made it easy for anyone to test different elements of their websites and marketing campaigns.
However that doesn't mean most of them know what they are doing.
As a result plenty of bad decisions are being made that are costing businesses lost revenue and leads.
Here are nine tips and tricks to make you a better tester:
Testing is as much art as it is science and you need professional advice to get the most from your testing efforts. You need both people who understand the numbers, and experts who know when things smell a funny colour.
That is, people who know what to look for and know if the statistical results don't conform to previous tests and other industry research.
You may not need them to do all your testing, but at least get them to give a professional opinion of your results.
Offline market researchers and pollsters often say that 200 is a good sample size, but online I wouldn't be happy with anything less than a thousand, and even that may be too small a sample.
If we take the average conversion rate of 2%, then a sample size of 1,000 is only going to give you 20 final conversions.
Personally I wouldn't base any decision on only 20 results.
Ten thousand would be a better starting sample size to give you 200 conversions, but for a small business site that number may simply be out of the question.
This brings up the issue of whether or not a small business site should even be testing if it can't get a big enough sample size.
Google Experiments is probably the testing tool that most online business owners and webmasters are familiar with. However it is a very blunt and flawed tool. In no way should you use it to make final business decisions.
It's useful for an overall view, or to get started, but if you want detailed information on your test results you will need to at least develop custom tags and variables within Google Analytics.
In other words cross check your results with another measurement tool. To really crunch the numbers, you should probably export the results to a spreadsheet and run your own calculations and charts.
In fact, whatever tool you use, you should make sure you understand its limitations.
This may sound obvious, but quite often people aren't really testing what they think they are testing. This is again where you need professional advice.
In many initial testing cases the results are confusing and inconclusive. In which case you will need to conduct further tests to find out what is really going on.
For instance if your site is a little slow to load, then adding a video and testing its effectiveness will be problematic. The video may have no effect on the results simply because people can;t be bothered waiting for it to load.
If however, you have a fast site, then a video may have a dramatic effect on you conversion rates.
So what were you really testing? The speed of the site or the effect of video? You can't tell, and you will need to conduct further tests to find out.
For many people, conversion rate is the ultimate measure; however it is not the only thing that needs to be taken into account. There are several other key metrics that all need to be taken into consideration together with the conversion rate.
- Bounce Rate: How many people came to the page and left the site from that page. This indicates that the page is not giving them what they want, or that there is some other problem preventing a user from engaging with you.
- Time on Page: Related to the above, how much time a prospective customer spends on your page or site could be a measure of how engaged they are or how hard they are having to work to find what they want. This metric especially needs to be used in conjunction with the others listed here.
- Pages Viewed: Again, related to the above and for the same reasons. It could mean they are highly engaged or having difficulty finding what they want.
- Pages Shared: If they share it, then presumably they like it and were engaged enough to take this action.
- Average Order Value: While the conversion rate for one variation may be higher, the other option might have a higher order value and therefore be making you more money overall.
There are many other metrics you can use, and again the key is to use a combination of measures to give you a complete picture.
As I've said in numerous other posts, the types of words, images and layout you use can have tremendous impact on your conversion rate. Simply changing the order of words or colour of a button can have significant effect.
For example if you are testing whether video works at improving your results, you need to take into account the tone, style and quality of the video. EG: Does a professionally made video work better than an amateur one?
Just adding a video to a page isn't enough.
The same goes for testing long or short copy. Just because it's short or long isn't enough, you need to test well written, professional copywriting against supplier supplied copy, your development agency's copy and your own writing efforts.
You can't just test long vs short copy. If neither of them were prepared by an experienced copy-writer, then your test results are meaningless.
The same goes for design elements, if they weren't done by a professional designer.
As mentioned above, one test is rarely enough. To get reliable results you will have to refine and retest your hypotheses several times before you get a meaningful result. So don't think testing is a once only solution.
Real world testing always shows up more unforeseen results than can ever be discovered from focus groups, lab testing and limited beta testing groups.
While there's always the possibility you can end up with egg on your face, you will learn valuable lessons. The disastrous meltdown of Click Frenzy is the classic example. Hugely embarrassing for the owners, but they learned heaps from it.
All too often testing is done at the end of a process, when things have already been built, resources committed and budgets spent.
If testing shows up problems at this stage there is often resistance to making any changes as people have already invested time and effort to get to this stage.
Testing often makes people feel invalidated, and they take it personally and will try and defend their original decisions.
It is always better to test early and discover any problems before too much has been invested in any production.
Need Content marketing and copywriting that gets results.
If you want tangible, bankable results from your copy or content, give us a call - 0411 353 693.