But are you sure you have the right winner? Which metric should you use?
Before starting we will have to sure we have enough data to declare a winner. So I will go to www.splittestcalculator.com.
This is a A/B test calculator.
Really awesome if you have a split test with 2 ads.
I have made this example to show you how to declare a winner of the split test. I have 5 ads competing against each other.
When creating an ad – you are also creating a keyword. When somebody searches for your keyword, your ad can be displayed.
So you will have to have the right keywords. With no impression you will have no display and therefor no sale.
So your measurement should always start with the impressions.
In my example here the ads all got 10.810 impressions.
Clicks and cost
Next measurement is the clicks. How many people are actually clicking on your ads? In my example ad 1 got 536 clicks, but ad 4 is leading with 1063 clicks.
You are only paying when somebody is clicking on your ad – so you will have to check out your costs too. Ad 2 is the ad with the lowest cost (182 dollars).
With cost and clicks you have now the possibility to calculate the CTR (Click Through Rate) and the CV (Conversion Rate).
But now you will have a problem. If you are using the Conversion Rate what about the impressions, the click through rate or the average sale amount?
What is the best scenario – to have a high CTR and a low CR? Or a low CR and a high CTR? Or maybe a mid-range CTR and a mid-range CR? You see the problem? Which ad is the winner?
The ad with the highest CTR is ad 4 with 9.83 percent. So maybe ad 4 is the winner. The ad with the highest CV is ad 2 with 10.95 percent. Well ad 2 might be the winner too.
Conversions – lets get some sale
We are not really interested in impressions or clicks right?
I would rather get some sale – so we have to find out how many sales or conversions we get from our ads.
In my example ad 4 get 62 sale and ad 3 is only getting 38 sales. So now we can calculate the Cost Per Acquisition. The winner in my example is ad 2 – because we want the ad with the lowest cost.
So maybe ad 2 is the winner?
When using the cost per acquisition you don’t take into account the volume (that’s the impressions and clicks) or your average sale amount.
We will have to use a metric that incorporates your CTR, your costs, your conversions and the average sales amounts into one metric.
So I will like to introduce you to Profit Per Impression (PPI).
Profit Per Impression (PPI) – a much better metric
To use the PPI you will need to calculate your revenue and your profit.
Revenue can be a little difficult to calculate if you are selling a lot of different products. My revenue is highest at ad 5 with 784 dollars.
Profit is your revenue – cost. In my example I get the highest profit using ad 1 with 438 dollars.
Now you can calculate your PPI. You just divide your profit with your impressions.
With ad 1 I get the highest profit per impression with 0.041 dollars. So ad 1 is my winner.
Well what about the Quality Score?
We just found out that ad 1 is the winner – but you will have to think of the Quality Score too (if you are working with Google Adwords).
“Quality Score is an estimate of how relevant your ads, keywords, and landing page are to a person seeing your ad. Having a high Quality Score means that our systems think your ad, keyword, and landing page are all relevant and useful to someone looking at your ad.
Having a low Quality Score, on the other hand, means that your ads, keywords, and landing page probably aren’t as relevant and useful to someone looking at your ad” (taken from Google Adwords).
If you pick an ad with an overall low CTR – it will hurt your Quality Score. A lower quality score will give you a lower average position of your ad, and a higher CPC.
So if you have 2 ads with almost the same PPI, you should pick the one with the highest CTR.
Well I will still pick ad number 1 in my example here. If you are in doubt you can make a test between the 2 last ads.
Always do a split test with your ads and be sure you use the right metrics.
First of all be sure you have enough data. Without enough data you might pick the wrong winner.
Next time go and use the PPI metric to get your winner.
Go and grab my free Complete SeoCustomer Guides 2011 and 2012 Q1 – full of tips and tricks about SEO, SEM, Social Media Traffic and Link Building
Have a great day out there…