One common consideration in PPC campaigns is wondering how well an account is performing compared to how it should be expected to perform. Usually we would have to wait a long time for enough data to become available, but sometimes an account presents an opportunity to get more information than we are provided with at face value – if we are able to parameterise our set of campaigns in a sensible way then we can compare like with like.
By way of an example, a Hotel Chain owns multiple Hotels of various sizes across the UK. They all run adverts every day and a conversion is counted when someone books a room in the Hotel. We can parameterise the Hotels by which Hotel it is and which day of the week the adverts are appearing. We would expect that each Hotel would do proportionately well on any given day of the week. E.g. if Hotel A converts twice as much than Hotel B on Mondays then we could expect that it did so on Tuesdays as well, without any more information to sway our decision (maybe Hotel A is in a town with a regular Monday tourist event that would attract more business).
The key statistics we can investigate with this method are Impressions, Clicks and Conversions. It can be applied further but caution must be exercised. Suppose we want to know if Hotel A is getting as many Impressions as expected on Mondays – what do we do? Well, since all the Hotels can be differently-sized, we would like to know how all the other Hotels perform and compare Hotel A’s performance with that. We therefore take for each other Hotel the number of Impressions on Mondays divided by that Hotel’s total number of Impressions. We then average those daily Impression-shares, and can see how Hotel A’s Impression-share compares. If it is more then we are happy with it and if it is less then we may be unhappy with it.
In the example above, we see that Hotel C in London had 20,088 of its 164,931 Impressions (about 12.18%) on Sundays, but Hotel B in Leeds had 6,399 of its 46,896 (13.65%) and Hotel A in Birmingham has 17,481 of its 137,936 (12.67%), so it appears that Hotel C may not be performing as well as expected in terms of Impressions on Sundays.
This isn’t the whole story though! If we get 9,990 Impressions but were expecting 10,000 then we probably don’t need to worry too much – how do we know when we are really receiving a bad number? We need to get an indication of not only how many on average we should be expecting, but how spread around that average the data is likely to be (what Mathematicians call “Variance”).
***Mathematics Aside – We can model the aforementioned statistics as Poisson Processes (Homogeneous on each day, Non-Homogenous taken across the week) since they are independent discrete events occurring over continuous time intervals with a constant average rate. A random variable which is the sum of independent Poisson random variables is itself Poisson (with parameter equal to the sum of the individual parameters), and (Raikov’s theorem) a Poisson random variable which we know is the sum of independent random variables must be the sum of independent Poisson random variables.***
The Variance for one of these statistics is equal to the average value itself, and it is a mathematical fact that 99% of the time a result will be received which is within three times the square root of the Variance away from the average. Any result outside of that range is quite a surprise and we should be aware that this is actually a good or bad result.
So how do we use this information? Well, that is another question altogether really. The mathematics shows you when something is surprising, but often it is not because something out of the ordinary has happened. Most of the time it is the case that you simply overlooked something, or made a bad assumption. Hopefully performing this analysis will help you to get a better insight on your own accounts.
Happy optimising!
Insights
