Back to results Interactive bytes : The problem with numbers By Alastair Thompson How can the monthly audience of Trade Me be bigger than the population of NZ? Why do my click stats tell me I have 1000 clicks but my analytics record only 500 arriving? Is my CTR good or bad? This website claims one million hits a week – is that a lot? Experienced online media planners well know that the most metricated medium around is oftentimes also the most confusing. Numbers are used by everybody all the time but what do they actually mean? In time perhaps some clear standards will emerge which make things easier for planners and publishers, but in the meantime the best advice – especially to new entrants to the mysteries of online advertising – would be to be careful about what you read and to be especially careful not to compare apples and oranges. To use online numbers well, it helps to understand what is being measured and how. Problem is that is hard. The devil is always in the detail. Two examples: The limitations of time measurement metrics Two key metrics measured by Google Analytics and Nielsen NetRatings (arguably the two most useful metrics technologies as they are widely used and can therefore compare apples and apples) are average page duration (aka APD) and average session duration (aka ASD). Typically these measures are seen as a proxy for engagement. NNR goes further and calculates what it calls ‘total time spent’ by multiplying APD by the number of pages served. The problem with APD is that browsers do not notify servers when a page is closed. The time spent on the final page within any given session is not recorded. When a website has a high bounce rate (and thanks to search-based browsing, many sites have rates of 60% or more) the page which has the longest time spent on it is not recorded. So for short session websites, all that is measured is the time spent on the navigation pages – not on the actual content. Cookie tracking – how many readers did my ad see? Sophisticated conversion tracking relies on ads placing cookies on readers’ computers. Later when the person buys something a reporting suite will try to identify which ads the user viewed on their path to purchase, and where. The thing is, though, the user didn’t necessarily see the ad. Rather the ad saw the user. A consequence of this method of measurement is that very high conversion rates claimed by some advertising providers are by some measures pretty meaningless. Ad-network claims 75% of people purchasing X saw an ad for X via its network before they purchased. But if you dig a bit deeper you find that 75% of internet users are reached by their network. However not all reach is equal – often network ads appear down page or even in the footer of pages. Network ads are by definition not premium placements. But even if the user hasn’t seen the ad, the ad has seen the reader and dropped a cookie on it, and later when post conversion analysis is done remarkable statistics pop out. The most common industry response to the numerous issues posed by metrication confusion is to take a performance measure – usually the click-through rate (CTR) – and use it as a proxy for everything else. However this also has its limitations as recent research indicates that different demographics have different levels of inclination to click. But in the absence of other reliable methods to compare value with value, the CTR is probably the best of a bad lot. Ultimately the key lessons to take from the problem with numbers are: Compare apples with apples: Beware of ‘custom’ statistics provided by advertising sales people. Industrywide collected metrics are best. While they may be flawed, at least that way you are comparing apples with apples. The devil is in the detail: When delving into the detail of all numbers, be careful, think and seek advice from the knowledgeable – if something stands out then ask; there may be a sensible explanation. Intuition still works: Sometimes it makes sense to also rely a little on intuition. Is this the right environment for my advertiser contextually? If it is, then it is probably worth a trial. Environment is not well measured by numbers. Try before you buy: Run test campaigns and see how they perform. But when doing so, still compare apples with apples by measuring sites with similar demographic profiles against each other. InterActive Bytes is compiled for AdMedia by Scoop.co.nz. Send feedback to co-editor Alastair Thompson (alastair@scoop.co.nz). © Copyright AdMedia magazine October 2009 All material appearing is copyright and cannot be reproduced without prior permission of the publisher. Please contact the copyright officer: Ph 0-9-845 5114, Email copyright@mediaweb.co.nz.	Username Password Keep me logged in on this computer Forgotten your password? Enter your email address and we will email your password to you:

Interactive bytes : The problem with numbers

By Alastair Thompson

How can the monthly audience of Trade Me be bigger than the population of NZ? Why do my click stats tell me I have 1000 clicks but my analytics record only 500 arriving? Is my CTR good or bad? This website claims one million hits a week – is that a lot?
Experienced online media planners well know that the most metricated medium around is oftentimes also the most confusing. Numbers are used by everybody all the time but what do they actually mean?
In time perhaps some clear standards will emerge which make things easier for planners and publishers, but in the meantime the best advice – especially to new entrants to the mysteries of online advertising – would be to be careful about what you read and to be especially careful not to compare apples and oranges.
To use online numbers well, it helps to understand what is being measured and how. Problem is that is hard. The devil is always in the detail. Two examples:

The limitations of time measurement metrics
Two key metrics measured by Google Analytics and Nielsen NetRatings (arguably the two most useful metrics technologies as they are widely used and can therefore compare apples and apples) are average page duration (aka APD) and average session duration (aka ASD). Typically these measures are seen as a proxy for engagement. NNR goes further and calculates what it calls ‘total time spent’ by multiplying APD by the number of pages served.
The problem with APD is that browsers do not notify servers when a page is closed. The time spent on the final page within any given session is not recorded. When a website has a high bounce rate (and thanks to search-based browsing, many sites have rates of 60% or more) the page which has the longest time spent on it is not recorded. So for short session websites, all that is measured is the time spent on the navigation pages – not on the actual content.

Cookie tracking – how many readers did my ad see?
Sophisticated conversion tracking relies on ads placing cookies on readers’ computers. Later when the person buys something a reporting suite will try to identify which ads the user viewed on their path to purchase, and where.
The thing is, though, the user didn’t necessarily see the ad. Rather the ad saw the user.
A consequence of this method of measurement is that very high conversion rates claimed by some advertising providers are by some measures pretty meaningless. Ad-network claims 75% of people purchasing X saw an ad for X via its network before they purchased. But if you dig a bit deeper you find that 75% of internet users are reached by their network.
However not all reach is equal – often network ads appear down page or even in the footer of pages. Network ads are by definition not premium placements. But even if the user hasn’t seen the ad, the ad has seen the reader and dropped a cookie on it, and later when post conversion analysis is done remarkable statistics pop out.
The most common industry response to the numerous issues posed by metrication confusion is to take a performance measure – usually the click-through rate (CTR) – and use it as a proxy for everything else.
However this also has its limitations as recent research indicates that different demographics have different levels of inclination to click. But in the absence of other reliable methods to compare value with value, the CTR is probably the best of a bad lot.
Ultimately the key lessons to take from the problem with numbers are:
Compare apples with apples: Beware of ‘custom’ statistics provided by advertising sales people. Industrywide collected metrics are best. While they may be flawed, at least that way you are comparing apples with apples.
The devil is in the detail: When delving into the detail of all numbers, be careful, think and seek advice from the knowledgeable – if something stands out then ask; there may be a sensible explanation.
Intuition still works: Sometimes it makes sense to also rely a little on intuition. Is this the right environment for my advertiser contextually? If it is, then it is probably worth a trial. Environment is not well measured by numbers.
Try before you buy: Run test campaigns and see how they perform. But when doing so, still compare apples with apples by measuring sites with similar demographic profiles against each other.

InterActive Bytes is compiled for AdMedia by Scoop.co.nz. Send feedback to co-editor Alastair Thompson (alastair@scoop.co.nz).

© Copyright AdMedia magazine October 2009

All material appearing is copyright and cannot be reproduced without prior permission of the publisher.

Please contact the copyright officer: Ph 0-9-845 5114, Email copyright@mediaweb.co.nz.

Username
Password
Keep me logged in on this computer

Forgotten your password?

Search text

Results per page

Order by

Publications to search

Fields to search

Date range

Interactive bytes : The problem with numbers