Monday’s Irish Times ran an article about Web analytics by Paul Cullen. The point Paul wanted to convey was that so few people visit Irish language versions of state Websites that investing in translating the Websites is not justified. This is one of the first times I have seen Web Analytics referenced in the main stream media and I commend Paul for his efforts.
The purpose of Web analytics is to provide decision makers with visibility on how a Website is used, thereby informing the Web strategy and design and allowing the site owner make evidence based and financially justified improvements to the site.
The very basic measurements Paul endeavoured to obtain from various state departments were
· the number of page views of the English and Irish versions of the Website in 2008
· the number of people who visited the English and Irish versions of the Website in 2008. (Unique Visitors)
The metrics provided indicate that state departments have very little visibility on the usage of their Websites. Even worse than having no visibility, they believe they know what is happening, but in fact the figures they provided do not add up.
Unique Visitors
Apparently, the Department of Social and Family Affairs has the most popular website, accessed by more than 1 in 4 of the population, with 1.2 million unique users in 2008.
Excuse me? Sanity check that figure, that is not plausible. Anecdotally, it is hard to believe that 1 in 4 people in the country have access to the internet, let alone are accessing this Website.
I suspect that this figure was calculated by taking a plausible monthly figure, say 100,000 unique visitors to the site in a single month, and multiplying this figure by 12 to get a yearly equivalent. This calculation does not give the number of unique visitors for a year.
A site regularly gets return visitors, and such visitors should not be counted again and again month after month. A true yearly unique visitors number needs to come from a Web analytics tool and cannot be calculated in this manner.
Similarly, the Department of Enterprise claims to have had 754,000 unique users in 2008. Unless the department are more enterprising than I have given them credit for, I don’t think 1 in 6 of every man, woman and child in the country visited their site last year.
Hits
The Revenue Commissioners reported 18 million hits.
A hit is merely a request to the server. If a page with 4 images is loaded, for example, 5 hits will be recorded: that’s one request for the page and four requests for images. This is totally confusing and impossible to get any actionable insight from and so the term hit ceased to be considered a suitable measure of Website usage circa 1995.
Log File Analysis
The Department of Foreign Affairs reported 879,000 unique visitors. I have no benchmark available to me to judge the accuracy of this figure, conceivably the foreign affairs office could have a wider reach than the citizens of this country and this may be an accurate figure.
Viewing the code for this site does not reveal the tool they are using to measure activity on this site. Basically there are two types of Web Analytics tools, javascript based tools and log file analyser tools.
Javascript tools work by executing a piece of javascript code that stores information about a visit every time a customer loads a page in a Web browser like Internet Explorer or Firefox. If a tool of this sort was used to measure the Department of Foreign Affairs site then the snippet of javascript would be visible in the code.
So it is likely that this measurement came from a log file analyser tool. These tools log visit information on the server, every time a page is requested from the server. The problem with such tools is that non human visits to the site often inflate the unique visitors figure. Non-human visitors called bots and spiders crawl the internet performing many tasks, for example indexing pages so search engines can find them. It takes some expertise to configure log file analyser tools to filter out the non-human figures and so unique visitor figures can be inflated.
A java script based web measurement tool is preferable, especially for non-expert use since bots and spiders do not execute javascript and their activity is never sent to the analytics tool. I suspect that the Department of Foreign Affairs figure may be inflated as a result of bot and spider traffic being counted as visitors.
On Monday I will post part two of this blog, in which I will review the Defence Forces Web Analytics implementation. Further, I will conduct some root cause analysis of the problems with the state’s Web analytics solutions. I will complete the series of articles then with a third instalment outlining my resolutions to these problems next Friday.
February 9, 2009 at 3:30 pm
Good analysis Niamh.
Perhaps such reporting should be “reality adjusted” in the same way that the government’s unemployment figures are “seasonally adjusted”.
For example, government might report that 300,000 people are unemployed – though when “seasonally adjusted” the probable figure is 250,000.
Similarly, the Department of Foreign Affairs might report 879,000 unique visitors to their website – but when “reality adjusted” the actual figure is 100,000.
Probably just as accurate!
February 10, 2009 at 5:08 am
How can you tell if any of these figures are for the Irish segment? I’d imagine a large number of visits come from other locales. Maybe they did segment out Irish visits, but this isn’t clear from your post?
Rgds
Richard
Pingback: » E-government and Web Analytics – Part 2 of 3 - iQ Blog
February 10, 2009 at 5:16 pm
Hi Richard
Many thanks for you comment.
The third paragraph in the original article indicates that the author considered the results he was provided with to be limited to the population of Ireland:
“The survey showed that the Department of Social and Family Affairs has the most popular website, accessed by more than one in four of the population, followed by the Departments of Foreign Affairs, Enterprise, Trade and Employment and Agriculture.”
In my opinion, it is unlikely that the figures Paul was provided with were for visitors with Irish IP addresses only as this would require knowledge of filters and if such knowledge was present so also would knowledge of the need to be specific about the filters applied when providing statistics. Critically I would have expected to have been advised whether internal Department traffic had been filtered out, as well as any geographical filters had any been applied.
Even if Irish IP addresses only were included this would not necessarily directly correspond with Irish citizens due to Irish citizens abroad and non-Irish citizens in Ireland visiting the sites.
So I take your point that the figures will be overestimated because non-Irish visitors visit these sites. I acknowledged this point myself by limiting the assumptions I made regarding the audience of the Dept of Foreign affairs,
“I have no benchmark available to me to judge the accuracy of this figure, conceivably the foreign affairs office could have a wider reach than the citizens of this country and this may be an accurate figure.”
I have however assumed that the audience of the Dept of Social and Family Affairs and the Dept of Enterprise are predominantly Irish citizens. This allowed me make the judgement that the reported annual unique visitors to these sites were over stated. I think it is a fair assumption and conclusion as summing the monthly unique visitors to get the annual figure is an understandable mistake I regularly encounter because these figures are usually extracted on a monthly basis and put into reports or KPI sheets, which are later summed. It is however one of those mistakes you only make once.
I hope you continue to read imminent instalments Richard, also I would like to welcome any input from the State Departments mentioned to clarify whether any filters were applied when calculating their annual unique visitors figures.
Mise le meas
Niamh
February 11, 2009 at 1:23 pm
Niamh,
Thanks for picking up on my small article in The Irish Times. I have to admit I don’t know much about web analytics; but as you said, I just thought it would be interesting to see what the traffic is through various Government in both English and Irish. Unfortunately, you have to add the gaps in my technical knowledge to the gaps in the knowledge of the people in the various press offices and then even perhaps the people working on the websites. In any event, the information was provided in a variety of forms and some of its seemed unlikely. I think it would be good if someone perhap in academia were to do a more thorough follow-up and study.
Irish language enthusiasts also made the point to me that most departments do not have fully-fledged Irish versions on their sites and this serves to depress the traffic in this direction.
Paul Cullen
Pingback: » E-government and Web Analytics - Part 3 of 3 - iQ Blog