Comments on: Analytics Apples And Oranges: Switching Web Analytics Tools Without Getting Fired

By: Ian

Ian — Thu, 31 Jul 2008 18:44:04 +0000

@Hitendra I need to know if WebTrends is using log file tracking or log file tracking plus javascript tracking…

By: Hitendra Molleti

Hitendra Molleti — Wed, 30 Jul 2008 09:33:03 +0000

Hi,
We are looking for a good analytics software for our business portal.
We were looking at using Urchin 6 as we did not want to go for Web Trends which is very pricy, to serve a group of sites with 50M impressions.
The problem here is when the access logs for one month for a site were fed into Default config of Web trends and Urchin, there was a huge difference in the visits. Urchin was almost 25% less than what Web trends suggested.
The page views were around 7% less.
Can someone guide us as to how would this happen, also suggest if Web trends is really better than Urchin.
Thanks
Hitendra

By: John

John — Wed, 09 Jul 2008 18:26:58 +0000

This is really important stuff. I work for a newspaper company and we went through the same thing last year when we switched from Urchin 5 (no UTM) to Google Analytics. The drop was even more pronounced than what you’ve laid out here.
As you can imagine, explaining to managers (not to mention advertisers) why our numbers were suddenly slashed was a challenge.
Hopefully, one day a standard is established, because apparently virtually no one’s numbers are 100% accurate.

By: Ian

Ian — Wed, 09 Jul 2008 13:52:08 +0000

@Aaron good point – I did a terrible job describing the client’s old WebTrends installation. It VERY old. See above for edits.

By: Dash Lavine

Dash Lavine — Wed, 09 Jul 2008 12:03:59 +0000

I am glad Chris points out that this isn’t a tool based issue per se, but rather a difference due to a tracking method change. As Chris briefly touches upon, WebTrends has the ability to track both server logs as well as logs generated via JavaScript tags. The SmartSource Data Collector is provided to software clients for collecting their own JavaScript generated data and the SAAS WebTrends OnDemand clients all utilize the JavaScript tagging methodology via requests to the WebTrends global data collection centers.

By: Aaron Gray

Aaron Gray — Wed, 09 Jul 2008 08:59:47 +0000

Ian,
You do a nice job here of highlighting some of the challenges you’ll face when moving from one analytics solution to another.
I did want to point a few things out, though.
Firstly, WebTrends also has a JavaScript (tag) data collection solution, called the SmartSource Data Collector. Of course, we still support log-file analysis, too. We also support multiple sessionization methods (ranging from IP address matching to 1st and 3rd party cookies, and even authenticated user sessionizing), the configuration of which can impact visit and visitor numbers. We’re pretty flexible all around.
Secondly, if you are analyzing log-files instead of using the tag, you can filter out file-types that you don’t want included as page-views.
And lastly, it isn’t entirely true that JavaScript tagged pages are only collected as pageviews once the page fully loads. Where the tag is placed in the page code can have an impact on when the page view is recorded (i.e. when it starts to load or when it finishes loading). If you’re placing your tag at the bottom of the page, the scenario you outline is true as the tag will be the last bit of code parsed.
This said, there are good arguements for placing the tag at the top so that you collect the entirety of actual user behavior on the site. Especially on slow loading sites, it’s possible that the visitor finds what they’re looking for before the page loads and clicks out. You probably want to collect that page view, as it represents real user behavior on your site. And, your path analysis reports will look funny without it.
As I mentioned, nice post. You made it nice and simple to understand. Keep it up.
Best,
Aaron Gray
WebTrends

By: Chris

Chris — Tue, 08 Jul 2008 19:39:24 +0000

Beautiful lively post. I’ll be referring a lot of people to it because it answers THE QUESTION so well.
I kinda wish you’d made it a bit more obvious(although you are explicit) that the important distinction is the type of raw data and not the brand name of the tool. As somebody who switched from WebTrends-Log-Analyzer to WebTrends-Tagged-Page-Analyzer a few years ago, I can say it’s truly a night and day difference … and the Tagged Page Analyzer’s reports are almost perfectly aligned with Google Analytics’ results.
I didn’t nearly get fired, but I nearly tore the head off the WebTrends tech support person … until he patiently got it into my newbie skull that I was in a whole new world.

By: Ian

Ian — Tue, 08 Jul 2008 19:35:54 +0000

@rexolio Oh, man, don’t even want to go there. We’d have to get Omniture and their ilk to turn over their algorithms. I think we’ll get Google to do it first…

By: rexolio

rexolio — Tue, 08 Jul 2008 17:07:27 +0000

Great post, Ian. I definitely learned the pains of trying to explain why traffic “dropped” when switching from a log file analytics solution to a javascript solution.
Now how about the difference (or drop) when comparing 2 different javascript solutions? :)