For anyone out there trying to do any form of semi-serious work on the internet (notably with websites), you often end up asking yourself: is what I’m doing having any effect whatsoever? It’s an important question — especially if there are monetary values attached to the work you’re doing — and it’s not always the easiest one to answer.
That’s where analytics packages come in handy. They can tell you who is visiting your site, where they’re from, what browser they’re using, their navigation path, search terms, etc. From a metrics perspective, it’s indispensable information. And there’s a lot of packages that’ll help you get all that.
But only one of them will get you into Google’s search index almost instantly.
Okay, let’s back up a sec. Going back a few years, one of the things I did frequently was work on search engine optimisation. I’d spend hours — days, even — writing code in specific ways so that it would be equally accessible to a search engine’s spider (such as Google) as it would to any normal human. But, most importantly, when the search engine read it, that without having human knowledge, it could categorise that information accordingly.
We regularly had to tell clients that significant changes to their sites (especially site redesigns) could take weeks, even more than a month, to show up in search engine indices. This was always a troubling point, as clients were often worried about making sure important information was indexed as soon as possible.
Today, my team launched a new website for our client, Big Rock Brewery. It was one we’d planned for a while, making sure the site was carefully built, and all the appropriate 301 redirects were in place to assist with the inevitable ranking shift due to URL changes.
Now, knowing how Google et al behaved in the past, one would think that I wouldn’t spend any time peering at Google’s results to see what had already changed, wouldn’t you? Well, maybe my old self would. My current self, however, is just too darned curious.
Imagine my shock when I saw, within less than two hours from seeing the first hits to the new site (which was on a completely different server), I saw the new URLs and even whole new pages showing up in Google’s index. Complete index, too, I might add — copy and description already in place.
“Stunned” doesn’t quite cut it. It’s almost as if the world had changed.
Naturally, being curious, I wanted to know why. So I pestered some colleagues of mine who deal with the search aspect a lot more than I do. Their thought? It’s Google Analytics.
Now, without actually peering under the hood (‘cuz even long-time Googlers don’t get that kind of privilege), I can only surmise that each hit from Google Analytics is being translated into an index request at Google’s end, which is then verified by a separate spider request (which obtains the copy and meta tags). I’m sure there’s more to it than that, but that seems pretty reasonable at a basic level.
I’d read somewhere a while back that Google chewed through something like 20 petabytes of data per day. (After some poking around, it looks like that was circa 2008, by the way.) During that time, it took a while for a simple change to appear. Now? It might as well be 20 petabytes an hour, or Google’s gone through some really significant changes in the indexing algorithms.
Frankly, I think it’s a double-edged situation. It’s a boon to those of us who plan releases, because we know it’ll get indexed pretty quickly. But it’s also a danger — one wrong move, and Google will have your mistake indexed before you know it.
