After an initial research i have come to a great discovery- Google index the web slower than before to its Panda updates! Apparently, Google still crawls at the same frequency and timing as before, but it index the pages in a slower rate…

Until a few months back, about 5-10 minutes after i have been posted a new article it was already in the Google index. However, a few weeks ago when i checked it again, i saw new posts in the Google index only after 1-2 hours and sometimes even 3 before they got indexed. At that time, I didn’t ascribed it to nothing significant and thought it’s just something temporary.

However when i examined it again a few days ago, i discovered something new: My posts still indexed after about two hours, but when they do appear in the results, it says “2 hours ago”… It means that Google still crawls them right after they being posted, but they only appear in the results (indexed) after 2 hours! What is happening during that time??

More Researches…

I didn’t want to attribute this discovery based upon solely the results from my humble website, so i decided to continue researching, this time with some of the most popular blogs. Here are my results:

Businessinsider.com – New posts indexed after 13-20 minutes (3 posts tested).

Wired.com – New posts indexed after 10-14 minutes (3 posts tested).

Gizmodo.com – New posts indexed after 40-50 minutes (2 posts tested).

Huffingtonpost.com – New posts indexed after 11-15 minutes (2 posts tested).

Techcrunch.com – New posts indexed after 11 minutes (1 post tested).

As you see, even some of the most reputable and popular content creators on the internet, which Google bots practically live in, are getting indexed only after a while. I can assume that for smaller sites it takes much longer.

My Conclusions:

The only thing i can think of is that Google is operating some sort of quality filter on the way between it crawls a page and index it. It’s well known that the Panda “identifies” somehow the quality of pages and i am guessing that after a page is crawled it goes through the “Panda Filter” to examine it- Texts, links, images, videos and anything else on the page. This process apparently takes time, even for a giant resources company like Google.

As for the future, i don’t think it would last much longer like this and soon enough Google will accelerate this process- We are just at Panda 2.3 and it looks like we are going to have much more updates ahead of us…