Google Caches IP Addresses Nightmare

Posted on November 3rd, 2005
by Lisa Stewart: : Better Rankings By Blogging

Made a discovery this week . I have been having alot of trouble with one of my clients sites- despite a steady link campaign and great content it has not been perfroming as well as it should. I went to my SEO check list over and over. No duplicate content, unique meta data for all the pages, etc. Despite everything pages are being dropped from googles index and rankings have dropped. Ruud came up with what we think is problem: The site was moved from its server about 3 months ago. Most likely Google is going to the cached IP and getting all kinds of page errors.
I spoke with several other SEO’s and SEO experts and soon discovered that this is a fairly rare occurance; howver it is very real and does happen.

http://www.seroundtable.com/archives/002358.html
(with information from a Google engineer. Note this: [...] until its able to resolve the address for your specific site. This can take up to 3 months to happen. In the meantime it will use the IP of the main site on this IP, often times the hosts site.)

http://groups.google.com/group/google.public.support.general/msg/173c1f58c8088755?hl=en (someone suffering from the caching issue where Googlebot keeps spidering the old server vs. the new…)

http://discuss.fogcreek.com/joelonsoftware2/default.asp?cmd=show&ixPost=47122 (see Kyle’s comment: various client programs and DNS client modules do their own caching, and do not process DNS TLL anywhere near correctly. This includes, for example, Google’s web spider. I moved my web site a while back, and more than a week past the DNS change and TTL expiration, the old server was still getting a slow stream of hits from Googlebot trying to spider the site.)

http://www.w3reports.com/index.php?itemid=549 (see Anthony’s comment: Search engines like Google will often store the IP address of the domain name (possibly a SiteID) when it discovers it. So taking an example from above, Google may first discover this URL:

http://www.somedomain.com/

And Google stores and associates the IP “123.456.789.012” to this domain. Why might they do this? To save on DNS queries which is the bottle neck of running a Web crawler of this size. What Google may do for future requests is first make a request for the IP (which is much more efficient than doing a domain name lookup) and once connected, make a request for the host name (domain name) and path.)

Did You Like This Post? Share with Your Favorite Social Media! These icons link to social bookmarking sites where readers can share and discover new web pages.