sitepronews.com — Google uses a number of spiders to crawl the Web. You can catch these spiders by examining your log files. This article will reveal some of the most important Google spiders, their function, and how they affect you as a web master.
Jul 19, 2006 View in Crawl 4
prk60091Jul 19, 2006
googles three laws: 1. a bot may not allow content to go without indexing or or, through inaction, allow content to go without indexing. 2. a bot must obey the orders given to it by advertisers except where such orders would conflict with the First Law. 3 a bot must protect its own existence, as long as such protection does not conflict with the First or Second Law or do evil.
matthewhallJul 19, 2006
Great little guide to GoogleBots.
baccillusJul 19, 2006
love this great article too. very well written and easy to understand.