drunkmenworkhere.org — "A large scale experiment on search engine behaviour was staged with more than two billion different web pages. This experiment lasted exactly one year, until April 13th. In this period the three mayor search engines requested more than one million pages of the tree, from more than hundred thousand different URLs."
May 7, 2006 View in Crawl 4
jcowdyMay 8, 2006
Yes, while Yahoo's bots might be the most active, they are not quite the brightest. If you take a look down toward the bottom and compare the yahoo and Google trees, you will notice a huge difference. Yahoo's search tree is very random and sparatic, it lacks a sense of depth and order. While Google's bot on the other hand searches more systematicly and orderly. It recognizes page rank and importance and crawls them accordingly.
eyrieowlMay 8, 2006
fascinating. thank you for that.
mrassmanMay 8, 2006
yeah but, this one was submitted by kevin rose..
nessguyMay 8, 2006
Very well written. Even though I didn't understand some of it I still found it fascinating.
andorMay 9, 2006
Yes, I agree. There is no conclusion to this. And like someone has mentioned, the writing is also sloppy. Not well made. But the graphs were good although the analysis was directionless.