googleblog.blogspot.com — "I'm often asked about how Google and search engines work. One key question is: how does Google know what parts of a website the site owner wants to have show up in search results? Can publishers specify that some parts of the site should be private and non-searchable? The good news is that those who publish on the web have a lot of control..."
Jan 26, 2007 View in Crawl 4
bitmanxJan 27, 2007
<a class="user" href="http://tool.motoricerca.info/robots-checker.phtml">http://tool.motoricerca.info/robots-checker.phtml</a> to do a quick test..
sohappyluckyJan 27, 2007
Unbelievable<a class="user" href="http://sohappylucky-beforeiforget.blogspot.com">http://sohappylucky-beforeiforget.blogspot.com</a>
phatvolvoJan 27, 2007
it'd have to be BETA.
azmoviezJan 28, 2007
Of course it says "all the major search engines". Could one write a malicious search engine? Aren't they somewhat implying that it's a security measure?
windwakerJan 28, 2007
Anyone who didn't already know about robots.txt is never going to know now.
darnitJan 28, 2007
Its all about the meta tags.....What a stupid digg.
patjkJan 29, 2007
Meta tags only seem to be used for the description under each link on the SERP's. Google is a mystery, especially PR wise. I see some lower level PR sites rank higher on the SERP's than higher PR sites.<a class="user" href="http://www.pjkcubed.com">http://www.pjkcubed.com</a><a class="user" href="http://www.cleancomputerhelp.com">http://www.cleancomputerhelp.com</a>
lemaxfloMar 15, 2010
I still don;t know enough<a class="user" href="http://ezinearticles.com/?Weight-Loss-Diet---6-Powerful-Weight-Loss-Diet-Tips&amp;id=3849910" rel="nofollow">http://ezinearticles.com/?Weight-Loss-Diet---6-Pow ...</a>