What is Digg?252 Comments
- shadowspawn, on 10/15/2007, -5/+317god damnit the first rule about fightclub...
- theoallardyce, on 10/18/2007, -17/+257the best way to get this bug fixed is to make site owners aware by digging this up to the front page
- sickswaystop, on 10/17/2007, -5/+231cool article, you can download the firefox extension here: https://addons.mozilla.org/en-US/firefox/addon/59
it definitely works but i still cant find any porn sites this works for. Anybody? - Felshadow, on 10/11/2007, -5/+223Omg....i knew the first comment would be about this hahah!
- speezer, on 10/11/2007, -9/+146http://www.hornymanatee.com/
- whatsupimphil, on 10/11/2007, -4/+111I wouldn't expect it to work for porn because there is no reason for the sites to allow Google access to video and audio content which won't be indexed.
- mattcoady, on 10/11/2007, -2/+83...strictly testing. No other reason....
- LifeIsAwesome, on 10/11/2007, -3/+75Then this hack is useless.
- oleeid, on 10/11/2007, -6/+69dont know if this is an old trick but ive just come across a site that explains how to disguise yourself as google bot. this allows you to access pay sites 4 free
Ever wondered why a google search returned search results that lead to sites that require a registration ? How the hell did googlebot index the site without a registration ? Many sites want their site indexed in google to recieve more hits and thus, make more money with advertisment on their site. Even more interesting is the following question: How do we use this to our advantage ?
Its fairly easy, we have to disguise ourselfs as the google bot and voila, many sites let us in without registration. The parameter we have to change is called the User Agent. The user agent defines the browser and version that you use. For example it would show Internet Explorer as your browser if you are using this one.
You need to alter your settings to the following:
User Agent: Googlebot/2.1
Compatible: http://www.googlebot.com/bot.html
You can do so in Opera with ease. Firefox offers an extension which is downloadable from the official website.
For Internet Explorer you need to change registry entries.
Windows Registry Editor Version 5.00 [HKEY_LOCAL_MACHINESOFTWAREMicrosoftWindowsCurrentVersionInternet Settings5.0User Agent] @="Googlebot/2.1" "Compatible"="+http://www.googlebot.com/bot.html"
Save this as ua.reg and execute.
To revert the changes back, you need the following:
Windows Registry Editor Version 5.00 [HKEY_LOCAL_MACHINESOFTWAREMicrosoftWindowsCurrentVersionInternet Settings5.0User Agent] @="Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
Save as oua.reg and execute.
im not sure how affective this is, has anyone tried it before??? - BlackAdderIII, on 10/11/2007, -2/+53Yes, that and other means of doing the same thing.
I recommend we bury it and don't talk about it, and keep doing it without it getting stopped. :) - AmishRefugee, on 10/11/2007, -1/+46does anyone have an example site to see if I (or anyone else really) set it up right?
- g4blows, on 10/11/2007, -6/+48go to www.getfirefox.com and follow the steps provided
- Rocketman1882, on 10/11/2007, -2/+41I got it to work on some porn sites, but it seems that you can't download any of the content without actually being logged in, probably a missing cookie, but other than that, you can go through everything else on the sites.
- AcePup, on 10/11/2007, -4/+42Mirror: http://duggmirror.com/security/Hack_Lets_You_Access_Pay_Sites_Free_by_Disguising_Your_Browser_as_GoogleBot/
- gr00vy, on 10/11/2007, -4/+42By "many" you mean "porn" right?
- byronm, on 10/11/2007, -5/+43Like the hack, but didn't google say that displaying content for the spider different than anonymous content is worthy of getting dropped?
- jellystones, on 10/11/2007, -2/+31If subscriber sites allowed google bots to scan their site, wouldnt their content appear on the google cache?
- biffsputnik, on 10/11/2007, -1/+29No, the first rule of project mayhem is that you do not ask questions
- thomble, on 10/11/2007, -1/+26There's an easier way to do this: Click on the "Cached" link in the Google search results page.
- JayD16, on 10/11/2007, -4/+29Whats the point if you can't access the content?
- GMorgan, on 10/30/2007, -1/+21They won't stop it. They are dependant on search engines. If they block the googlebot then they die.
- KnightMareInc, on 10/11/2007, -3/+22ok, thats clever.I had no idea sites allow google's bot to access for free.
- robcornelius, on 10/11/2007, -0/+19The company I work for provides about 23 million academic papers. Google loves us (especially google scholar) We now have one whole instance of our sites installation just for googlebot to index non-stop 24/7 as new content gets added all the time.
This wont work for our site as we route the googlebot based on its IP address range so I guess it wont work for a lot of sites that have a similar arrangement with Google. - SurrealDream, on 10/11/2007, -0/+18OH THE HUGE MANATEE!
...Go on, say it fast. - virtualball, on 10/11/2007, -0/+17God bless Conan O'Brien
- jcaino, on 10/11/2007, -3/+20all this will do is cause site owners to add a line to the .htaccess file denying access to the bot agent, except for certain ip ranges.
- conman16x, on 10/11/2007, -0/+16Wow. That's several different kinds of failure.
- mitrovarr, on 10/11/2007, -1/+17Am I the only one who thinks that Google shouldn't index pay content? You're basically spamming the search with stuff people can't actually use and burying things that might actually help them. Googlebot shouldn't be allowed anywhere that random internet people aren't, and the only part of your site that should show up in a normal google search is the front page. I'm sick of picking through twenty crappy pay sites for good free ones.
- esrun, on 10/11/2007, -0/+16This is extremely old and nearly all pay sites have updated to check the originating IP address against that of a list of known googlebots.
- andyduncan, on 10/11/2007, -1/+16Good, hopefully the sites will stop spamming google with their inaccessible content. The worst abusers of this are the "tech-answers" sites, where they let Google index the page, and then when you try to see the article describing some sort of solution, they hide it behind a for-pay wall.
I have no problem with people charging for their content, but this kind of google spamming is dishonest, sneaky and annoying. If they want to advertise their for-pay services, they should use adwords.
If getting this technique out in the hands of thousands of rabid Digg users gets these jackasses to stop their evil tactics, then yeah, it's worth "spoiling" it by letting everyone know the secret. - DanAtkinson, on 10/11/2007, -1/+15Because we always love it when someone cracks a joke which is further up the page...
- Bdog2g2, on 10/11/2007, -3/+17I'm sorry did I insult your boyfriend or what?
- angrykeyboarder, on 10/11/2007, -0/+13For starters, if you would like to know if your browser is transmitting the faked user agent string vist http://whatsmyuseragent.com
- fenderd1967, on 10/11/2007, -1/+14Open Opera
Type about:config
Press Enter/Return
Scroll down to User Agent
Type Googlebot/2.1 http://www.googlebot.com/bot.html
Click Save
Then close Opera and reopen. - iloveazngurlzs, on 10/11/2007, -0/+13everyone and their own pr0n addicted mother will be using this one
- WallnutBoy, on 10/11/2007, -1/+14Using the firefox plug in linked above ( https://addons.mozilla.org/en-US/firefox/addon/59 ) all you have to do is bring up the options for User Agent Switcher, go to the list of available agents, click Add and in Description field type anything you want.. For example I just put 'Google'. Then in User Agent field just type in :
Googlebot/2.1 (http://www.googlebot.com/bot.html)
none of the other fields need to have anything in them.. Just save it, change agents and there ya go!
anyone got any cool sites to test this on?? - apollomurga, on 10/11/2007, -5/+17Guys stop digging it cause we don't want this on front page you know, so don't digg it up.
- inactive, on 10/11/2007, -6/+18Stop bloody digging this article. If you like it the best thing you can do is not dig it
- Mwongozi, on 10/11/2007, -1/+12In the Terminal do: defaults write com.apple.safari IncludeDebugMenu 1
This will enable a Debug menu which lets you change the user agent. - hinterteile, on 10/11/2007, -4/+14http://www.avivadirectory.com/bethebot/ is a handy site to do the same thing.
- drmessano, on 10/11/2007, -4/+14Site or it didn't happen
- esrun, on 10/11/2007, -0/+10Sites that want their pages indexed in search engines for more click through traffic but at the same time don't want to give the content away free.
- archlich, on 10/11/2007, -3/+12Welp, time for me to get a job at google.
- ricepicker4000, on 10/11/2007, -4/+13You don't......
- loswr86, on 10/11/2007, -1/+10doesn't work for espn insider
- flagg85, on 10/11/2007, -2/+10I use to like manatees.
- Supernova36, on 10/11/2007, -0/+8Not for paysites..
- CamperBob, on 10/11/2007, -0/+8Agreed 100%. I am REALLY sick of getting 8 pages of paywalled IEEE spam every time I query an electronics topic in Google.
- HalFTW, on 10/11/2007, -0/+8It works perfectly on http://www.experts-exchange.com
Search for some questions and try viewing them with and without googlebot spoofing: http://www.google.com/search?q=site%3Awww.experts-exchange.com+noob -
Show 51 - 100 of 247 discussions
The Digg Toolbar for Firefox lets you Digg, submit content, and keep track of Digg even when you're not on the Digg site. Download the official 
© Digg Inc. 2009
— Content created and posted by Digg users is