What is Digg?71 Comments
- jesusphreak, on 10/12/2007, -2/+45You almost get the feeling that small sites hit by Digg or Slashdot are more or less getting raped (especially Digg).
Everyone goes and oohs and aahs at the article for a few hours, probably crashing the server, always using up a good amount of bandwidth, and then most of them leave without ever coming back.
There's almost something inherently disrespectful about that. - Mongoose, on 10/12/2007, -2/+21I find it hilarious that a story toting immunity to the digg effect falls victim to it itself.
- Otto, on 10/12/2007, -1/+13That's an awful lot of PHP to duplicate what would be quite a bit faster to just do in an .htaccess file with mod_rewrite.
RewriteEngine on
RewriteCond %{REMOTE_ADDR} !^aaa.bbb.ccc.ddd$
RewriteCond %{HTTP_USER_AGENT} !^CoralWebPrx
RewriteCond %{QUERY_STRING} !(^|&)coral-no-serve$
RewriteCond %{HTTP_REFERER} ^http://(www.)?digg.com [OR]
RewriteCond %{HTTP_REFERER} ^http://(www.)?slashdot.org [OR]
RewriteCond %{HTTP_REFERER} ^http://(www.)?slashdot.com [OR]
RewriteCond %{HTTP_REFERER} ^http://(www.)?fark.com [OR]
RewriteCond %{HTTP_REFERER} ^http://(www.)?somethingawful.com [OR]
RewriteCond %{HTTP_REFERER} ^http://(www.)?kuro5hin.org [OR]
RewriteCond %{HTTP_REFERER} ^http://(www.)?engadget.com [OR]
RewriteCond %{HTTP_REFERER} ^http://(www.)?boingboing.net [OR]
RewriteCond %{HTTP_REFERER} ^http://(www.)?del.icio.us
RewriteRule ^(.*)$ http://www.yourdomain.com.nyud.net:8080/$1 [R,L]
Replace aaa.bbb.ccc.ddd with your own IP (so you don't get redirected) and replace the last line with your server's name (with the nyud.net:8080 appended to it, of course).
Those who don't understand mod_rewrite are doomed to reinvent it. - Otto, on 10/12/2007, -1/+13thewebguy: The problem with a PHP solution like this is that wordpress still has to load into memory and such, and the PHP code has to be interpreted. Whereas with a mod_rewrite scheme, the redirection happens entirely in Apache, the blog never even has to load up into memory or anything.
Rewrite rules are the better way to go. The only downside is that you don't get a hit to your Wordpress, if you are tracking referrers from there. You'll still get a hit in your Apache server logs though, which is the better way to track that sort of thing anyway.
See here for the RewriteRules to accomplish exactly what this plugin does: http://ottodestruct.com/diggprotectionrules.txt - mmichie, on 10/12/2007, -0/+12Not only that but most digg users don't click ads either. It can often be a net loss for someone that pays a lot for their bandwidth.
- inactive, on 10/12/2007, -0/+8actually, the cache is slow...
the site might still be pretty quick.
so its actually doing its job..
check the site thats given here, and the site that actually opens..
it automatically switches to coral cache ;)
edit : yep, open a new tab, and copy paste the link there...its still damn fast...
digg for a great concept :) - Falcorian, on 10/12/2007, -3/+8Or heck, adblock them out, I know I do.
- benjaminlyu, on 10/12/2007, -0/+4Great example! A few modifications I made:
1. Don't redirect the Googlebot
2. Also redirect from any subdomain of the high traffic sites, not just www.
3. Redirect the main page too /. the ^(.*)$ seemed to not redirect the root on dreamhosts' apache installations. I changed it to ^(.*)?$
RewriteEngine on
RewriteCond %{HTTP_USER_AGENT} !^Googlebot
RewriteCond %{HTTP_USER_AGENT} !^CoralWebPrx
RewriteCond %{QUERY_STRING} !(^|&)coral-no-serve$
RewriteCond %{HTTP_REFERER} ^http://([^/]+.)?digg.com [OR]
RewriteCond %{HTTP_REFERER} ^http://([^/]+.)?slashdot.org [OR]
RewriteCond %{HTTP_REFERER} ^http://([^/]+.)?slashdot.com [OR]
RewriteCond %{HTTP_REFERER} ^http://([^/]+.)?fark.com [OR]
RewriteCond %{HTTP_REFERER} ^http://([^/]+.)?somethingawful.com [OR]
RewriteCond %{HTTP_REFERER} ^http://([^/]+.)?kuro5hin.org [OR]
RewriteCond %{HTTP_REFERER} ^http://([^/]+.)?engadget.com [OR]
RewriteCond %{HTTP_REFERER} ^http://([^/]+.)?boingboing.net [OR]
RewriteCond %{HTTP_REFERER} ^http://([^/]+.)?del.icio.us
RewriteRule ^(.*)?$ http://example.com.nyud.net:8080/$1 [R,L] - inactive, on 10/12/2007, -0/+4digg should delop its own cache for every story that hits the front page
-wolf - Daniel591992, on 10/12/2007, -0/+4@mmichie (just to add something)
and everyone is always complaining when the page has ads. - Otto, on 10/12/2007, -0/+4ecb29: You missed my point. It's done in PHP. That means, at the very least, that the PHP has to be interpreted. Now, that may be very fast indeed if your site is running the php module for Apache (as opposed to using php.exe or some such), but it's still going to be slower than going through the rewrite engine, especially since the rewrite engine will have finished processing by the time the php gets around to executing your code.
Okay, okay, so the full out Wordpress code won't need to be executed. But why wait for the actual request processing part of the apache execution path when you can shortcut it entirely? - Otto, on 10/12/2007, -1/+5Argh.. Digg messed up the formatting. Look here for the actual code:
http://ottodestruct.com/diggprotectionrules.txt
You can tell it works by the way you get redirected. :) - toveling, on 10/12/2007, -1/+4@Otto
If you use a php interpreter cache (like APC), the PHP code only has to be parsed once, then it is loaded from cache. For something like a Drupal blog, that would have immense effects (5-10x faster). Mod_rewrite is still much faster, but this is a cleaner approach. - jasonhazel, on 10/12/2007, -0/+3this has been brought up numerous times, and if memory serves, it was decided against.
if every front page story was cached, ad revenue to the sites that are providing the content would be lost. - inactive, on 10/12/2007, -0/+3It's neat--it's what's protecting my blog right now from you guys. You could say you're all my beta testers ;)
- inactive, on 10/12/2007, -0/+2The "can I write this or not" section could probably be cleaned up a significant amount. The rest of the code is very, very simple.
- inactive, on 10/12/2007, -1/+3That's not true--this plugin loads before Wordpress does using a special hook for caching plugins. That's why it's fast ;)
- Eddible, on 10/12/2007, -0/+2Despite that though, more needs to be done to protect sites that will be raped by the Dugg effect, I feel Digg has an expectation to protect sites from being Dugg too hard. I think this plugin is a definate step in the right direction though. But, of course, it is only wordpress blogs that have some sort of solution at the moment.
- Jeffrey903, on 10/12/2007, -0/+2This seems very cool. Basically what it does is that if the referrer URL is from one of the popular link websites (digg, slashdot, delicious, etc), it redirects it to the coral cache version of that page. If you were to go to the site normally, you would get the regular site (no coral cache).
- vdxc, on 09/29/2008, -0/+2actually I think this code has been written badly and could be done a lot simpler in PHP.
- shindig111, on 10/12/2007, -2/+4I just tested this plugin... There are some things that I took note of...
1. The ad is not visible with the redirection
2. The cache is pretty slow. Slower than most websites.
3. Stats don't work.
4. Any changes done to the article at a later time does not reflect on the cache...
These are just constructive criticisms.
I tested it on my website : http://technopedia.info/tech
If you have any questions, feel free to contact me... - mugatu, on 10/12/2007, -0/+2I scripted up a system a while back that is somewhat similar, but not WordPress-specific: http://blog.mugfu.com/2006/01/26/auto-coralized-virtual-hosts/
(note: that site does not actually make use of the technique described, but I do have it enabled on other hosts) - Mesum, on 10/12/2007, -0/+2While it's understandable that WP dies much faster than a static site, Digg effect is something way much more than that, it can crash a whole server and suck any amount of bandwidth within few hours.
- Mongoose, on 10/12/2007, -2/+3What does this have to do with anything..?
- Dudemullet, on 10/12/2007, -0/+1I really hope this stops the digg overkills that have been going on, its uncool to check out what would seem like a really cool story, only to find an error page returned. I hope that webmasters get a hand on this plug-in quick.
- GamingFox, on 10/12/2007, -0/+1Looking very nice plugin... I really hope someone created a module like that for Drupal...
- Splizxer, on 10/12/2007, -0/+1How ironic
- evilspoons, on 10/12/2007, -0/+1It looks like the cache got pwned this time around... mine redirected to nyud.net instantly but then THAT took agest to load.
- ccheath, on 10/12/2007, -0/+1i'm got the dugg story link to come up fine but otto's link redirected to nyud and is really slow, but I eventually got it...
this problem is not trivial - mattroberts, on 10/12/2007, -0/+1Sites back up - and no noticeable slowness
- inactive, on 10/12/2007, -0/+1Yeah site is up and working. Plus you must make allowance for the fact that it has many highly dugg stories. http://elliottback.com/wp/digg-me/. So even with protection it can go down.
- molgar, on 10/12/2007, -0/+1Did you actually read what this plugin does? RTFM!!!
- inactive, on 10/12/2007, -0/+1You know, Wordpress removed the mod_rewrite rules in favor of an all PHP solution. So, it's within their design strategy to handle this kind of thing within their engine.
- acidreflex, on 10/12/2007, -0/+1DSR - Digg Self-reference #2132
- Otto, on 10/12/2007, -0/+1Wordpress removed the mod_rewrite rules because they were having difficulties with keeping the .htaccess file up to date as well as changing the .htaccess file while the server was in the middle of accessing it. It's not that it's part of their "design strategy", so much as it was "it wasn't working the way we did it before".
Dynamic .htaccess files can be a problem.
Static ones, not so much.
Rewriting is simply going to be faster any way you slice it. When you're talking about optimizing for huge transitory loads, then you really don't have any other valid choices. - jer2eydevil88, on 10/12/2007, -0/+1It didn't load for me, I guess the cache is being dugg to death now..
- Drizzit, on 10/12/2007, -0/+1I've had that idea for awhile I just never got around to doing anything about it. My method still used php but to rewrite rules for mod_rewrite on the fly depending on traffic load and where it was coming from.
- Mysk, on 10/12/2007, -1/+1Nice work.
- thewebguy, on 10/12/2007, -1/+1it seems that a huge percentage of the sites are wordpress sites, so it seems that a good plugin would reduce the amount of server side work that wordpress does once it receives a certain number of hits in a short period of time.
i donno - raccettura, on 10/12/2007, -1/+1Ok, this was funny that the site went down.
I guess if you create something like this... you need to test it. Or just don't promise something will work. - ahmerhussain, on 10/12/2007, -1/+1I guess this is putting it to the test.
- Aurous, on 10/10/2007, -0/+0True, maybe Digg can come up with a revenue sharing plan with the webmasters to pay for all the extra bandwidth! But I guess they can't really complain, websites are for the WWW to see!
- Aurous, on 10/10/2007, -0/+0Its really a good idea, but I don't think the plugin works at its best. :(
Hope he develops it some more, it would really be worth the trouble then! - Aurous, on 10/10/2007, -0/+0How is it actually different from WP Cache?
Doesn't it just serve it from the cache, lowering the server demand? - Peve, on 10/12/2007, -1/+0I am surprised but it seems to be working really well
- grimzy, on 10/12/2007, -9/+8Looks like its not working that well.
- Aurous, on 10/10/2007, -1/+0yes, but digg users are known NOT to click on ads at all!
in fact, that is the principal complaint of webmasters, digg users rape their websites and pay them petty change! - protiek, on 10/12/2007, -2/+1Works fine for me?
- kungfustickman, on 10/12/2007, -4/+3Is there a pluggin that will deflect digg traffic? Like the privacy script in google.
- inactive, on 10/12/2007, -7/+5What a jackass...
-
Show 51 - 69 of 69 discussions
© Digg Inc. 2009
— Content created and posted by Digg users is