brian.shaler.name — "Digg.com is an ideal target for creating instant gratification content. You can come out of nowhere and have tens of thousands of new users in 24 hours. An inevitable woe is the Digg Effect -- a sign of success and often the key to server failure." Sharing a programmer's experiences with and advice about avoiding the Digg Effect.
May 6, 2007 View in Crawl 4
invaderMay 6, 2007Submitter
Thanks! Let's just hope the bombardment continues, since that would mean more successful tools. :-)I hope this article helps shed some light on what 3rd party Digg developers are faced with.
scabbersMay 6, 2007
Blue text on a white background - the future of web 2.0
mrfunkeyeMay 6, 2007
if you have administrated content in a database that only changes when edited it is best practice to serialize it as array, hash or xml and then write this to a filewhen needed, deserialize iteven busy, commented pages can be fast using this techniqueonly perform queries when you add a comment, then fetch all comments including the last one and serialize it again
invaderMay 6, 2007Submitter
That's a very good point.The article was geared more toward general web services behind 3rd party Digg tools, than toward blogs (with comments) where this strategy would be entirely necessary. A lot of the cases where users are querying an application, it would be less economical to write the results to files if there is a likelihood it will not be fetched again. However, with moderately static content, like comments, it will save your life (well, your server's life).
blemisherMay 6, 2007
Maybe it won't for long ;) Digg this up and lets see if it can escape the digg effect! No Prisoners!
nickdaddykingMay 6, 2007
when he talks about it, its like a movie " i ran an experiment during the 'revolution'"
mojaamMay 6, 2007
I'm don't know much about servers and web development but reading that, I still learned some things. Caching seems to be key when dealing with large amount of users, Facebook also mentioned that in a recent blog entry. Keep up the good work and hope more success comes to ya.
jackhoffmanMay 7, 2007
My website has recently been on the Digg frontpage and the Digg effect isn't quite as problematic as some people make you believe. From the article: "A logo at the top of the page can be all it takes to push your web server over the edge." I don't think so. Scripts will push your web server over the edge. Static content is practically safe, unless you're on a volume limited account or try to serve files that are bigger than the cache of the server. My logs show more than 6GBytes transferred in the first 1.5 hours of the Digg onslaught. Each visit consisted of about 20 static files with a total size of half a megabyte (most of it in one file). That's an average rate of about 10Mbps, and the site was available all the time. It didn't even show signs of slowdown. At the peak, there were 4 pageviews per second (80 files per second). Yes, that's shared hosting, not a dedicated server. You do have to account for the total transfer volume though: in the two weeks after the story made the frontpage the visitors from Digg alone caused 40GB of traffic.The problems start when you generate content dynamically and don't run completely in RAM: If your script takes 2 seconds to complete, 4 pageviews per second translate to 8 concurrent instances of your script. If the script needs to access the harddisk, the game is over, because disk thrashing will increase the runtime of the script, which means more concurrent instances, which means more disk thrashing...
shredderwareJul 24, 2008
Writing Tools for Digg Users: The Inevitable Digg Effect<a class="user" href="http://www.squidoo.com/shredderwarehouse">http://www.squidoo.com/shredderwarehouse</a>