Submitted by
Users who Dugg This
Dave Beckett
1063 Followers
Scott Reynolds
611 Followers
Will Larson
1679 Followers
Rich Schumacher
1281 Followers
Mike Cieri
346 Followers
Emily Crume
1613 Followers
Dan Contento
244 Followers
aaron deraps
456 Followers
danielrh9Mar 29, 2011
I don't think what's currently being used should even be referred to as "V4" anymore. The backbone of Digg today is entirely different than the buggy product released last year.
lethainMar 29, 2011Staff
Most of the infrastructure has remained constant over the last six months, it's more that we've improved the way we're using it. Lots of improvements in terms of deploying and configuring Cassandra, utilizing Redis for certain functionality where it's better suited, lots of additional application side caching.
danielrh9Mar 29, 2011
Everything that's been done has certainly made it a vastly better product, you guys have done a good job.
The "V4" branding still has a certain stigma associated with it though. Most people associate that with the problems the site had back in August and September. The site works so much better now, going with that same branding is kind of unfair to all of the good work you've done over the past six months.
lethainMar 29, 2011Staff
Definitely agree that V4 isn't a brand we should be cherishing. We definitely appreciate your kind words! :-)
dajobeMar 29, 2011StaffSubmitter
Can I 'retcon' it out of the article? Would anyone notice?
vatosplaceMar 30, 2011
If'n you aient fixed teh 'flexor valve' by now, you probably aient gunna git it.
atomic1fireMar 30, 2011
How about V4.1?
cdurukMar 29, 2011
I'll just leave this here. I believe it is highly relevant to the current discussion you guys are having.
http://vimeo.com/14782834
xraymasterMar 29, 2011
Are you guys working on my sub-category -> one click to upcoming section suggestion?
cdurukMar 30, 2011
Hey! We are always working improvements to the navigation and we did look into your suggestion too. Keep watching this space!
xraymasterMar 30, 2011
http://i.imgur.com/J3lvC.jpg
revovisionaryMar 29, 2011
Wow, that video just embodied a decade of the Internet's power to keep me side tracked. Oh the memories...
Closed AccountMar 30, 2011
yet the top news still takes up twice as much space as it needs to, and is BELOW A GIGANTIC ADVERTISEMENT
thekadMar 29, 2011Staff
I think you already got your response back from lethain, but just to be clear: the only difference (infrastructure-wise) from six months ago is redis and on what clusters we do stuff (like, using redis instead of memcache or instead of cassandra for some processes). Most of the infrastructure was already in place and all the changes you've seen are only on the product side. Weird, right?
weirddemonMar 30, 2011
It's not what you use, but how you use it ;)
kitsuaMar 30, 2011
That's all technical stuff. From a user's perspective, it's practically a different site altogether. In a good way.
ajajadudeMar 29, 2011
I agree that the way this site functions seems like night and day compared to when v4 first rolled out (I can't believe it was that long ago). I truly thought I was done with Digg after the change, but I'm glad to see it getting back on track.
Plus I like the interactions with the staff in these threads.
hulashakesMar 29, 2011
It's probably not buggy anymore because no one is here except for advertisers. Its never under any heavy load.
FormerTrojansMar 29, 2011
with Tigerblood...
Closed AccountMar 29, 2011
new digg is tired of pretending it's not a total bitchin’ rock star from Mars
bs0lMar 30, 2011
If you borrowed digg's brain for five seconds, you’d be like, ‘Dude! Can’t handle it, unplug this bastard!’ It fires in a way that’s maybe not from, uh… this terrestrial realm.
notalessMar 29, 2011
Great stuff.
sfrenchMar 29, 2011
I'd be interested in how you guys are using memcached and redis these days. And by that I mean, which cache objects head to redis as opposed to memcached, and how did redis solve a problem that memcache couldn't?
thekadMar 29, 2011Staff
@sfrench: FTA "[Redis is] The primary store for the personalized news data because it needs to be different for every user and quick to access and update. We use Redis to provide the Digg Streaming API and also for the real time view and click counts since it provides super low latency as a memory-based data storage system."
This basically means 1) we run the mynews part of the site on top of redis these days, instead of cassandra and 2) we serve the streaming API with redis as a backend
I don't think we use redis as a memcache replacement, and I believe we still do the same caching in memcached as we did 6 months ago.
superman101Mar 29, 2011
We do not use redis as a memcache replacement. Memcache is used for object caching.
dajobeMar 29, 2011StaffSubmitter
This confusion is probably my fault from the diagram that has Redis under the "Application Data Caching" box.
dajobeMar 29, 2011StaffSubmitter
Just for the record there are some other comments over on Hacker News at http://news.ycombinator.com/item?id=2383543
chasegreene6Mar 29, 2011
More comments about Digg, on a non-digg site. Lol. Proof digg is dead.
weirddemonMar 30, 2011
Negative. Digg is very much alive and that link has 10 comments.
anomaly100Mar 30, 2011
You're wrong. It's ALIVE!!!! Eeek!
breadfredMar 30, 2011
chasegreene6, in that case, does that make you a necrophiliac?
chickenlocoMar 29, 2011
"We are trying to build social news sites based on user-submitted stories and advertiser-submitted ad content."
A few ads are fine. But building your site around advertiser submitted content? Fail.Comment is buried, click here to see the rest.
revovisionaryMar 29, 2011
Buried for the use of the overused word "Fail". That and Ads are a small price to pay to keep a site running. Just ask Reddit...
chasegreene6Mar 29, 2011
Don't even compare the ads on reddit to the adds on digg. Digg ads are obstructive, in your face, try to pass themselves off as real stories just to get a fake click, and aren't even really ads. Like "Wow! Pay $3.40 for a 60 inch HDTV, Find Out How! Click Here! Hurr hurr!"
That's not an ad. That is spam.
revovisionaryMar 30, 2011
And reddit's been reduced to posting stories on Digg in hopes to get donations to keep the site running. Sure Ads are annoying, but you got to pay the bills.
weirddemonMar 30, 2011
Do you even look at the Digg ads? They're nothing like that. Heck, sometimes, they even seem like real submissions.
confuciussayMar 29, 2011
I wonder if Digg really benefited by changing to a noSQL solution like Casandra.I would think that Digg's database should be quite small compared to the likes of FB who hold a mammoth of data and MySQL + memcached should have no problem handling Diggs overall requirements in terms of scalability. I know Facebook uses it only for their inbox search, but the question is.. has Digg actually gained from a performance perspective because I know the platform had to sacrifice a few functionalities according to Kevin Rose due to Cassandra such a dealing with the pagination of pages.
sfrenchMar 29, 2011
The page load times dropped dramatically. If you look at the bottom bar, you see page load times that are quite often < 500ms. I'm not sure if you remember how fast pre-v4 was, but it was *definitely* not that fast.
Also, with cassandra you get a highly available datastore where there is no "master" and every node can accept writes and reads (as opposed to a replicated mysql solution where you only have a single master db, and slaves which only accept reads)
I'd reccomend this paper if you've never read it. It gives a good understanding on some of the building blocks of dynamo-based key value stores (Cassandra is one of them). http://www.allthingsdistributed.com/2007/10/amazons_dynamo.html
piratearggghhhMar 29, 2011
I love digg but with that said - this is the problem: it's not about the technical infrastructure - the average digger doesn't care about what's happening on the back end. Instead of focusing on or even drawing attention to this, the focus should be community infrastructure building - something intagible that digg has lost. Something more than just switching some links around Kevin and Alex doing diggnation. Years ago, I submitted a story that got dugg to the front page, just cuz it was interesting. I've tried to do that a few more times and it's not even worth my time. Maybe time to think outside the box .. maybe divide submitters into regions that have a better chance to promoting a story? *shrug*
xraymasterMar 29, 2011
Maybe link to the newly submitted stories from each sub-category? People won't click 3 times to find the new stories submitted in their choice of sub-category which is the case with the current UI.
At the moment, users have to click Upcoming ->sub-category->new, in order to find new stories and "My News" just hasn't worked the way it was expected to.
ghounds07Mar 30, 2011
I didn't read the article but if Digg was smart they would be creating algorithms to suggest stories that you might be interested in reading based upon past digging/burying and users with similar interests. Along with the regular popular stories that may interest everyone.
phoufrosMar 29, 2011
Face it guys - digg is dying fast and there is no way to save it, times of digg are long gone - start looking for new jobs.
chickenlocoMar 29, 2011
except reverting back to v3
steve8millerMar 29, 2011
Very detailed information here on how Digg works. I would suggest everyone who uses Digg read this today!
Stubs3dMar 29, 2011
Do we really care? Why do stories about digg always "Magically" make it to the top of the news pile?
bchadraMar 29, 2011
I can digg it!
chasegreene6Mar 29, 2011
We don't care about the infrastructure of Digg. We care(d) about content, users, and originality - something you threw out the window when you released V4. Your frontpage news stories usually don't even break 50 comments.
Digg is a ghost town.
smurfzMar 29, 2011
Why not be open source like Reddit and let users create their own Sub-Diggs.
vatosplaceMar 30, 2011
No more MrBabyMan FTW!
weirddemonMar 30, 2011
Oh, he's still here.
Lurking...
mrbabymanMar 30, 2011
O RLY?
antdudeApr 1, 2011
YA RLY!
vtbarreraMar 30, 2011
You're like the villain in Gotham City that goes "No more Batman FTW!" and then gets dropped on. Don't doubt the presence of the one and only mrbabyman.
xraymasterMar 30, 2011
http://i.imgur.com/JkWRE.gif
anomaly100Mar 30, 2011
You're on the right track and I predict with my anomalous psychic abilities, Digg will triumph!
mhuntMar 30, 2011
No ruby?
dajobeMar 30, 2011StaffSubmitter
Some of the things we use are written in Ruby such as Puppet ( http://www.puppetlabs.com/puppet/introduction/ ) but we don't use it in application code.
djgump35Mar 30, 2011
I didn't think it was broke the last time it got fixed. Now what is broke, is not really the website, but the people, and I for one am thankful most are gone, but perhaps others can step up. I am not one to aggregate or further such things along. I appreciate it when others do, but really I rarely ever found anything new here or TC or anywhere else. Perhaps if there were more done to further a culture of digg, rather than a mainstream traffic increase, a la fark.com, just saying, that would be cool.
imagepushMar 30, 2011
Nice to see that Redis is getting support on Digg. Do you use PHP framework on digg or everything is made in-house (by historical reasons)?
superman101Mar 30, 2011
https://github.com/digg/App hasn't been publicly updated in a long time but we wrote this Controller framework. It is what we use.
It is really really barebones, and it is just about Controllers, not Models or Views. But in it you can see on a HTTP request from Apache gets built in App/Request.php and then which Controller method is chosen with the App/Middleware/Event.php. There is a lot more going on in there as it really sets the ground work and the interfaces on how the HTTP Request is handled.
We use Smarty for View templating. And from the App framework you can see an Interface for the Context Processor. A Context Processor is allowed to inject more variables into the View context after the Controller has run.
PHP frontend communicates with the backend over Thrift: http://incubator.apache.org/thrift/. Thrift froms the basics of our Models but because Thrift creates classes that are really 'structs' we extend those basic Thrift classes with more indepth Models that contain useful methods.
We use PHPUnit for are testing with a custom code sniffer.
*Phew! I think that covers most of the visible PHP stuff.
imagepushMar 30, 2011
Thanks for such detailed answer. I suppose, Smarty is there by historical reasons. Yes, Smarty is very powerful, but as soon as Digg has quite a few options for customizing views, it seems that plain PHP might work the same well (and faster). Or does Digg hardly utilize compiling and caching features of Smarty ?
superman101Mar 30, 2011
We do use compiling and caching of Smarty3 and whether or not its faster then vanilla PHP isn't something we aren't actively concerned with. The difference, based solely on intuition, I would expect to be very very minor.
imagepushMar 30, 2011
My concern about Smarty was that I didn't want to learn new markup if I can do the same with plain PHP. Speed is not a major issue, agree.
cdurukMar 30, 2011
Smarty markup is really simple, if you write HTML by hand, writing smarty shouldn't be an issue at all.
The main reason we use smarty is that it helps keep our presentation layer code (HTML) separate from the application code (PHP).
We also use a lot of custom Smarty plugins to do stuff like number formatting, assigning CSS classes, time formatting and such. That allows to write complicated view layer code once and reuse it again.
dajobeMar 30, 2011StaffSubmitter
I vote @superman101 turns this into a blog post :)
CottonPickingDiggerApr 21, 2011
I concur - always love reading-up on the technical, back-end aspects of Digg.
Also, it'd be nice if the posts on Digg's blog could could support tags, so that we could, say, only filter-in posts about Digg's technical side.
DevanshilocMar 30, 2011
I wonder what it would have been like had there been no Digg ! :D
pala53Mar 30, 2011
Work, and much good "V4!
monvalleyMar 30, 2011
If an article is submitted, can it be deleted by other users?
avalexlubicichMar 30, 2011
FYI
bosem05Mar 30, 2011
nice
january14nMar 30, 2011
i love digg because i get myself updated with whats people love to read.
macharborguyMar 30, 2011
This is the way that Digg is built
this is the way that Digg is built
this is the way that Digg is built
Not with a Digg but a Bury
roulettered56Mar 30, 2011
I AM NOT LEGO!
mmarkgrafMar 30, 2011
i need some more diggs help please ill follow you!!
ganeshitMar 30, 2011
Great
vtbarreraMar 30, 2011
I had the privilege of meeting some of the Digg staffers in Austin; they're a very driven group of folks and their hard work certainly shows. Keep up the great work guys! =)
drake121Mar 30, 2011
very interesting and good
illichmMar 30, 2011
One thing wasn't clear in the article: is the main digg.com website also powered by Drupal?
stignordasMar 30, 2011
Time to shed the v4 moniker. I suggest "IB" for "is back!"