digg

Facebook Connect Join Digg About
See the original image at readwriteweb.com

Digg Demos Optimized Datastreams

readwriteweb.com — "Digg [has] revealed a new technology they MXHR, or Multi-Part XML HTTP Requests, as a method for optimizing delivery of Digg's complex AJAX-enhanced site."

Who dugg this? Made popular Apr 23, 2009

Digg DudeWhat is Digg?

Digg is a place for people to discover and share content from anywhere on the web. Digg surfaces the best stuff as submitted and voted on by the community. Take the Tour to learn more.

57 Comments

show profanity settings
expand all
  • seandfeeney, on 04/22/2009, -1/+44In case you missed it (as I almost did):

    Live text demo: http://demos.digg.com/stream/streamDemo.html
    Live image demo: http://demos.digg.com/stream/imageDemo.html
  • linuxwarz, on 04/22/2009, -1/+34Latest firefox- MXHR: 285ms vs Normal: 125ms
  • stef686, on 04/22/2009, -0/+33The image demo is a massive improvement for MXHR, not so on the text demo
  • ahhell, on 04/23/2009, -5/+32So Kevin is posting dupes of stories. That explains a lot.
    http://digg.com/tech_news/Digg_Blog_DUI_Stream_and ...
  • le0pard, on 04/22/2009, -0/+21I like the part where the images load really fast.
  • ssttuu, on 04/22/2009, -1/+17Is it only my connection which loads the Stream Demo in favor of Normal rendering 602ms vs. 468ms?
  • EricAnderton, on 04/23/2009, -0/+13Okay, so the author of the article completely dances around the point, and the technical demos also totally miss the mark of explaining what's going on.

    By sending image data as mime embedded in HTML ala [object data="data:image/gif;base64..."], they can cram a whole bunch of images into a single TCP stream, instead of forcing the browser to create a new connection for every image. So the TCP communication overhead is dramatically reduced by this technique; not to mention that the server's connection pool doesn't get beat up as easily, which only improves performance in other ways.

    I suppose it's also extended to bundling up entire chunks of dynamic text (forum responses in digg.com's case) in the same way.

    Overall, it makes a lot of sense, provided the client machine can handle the extra overhead of cracking that data back out into something usable.
  • inactive, on 04/23/2009, -1/+12If I digg you up, will I get banned?
  • bratterscain, on 04/23/2009, -0/+10For text, normal was almost 5 times faster using latest FF. Though images were greatly faster with MXHR.
  • seandfeeney, on 04/22/2009, -0/+10the MXHR won in all my attempts. I am using the latest Firefox.
  • Cl1mh4224rd, on 04/23/2009, -0/+9Man. Epic fail...

    __Text__
    Stream took: 499ms
    Normal took: 161ms

    __Images__
    Stream took: 25369ms
    Normal, uncached took: 14392ms

    (Using Firefox 3.0.9 on Windows XP SP3.)
  • inactive, on 04/23/2009, -0/+9Submitter accidently the whole MXHR
  • xtraa, on 04/23/2009, -0/+6wow, the graphics load fast
  • brianpeiris, on 04/23/2009, -0/+6A side-effect of this technique is that the streamed images are actually "object" elements with image data encoded in base64 so they don't actually point back to a URL
  • LanceUppercut, on 04/23/2009, -0/+6Of course, most digg users run adblock so they have tons of free cpu cycles to devote to expanding the TCP/IP stream instead of rendering flyswatter + goat ads.
  • eng1ne, on 04/23/2009, -0/+5Where were you when I was writing the post! Seriously though, that's a great explanation. I'm off to dig up my old Celeron laptop to see how well it does on the image test.
  • Xsecrets, on 04/23/2009, -0/+5don't know why this one is getting dugg down In safari on windows the difference is not that much and in chrome on windows both the normal ones are faster. It looks like this is HIGHLY tuned for firefox.
  • misterhektik, on 04/23/2009, -2/+7Firefox 3.1b3
    MXHR Text: 250ms / Normal Text: 101ms
    MXHR Image: 1171ms / Normal Image: 3119ms

    I'm guessing millage varies?
  • Landragoran, on 04/23/2009, -0/+5yeah, the streamed image loaded in roughly half the time of the normal image for me.

    the text on the other hand loaded faster in normal than in the stream.
  • glitchbit, on 04/23/2009, -0/+4so MXHR is pretty much like.. let's zip it all up into one continuous file then send it out and normal http is like lets send a bunch of these files individually... Since the server has to do a bit more thinking I can see that is why it takes longer for the text example where as the benefit is seen in larger data quantities. I suppose the line is the bigger bottleneck than the CPU these days... and the storage medium for that matter.
  • DaviDTC, on 04/23/2009, -1/+5haha buried as dupe.
  • lazyleo, on 04/23/2009, -1/+5FF 3.1 Beta 3 - text
    MXHR Stream,Stream took: 1339ms
    Normal Stream took: 297ms
  • reqage, on 04/23/2009, -0/+3MXHR Image: 2922ms Normal Image: 13320ms
  • ZhiZaki, on 04/23/2009, -0/+3That was my experience.
  • MrManiac, on 04/23/2009, -0/+3On Google Chrome I'm getting substantial gains from MXHR. Try reloading the page a couple of times to get a better data set.
  • fnord185, on 04/23/2009, -3/+6This might be a different article, but he duped me a while ago in case you're keeping track:

    Mine:
    http://digg.com/arts_culture/Elizabeth_Gilbert_on_ ...

    Kevin's:
    http://digg.com/arts_culture/A_Different_Way_To_Th ...
  • eng1ne, on 04/22/2009, -0/+3For the text demo, results vary a lot between different browser in my testing. In Firefox the text demo results were roughly on par with a slight hit to stream render occasionally, but in Chrome, the normal render was often much slower.
  • EricAnderton, on 04/23/2009, -0/+3The whole thing?
  • AndrewDB, on 04/23/2009, -1/+3Disable the stupid thing.
  • krokodil, on 04/23/2009, -5/+7On Safari 4.

    Text:
    Normal took: 46ms
    Stream took: 80ms

    Image:
    Stream took: 2478mss
    Stream took: 2478ms

    Looks like normal is faster in both cases!

  • LazyBeachBum, on 04/23/2009, -0/+2I saw what you did there

    http://www.youtube.com/watch?v=z4t6zNZ-b0A
  • larsonc, on 04/23/2009, -0/+2Are they supposed to load simultaneously? I refreshed a few times with always the same behavior (MXHR loading first, then normal started afterwards).
  • reqage, on 04/23/2009, -2/+4Latest Firefox: MXHR: 536ms vs Normal: 158ms
  • GeckoSlayer, on 04/23/2009, -0/+2Using a nightly build of Minefield 64-bit, I get:

    Text:
    Stream: 2330ms
    Normal: 183ms

    Images:
    Stream: 3436ms
    Normal: 30484ms
  • fcukthisgame, on 04/23/2009, -0/+2I've got 'normal' being lower than MXHR in both... newest FF in 10.5.6
  • DevinWatson, on 05/10/2009, -0/+2░░██▒░░░██▒░░████████▒░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░
    ░░██▒░░░██▒░░░░░██▒░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░
    ░░██▒░░░██▒░░░░░██▒░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░
    ░░██▒░░░██▒░░░░░██▒░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░
    ░░██▒░░░██▒░░░░░██▒░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░
    ░░████████▒░░░░░██▒░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░
    ░░██▒░░░██▒░░░░░██▒░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░
    ░░██▒░░░██▒░░░░░██▒░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░
    ░░██▒░░░██▒░░░░░██▒░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░
    ░░██▒░░░██▒░░░░░██▒░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░
    ░░██▒░░░██▒░░████████▒░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░
    ░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░
    ░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░
    ░░██▒░░░██▒░░████████▒░░██▒░░░██▒░░████████▒░░████▒░██▒░░██████▒░░
    ░░██▒░░██▒░░░██▒░░░░░░░░██▒░░░██▒░░░░░██▒░░░░░████▒░██▒░░██████▒░░
    ░░██▒░██▒░░░░██▒░░░░░░░░██▒░░░██▒░░░░░██▒░░░░░██▒█▒░██▒░░██████▒░░
    ░░██▒██▒░░░░░██▒░░░░░░░░██▒░░░██▒░░░░░██▒░░░░░██▒██▒██▒░░░████▒░░░
    ░░████▒░░░░░░██████▒░░░░░██▒░██▒░░░░░░██▒░░░░░██▒██▒██▒░░░████▒░░░
    ░░████▒░░░░░░██▒░░░░░░░░░██▒░██▒░░░░░░██▒░░░░░██▒██▒██▒░░░░██▒░░░░
    ░░██▒██▒░░░░░██▒░░░░░░░░░██▒░██▒░░░░░░██▒░░░░░██▒██▒██▒░░░░██▒░░░░
    ░░██▒░██▒░░░░██▒░░░░░░░░░░████▒░░░░░░░██▒░░░░░██▒██▒██▒░░░░░░░░░░░
    ░░██▒░░██▒░░░██▒░░░░░░░░░░████▒░░░░░░░██▒░░░░░██▒░████▒░░░░██▒░░░░
    ░░██▒░░░██▒░░████████▒░░░░░██▒░░░░░████████▒░░██▒░████▒░░░░██▒░░░░
  • sharex21, on 04/23/2009, -0/+2chrome 2.0.174.0

    Text:
    Stream took: 2092ms
    Normal took: 44ms

    Images:
    Stream took: 9448ms
    Normal, uncached took: 33828ms
  • AngelBunny, on 04/23/2009, -0/+2weird.. firefoc v3.0.9 both the text and image tests the stream is 2x in speed.
    however in safari v4 on both tests the normal is almost 3x the speed.

    newer browsers are optimized so much I don't see how this matters. If this helps server load that is one thing but on the client side this looks somewhat pointless imho.
  • Trueblood, on 04/23/2009, -1/+2Not exactly a revolutionary concept, but a nice refinement; coders have been bundling multiple chunks of data together for a while with JSON or XML responses. The refinement is in using the MIME format, and the light-weight of the rendering code, both are VERY welcome additions.

    This needs a good collection of back-end libraries to automatically, maybe even asynchronously, bundle the content to be sent to the client.
  • H0M1E, on 04/23/2009, -0/+1Safari Data Collection

    Text Loading Test in Safari 3.1.2 Using MXHR and Normal Streams
    Trail 1|Trail 2 |Trail 3 |Trail 4 |Trail 5 |Trail 6 |Trail 7 |Trail 8 |Trail 9 |Trail 10
    MXHR 829ms|492ms|499ms|537ms|489ms|2279ms|3067ms|412ms|486ms|608ms
    Normal 56ms |421ms|206ms|220ms|295ms|76ms |103ms |422ms|314ms|53ms

    Notes: It's very odd that when MXHR is slow normal streaming seems to go faster.

    Image Loading Test in Safari 3.1.2 Using MXHR and Normal Streams
    Trail 1 |Trail 2 |Trail 3 |Trail 4 |Trail 5 |Trail 6 |Trail 7 |Trail 8 |Trail 9 |Trail 10
    MXHR 2562ms|2224ms|2014ms|2112ms|2239ms|1346ms|5938ms|1714ms|2209ms|6706ms
    Normal 6814ms|5139ms|945ms |1338ms|714ms |2200ms|3818ms|6028ms|7202ms|4257ms

    Notes: Normal image streaming with Safari appears to give a speed before all images are loaded.
    Previous experimentation showed the MXHR stream spiking then dropping every other trail; and the normal stream moving in sync but opposite. Ex. MXHR 6000ms/Normal 1000ms MXHR 1000ms/Normal 6000ms
    MXHR Image Test Trail 3- Failure to load 2 images
    MXHR Image Test Trail 4- Failure to load 1 image
    MXHR Image Test Trail 5- Failure to load 2 images
    MXHR Image Test Trail 9- Failure to load 1 image

    Firefox Data Collection

    Text Loading Test in Firefox 3.0.9 Using MXHR and Normal Streams
    Trail 1|Trail 2 |Trail 3 |Trail 4 |Trail 5 |Trail 6 |Trail 7 |Trail 8|Trail 9|Trail 10
    MXHR 388ms|102ms|301ms|114ms |100ms|119ms|105ms|99ms |416ms|100ms
    Normal 90ms |90ms |90ms |91ms |91ms |93ms |90ms |91ms |90ms | 91ms

    Notes: Normal streaming is incredibly consistent.

    Image Loading Test in Firefox 3.0.9 Using MXHR and Normal Streams
    Trail 1 |Trail 2 |Trail 3 |Trail 4 |Trail 5 |Trail 6 |Trail 7 |Trail 8 |Trail 9 |Trail 10
    MXHR 1411ms|1913ms|2230ms|6202ms|2265ms|1376ms|2239ms|2187ms|985ms |2616ms
    Normal 5280ms|5150ms|5356ms|5208ms|5251ms|5381ms|5337ms|5123ms|5043ms|5279ms

    Notes: Again, normal streaming (given how much more room there is for speed change) is incredibly consistent.

    Opera Data Collection

    Text Loading Test in Opera 9.64 Using MXHR and Normal Streams
    Trail 1|Trail 2 |Trail 3 |Trail 4 |Trail 5|Trail 6 |Trail 7 |Trail 8 |Trail 9|Trail 10
    MXHR 420ms|635ms|424ms|325ms|400ms|326ms|425ms|398ms|517ms|397ms
    Normal 429ms|406ms|391ms|335ms|519ms|444ms|431ms|437ms|422ms|366ms

    Notes: MXHR and normal streams are much slower then the other browsers in the text loading test.

    Image Loading Test in Opera 9.64 Using MXHR and Normal Streams
    Trail 1 |Trail 2 |Trail 3 |Trail 4 |Trail 5 |Trail 6 |Trail 7 |Trail 8 |Trail 9 |Trail 10
    MXHR 1771ms|2100ms|2441ms|2112ms|1901ms|2140ms|2119ms|1950ms|1674ms|1188ms
    Normal 7922ms|7989ms|8242ms|7338ms|7873ms|7761ms|7718ms|7850ms|7850ms|8000ms

    Notes: MXHR really helps Opera’s weak normal image stream speed.

    Camino Data Collection

    Text Loading Test in Camino 1.6.7 Using MXHR and Normal Streams
    Trail 1|Trail 2 |Trail 3 |Trail 4|Trail 5|Trail 6 |Trail 7 |Trail 8 |Trail 9|Trail 10
    MXHR 124ms|314ms|100ms|99ms |100ms|215ms|102ms|100ms|100ms|98ms
    Normal 113ms|111ms|113ms|113ms|112ms|113ms|112ms|112ms|113ms|114ms

    Notes: I hypothesized Camino's data would be extremely similar with Firefox's data since they are both Mozilla products. Firefox does seems to be 22-23ms faster but that is a small margin.

    Image Loading Test in Camino 1.6.7 Using MXHR and Normal Streams
    Trail 1 |Trail 2 |Trail 3 |Trail 4 |Trail 5 |Trail 6 |Trail 7 |Trail 8 |Trail 9 |Trail 10
    MXHR 2187ms |2217ms |1541ms |1946ms |1620ms |2191ms |2160ms |1509ms |1950ms |2107ms
    Normal 14926ms|14791ms|14885ms|14894ms|14934ms|14698ms|14775ms|14987ms|15415ms|16189ms

    Notes: I find this data appalling. Camino showed great speed in the text test but falls face first in images. MXHR's effect is dramatically shown in Camino with monstrous speed increases in image loading.
    MXHR Image Test Trail 10- Failure to load 1 image

    Conclusion

    MXHR seems useless for text data because of most normal streams already fast process. The effect of MXHR in the text-tests are either to small a gain to be significant or actually slower then normal streaming. However, images on the other-hand are a different story. Every browser i used benefitted from the changes with the exception of a few trails (Firefox had one occurrence normal streaming was better and Safari had five). The only problem I encountered was images not loading and random speed drops. It would be interesting to see how IE8, Chrome, and the Windows versions of Opera and Firefox compare to the data I found. For those of you who are wondering, I'm using an iMac 7,1 on a 10mb/s wireless N connection. I get mediocre connection strength so your data might be much better then mine. Try it yourself with your browsers and I hope my experimentation gives you an idea of MXHR effects on the browsers I have available to me.
  • xqb4dpx, on 04/23/2009, -2/+3OK, the SAME story is in the Top 10 together.
  • markdall, on 04/23/2009, -1/+2On IE 8.0, normal (78ms) was faster for text with the stream taking 125ms), the website won't even try to run the images.

    On FF 3.0.8 (3.0.9 is download right now) normal took 133ms, MXHR took 168ms for text.

    For images, FF 3.0.8 took 977ms for streaming, and 18273ms for normal/uncached.
  • benologist, on 04/23/2009, -0/+1Opera consistently gives me up to 3x faster in Normal.
  • linarose, on 04/23/2009, -1/+2Image was already loaded in MXHR when normal started loading
  • robdiggity, on 04/23/2009, -0/+1Odd. FF3/linux, and it pegged my CPU for 18 seconds, the refreshed the display showing the streamed output completed, and the normal output just finishing loading.

    I gotta think this approach is going to be severely impacted by your client side script implementation, and any perf tweaks you may have applied to about:
  • VertigoZA, on 04/23/2009, -1/+2umm... not sure this is right? Text via MXHR Stream took 6853ms to load, and 43ms for Normal? (Safari) Image load was around 1100ms for both.

    Could it be cause I'm in South Africa

    EDIT: just refreshed it 382ms for Normal and 1478ms for MXHR
  • inactive, on 04/23/2009, -1/+2Not sure why, I'm having a hard time not laughing at the "new technology" acronyms naming. Results are spotty at best, not to underestimate their effort, but for some reason I'm picturing some sort of explosive mix of hackish javascript wtf.
  • gbhall, on 04/23/2009, -0/+1I got 630 vs 274 :S
  • inactiv, on 04/23/2009, -0/+1it seems to work fine for me, however if I use internet explorer somehow the normal one renders faster
  • Fetching more discussions...
    Show 51 - 60 of 60 discussions

  • Add a Comment

    Login or register to comment!
    Or, sign-in super fast with your Facebook account ...


© Digg Inc. 2009 — Content created and posted by Digg users is dedicated to the public domain | Terms of Use Updated | Privacy Policy
DIGG, DIGG IT, DUGG, DIGG THIS, Digg graphics, logos, designs, page headers, button icons, scripts, and other service names are the trademarks of Digg Inc.