Discover the best of the web!
Learn more about Digg by taking the tour.
Open HTTP Connections - The Next Ajax
alex.dojotoolkit.org — Using just what ’s available to the browser, it’s possible to have the server push data encapsulated in blocks and rely on a progressive rendering behavior that every modern browser implements to dispatch events in near real-time (compared to full page refresh or polling delay).
alex.bosworth- 829 diggs
- digg it
- en3r0, on 10/12/2007, -4/+0Sounds promising, I can't wait to see this in action!
___________
-en3r0
http://virtenu.com - huper, on 10/12/2007, -0/+0I am pretty sure this is how the guys at meebo.com have done. They connect.... send data, send data, send data... timeout and then do it again.
Interesting stuff indeed.
-C - JamesWilson, on 10/12/2007, -3/+0I have created a currently unpublished and more reliable method than the above, works in most browsers on most platforms. If you'd like to discuss pricing or details, contact me on aim. OsirisX11
- helensoots, on 10/12/2007, -0/+1Seen www.lightstreamer.com ? Very cool
- ENEX, on 10/12/2007, -0/+0Is there any stripped down + simplified version of the feature so I can look at how the code is written and steps, etc. ?
- JosephC, on 10/12/2007, -2/+1Well this information is useless for Mac/Linux users. I mean the point of a web page/site is that its accessible from different browsers and platforms. If you didn't care about browsers or platforms then why not make it an app? Java? .Net? etc. If you are going to do it via a web browser why not even do it via activex since you don't care about the user?
No DIGG - spyhunter, on 10/12/2007, -0/+0JosephC: the technique works in Firefox too; the ActiveX thing is a *hack* to get around IE's limitations.
- Guspaz, on 10/12/2007, -0/+2I've seen never-ending-connections used in HTTP for YEARS to do cool stuff. For example, CGI:IRC, a server-side IRC client that streams the activity in the various IRC frames to users in realtime using a similar system. This is nothing new.
- centinall, on 10/12/2007, -0/+0so i assume that this mod_pubsub module that he's working changes the server's connection functionality completely? instead of a single connection using a single thread at a time, a thread can now handle multiple connections. That way you can have a never-ending connection without the performance issues?
- v3xt0r, on 10/12/2007, -4/+0n00bs
- jerub, on 10/12/2007, -0/+1Hah, this technology is *old*, pre-ajax-as-a-buzzword.
LivePage, from Nevow, http://divmod.org/trac/wiki/DivmodNevow is based on this concept, and is the current generation of code that was first implemented back in 2002. That was 4 years ago.
The major problem is running it on a server that is sufficiently asyncronous to handle thousands of concurrent incomplete requests - php on apache just doesn't cut it, you need an async web server like Twisted.Web (what nevow and the original woven livepage are based on). - mkjones, on 10/12/2007, -0/+1"Sounds promising, I can't wait to see this in action!"
Well, go to gmail. - rjrjr, on 10/12/2007, -0/+0Or AIM Express http://www.aimexpress.aol.com
Works fine on Mac, Strange JosephC person. - slippery, on 10/12/2007, -4/+0Scripting ActiveX -- no digg.
- choco, on 10/12/2007, -0/+0Check out RTRails: http://rubyforge.org/projects/rtrails/
Live demo: http://ccml01.medien.uni-weimar.de
Uses HTTP streaming over ajax for chat communication and a shared playlist music player - Bogtha, on 10/12/2007, -0/+0@slippery,
This uses exactly the same amount of ActiveX as AJAX - that is to say, other browsers handle it fine without ActiveX, and ActiveX is only necessary with Internet Explorer because it doesn't support doing it any other way. - GooeyMonkey, on 10/12/2007, -2/+0This technique is state of the art...
...for 1998. - bede, on 10/12/2007, -1/+0Looks interesting. Digg
- steakfry, on 10/12/2007, -0/+1Are you people serious? This is how we used to do streaming video around 1997. It sucked. There's nothing even remotely new or exciting about this and there's a reason it's not used anymore, even though it's been around for a long time. Several, in fact: it's unreliable on the client side, it kills the server, and relying on something with as much unpredictability in routing and latency as the Internet for the timing of a display is just plain stupid.
If you can use XML, you can use window.setTimeout(). It's been there since the beginning in JavaScript 1, which was released in Netscape 2.0B1 (along with other new high tech features like frames and plug-ins) in 1995. - TokenUser, on 10/12/2007, -0/+0Ditto what steakfry said.
However, its worth revisiting a lot of the old ideas that didn't quite work because of tech limits of the time to re-evaluate if they would work better in a modern environment. - error401, on 10/12/2007, -0/+0The 'new' way of doing this is to do basically the same thing with AJAX requests. On the client-side you always have an AJAX request open. On the server side, you just block it and don't return any data (other than the header) if there isn't any ready. When an event comes in that you need to dispatch, you send a complete response and finish the request. The client handles it and makes a new one. Generally you'd want to time out the requests after a number of seconds (say, 10) on both client and server-side. On the client, just make a new one. This is to get around proxies and just general timeouts on connections. Do a bit of buffering on both ends and you've got a very usable system. This is how anyterm works, and it's amazingly responsive, definitely usable as a rudimentary terminal.
Just shoving JS at the browser as necessary is an age-old technique. There're better, less problematic ways of achieving the same thing with modern technology, though they do have more overhead. - thecrazyitguy, on 10/12/2007, -0/+0I agree with tokenuser. There is no harm in looking at OpenHTTP, and perhaps being able to integrate the technologies to make it better, faster, stronger. Look at AJAX...seems to me that there are a few technologies melded together, to make it better.
- BRSQUIRRL, on 10/12/2007, -1/+0Let's look at a couple of tasty bits from the article:
>> "How? By cleverly abusing another safe-for-scripting ActiveX control in IE." (details of "abuse" follow)
>> "This is the kind of fundamental technique that is critical to making the next generation of interactive experiences a reality."
So..."clever abuse" is now a fundamental part of web development? Is this the *****-poor state to which web development has been reduced? Hooray for Web 2.0!! - eatmyshorts, on 10/12/2007, -0/+0Steakfry, you are right on the money. This concept has been around as long as HTTP/1.1 has been around. There's nothing new here. Furthermore, taking a stateless technology, with all its benefits in scalability and capacity, and forcing it to do stateful connections is just a bad idea. Sure, there are isolated instances where it makes sense, but the downsides are way too big.
Give me polling with stateless XMLHttpRequest or hidden IFrames any day. Polling ain't that bad, and it sure as hell allows webservers the capacity to handle 100x the number of users as persistent HTTP connections. - Guspaz, on 10/12/2007, -0/+0In most cases, using an XML request to grab updated data from the server every 1 second (or perhaps 5 seconds, or something) is a better choice than this. Way easier to code for, easier to optimize, uses less CPU/memory resources, and doesn't require trickery. Even for an online instant messenger, getting messages within a few seconds is fast enough. You don't need instant delivery, a one second delay is fine.
What I'd do is execute a check once per second, normally returning a very short "no data" type message. If there was data, like a new IM, I'd return that instead. There is overhead from the constant checking, but using up a few bytes per second of bandwidth is insignificant.
Perhaps another method would be an event-driven system. One javascript timer would check once per second for events. When an event was received, it would call the appropriate function to make a full request for whatever data is needed. Slightly more overhead, but easier to code.
Heck, in most cases you could probably increase the time between checks higher than one second. 5 seconds would probably be fine for most things. - JulianMorrison, on 10/12/2007, -0/+0I can see this being useful in one instance: for steganographic/obfuscated tunneling through firewalls. Pipelined "HTTP requests" one way, "Open HTTP" the other. Keep it all on one TCP connection and make it all look like requests for image files or something. Even a packet parsing firewall that understands HTTP wouldn't block it.
- sblinn, on 10/12/2007, -0/+0No polling or IFrames, heck, no HTTP:
AFLAX can use Flash (written only in Javascript) to communicate over TCP, meaning, well, if you've read this far you already know. - Xopl, on 10/12/2007, -0/+0I have to call ***** on this one. Sounds like somebody forgot their 90s WWW.
Hell they used to use this to do animations before animated GIFs were widely supported.
I don't believe this solution is anywhere near as robust or as supported as anyone would tell you or we'd never have stopped using it. People stop using it every time it is discovered.
XMLHTTPRequest/"AJAX" isn't a hack anymore. It has a defined API with defined behaviour. - Xopl, on 10/12/2007, -0/+0Open HTTP Connections - The Old AJAX
That would be the correct title. - meson2k, on 10/12/2007, -0/+0This isn't at all "new". dude i've used this technique like 3 years ago and frankly it suks.
- mercano, on 10/12/2007, -0/+0@guspaz:
In most cases, using an XML request to grab updated data from the server every 1 second (or perhaps 5 seconds, or something) is a better choice than this. [...] There is overhead from the constant checking, but using up a few bytes per second of bandwidth is insignificant.
Plus the constant setup and tear down of TCP connections. Each new request is going to require the setup and eventual destruction of a new TCP pipe, which is going to add at least another 100 bytes and 2.5*Ping of wait time. Might not seem like too much, but can add up if you want your server to be servicing thousands of requests per second. Comes down to whether you want n open continuous connection or n opened and completed connections every t seconds.
Obligatory wikipedia link: http://en.wikipedia.org/wiki/Transmission_Control_Protocol - dolby, on 10/12/2007, -0/+0Anything that says AJAX make it to the front page... but cool digg anyway.
- ph713, on 10/12/2007, -0/+0
The primary show-stopper problem we had with this technique in the 90's was server scalability with so many long-standing open connections.
I would imagine that today, with a carefully crafted application written on a clean framework like Catalyst, running on a very modern and very fast multi-processor Linux 2.6 box, and perhaps most importantly Lighttpd+FastCGI in place of mod_perl/mod_php-style solutions, it's probably doable.
There will still be scalability limits, but you can probably do thousands of simultaneous users of a lightweight application in this style on a simple dual-processor server. More for bigger hardware, or scale via clusters, if you're that big. - hoffmabc, on 10/12/2007, -0/+1Here's some more info on this.
http://www.bluishcoder.co.nz/2005/11/more-on-ajax-and-server-push.html - CaughtThinking, on 10/12/2007, -0/+0this is so old my grandfather coded with it. it's been in like *every* javascript book ever released.
- CaughtThinking, on 10/12/2007, -0/+1I keep thinking all these DHTML "gurus" 10 years from now will wake up one day and go, "OMG I just discovered a new technology, it's called Macromedia *Flash* and apparently it does everything we've been trying to do for years now!"
- merreborn, on 10/12/2007, -0/+0"Open HTTP Connections - The Old AJAX
That would be the correct title."
Damn straight. I worked for a company that had a robust client/server technology based on this technique back in 2001. - cmdrNacho, on 10/12/2007, -0/+0ajax has been around before 2001... it only became hot recently
- nates, on 10/12/2007, -0/+0Welcome to 1998! Wow, this is so modern!
The bad thing is that if you've got 100,000 clients, your server freaks out trying to maintain 100,000 open TCP connections. - Demian.Max, on 10/12/2007, -0/+0a note to everyone saying "wow, this is so old", "welcome to 1998". Read The F-ing Article!
It's clearly explained that this is an old technique that google has implemented in an excellent way (and also talks a little about the server side impact and how livepage/mod_pubsub have handled it).
The person who submitted this obviously wrote up a bad headline and blurb but you still have to read the article, it's not that hard (and really most of the articles are about cold fusion/crystal healing and perpetual motion machines these days and you don't have to read those) - Bogtha, on 10/12/2007, -0/+0@guspaz,
> What I'd do is execute a check once per second, normally returning a very short "no data" type message. If there was data, like a new IM, I'd return that instead. There is overhead from the constant checking, but using up a few bytes per second of bandwidth is insignificant.
Few bytes? A typical HTTP request with no body is ~1K. If Google have a million users who leave their IM client open from 8:30am to 5:30pm¹, that roughly translates to 30 terabytes of data. For *nothing but polling*. I wouldn't call that insignificant.
¹ Yes, I've pulled those numbers out of thin air, but that's the kind of magnitude Google needs to plan for, this isn't a little weblog with a few thousand visitors we're talking about. - sblinn, on 10/12/2007, -0/+0@nates: "The bad thing is that if you've got 100,000 clients, your server freaks out trying to maintain 100,000 open TCP connections."
Hence Yaws or some other Erlang-based server:
http://www.sics.se/~joe/apachevsyaws.html
(Not that there aren't other problems with this solution, but 100K TCP connections isn't so bad.) - jerub, on 10/12/2007, -0/+1@demian.max
if I wanted crappy writeups and poorly thought thru titles, I'd be reading slashdot. - koreth, on 10/12/2007, -0/+0This is news? One of my customers has been doing this for years at their online auction site (http://www.ironplanet.com) to stream bids in real time. I wrote the backend server and the Javascript -- not really that difficult to code up. (Well, stopping the throbber was a bit annoying, but only took a day or two to figure out.) Streaming persistent connections to web pages have been around for so long that entire companies selling toolkits for it have started up and gone out of business already! (Bang Networks, for example.) Other companies like KnowNow have been offering it commercially for years.
I guess it's nice that more people are cluing into the idea, though. - pigdart, on 10/12/2007, -0/+0@Bogtha "ActiveX is only necessary with Internet Explorer because it doesn't support doing it any other way." Active-x is not required. Most people are just too stupid to use anything else. Two simple examples would be: data islands and changing the source attribute on a script include.
- Matt2k, on 10/12/2007, -0/+0Java and Flash called. They said you can keep the change.
© Digg Inc. 2008 —
Content posted by