When I was in High School, LambdaMOO opened to the public. I spent a lot of time in high school and college hanging out there, programming, and even did my first web application development on E_MOO, which had an HTTP server implemented in moocode. After E_MOO went down, I decided that I wanted to recreate the MOO experience as a graphical multiuser networked programming environment. The ability to log in to a machine, edit some code, manipulate the "object" whose code you just edited, and hand it to your friend halfway around the world for debugging is a very compelling experience. I hadn't seen it recreated to my satisfaction and I wanted to do it.
I spent a few years coding, but it was slow going. Both network applications and GUI applications require a different mindset than the batch process style of programming which is what you learn in computer science classes. Event driven programming takes a lot of getting used to, and even when you understand what you are doing, it can be difficult to write and debug event-driven in C or C++, the language of choice in the mid 90s.
Shortly thereafter I got a full-time job writing Python web applications. After developing for a few years in Webware and Zope, I decided to play around with some ideas I had for an easier to use templating system. This led to the creation of DOMTemplate, Woven, and Nevow, and it turned out to be a longer process than I had hoped. Sometime in late 2002 I decided to revisit the idea of performing out-of-band communications with a server to allow the web application running serverside to push and pull information into and out of the browser. I called it LivePage, and it was based on a highly transparent Model-View-Controller design, where controllers received events from the browser, updated models, and views automatically re-rendered themselves based on model dependencies. The results were then shipped to the browser and some DOM hackery was employed to replace the old DOM fragment with the newly rendered view.
The original Woven implementation of LivePage used Flash XMLSocket as the out-of-band event conduit. XMLHttpRequest was around at the time, and I did attempt to use it, but it reeked of MSIE nastiness and wasn't very standard cross-browser. XMLHttpRequest also has the distinct disadvantage of being based on HTTP. XMLSocket is a persistent, two-way, asynchronous socket architecture that made it delightfully easy to implement both Client-to-Server events and Server-to-Client events. XMLHttpRequest can only send data to the server once, and if you repeatedly send data from the server to the client and handle it incrementally, the document will consume memory without bound.
People thought I was crazy to even try to make web browsers do the things I wanted to do. It'll never work, it'll never be cross-browser, browsers can't handle it, people won't understand it... But I persevered. Browser stability was a real problem. After leaving a live page open for a while, with lots of changes being sent, browsers would leak all over the place and eventually crash. Both IE and Mozilla crashed like crazy. Eventually, after Gmail came out, the browser vendors seemed to get things under control and DOM mutation is relatively reliable now.
In October of 2003 when I sat down to spend a bit of hacking to try to come up with ways to simplify Woven, Nevow went from proof-of-concept to ready to use in a weekend. It wasn't until Gmail was in private beta that I decided it was time to revisit LivePage. This time, the aim was to keep the implementation as short and easy to understand as possible, with the hopes that other people would be able to figure out how and why to use it. I also decided to use XMLHttpRequest as an experiment, to reduce the number of dependencies. XMLHttpRequest makes some aspects of the architecture more difficult (Server-to-Client events) but in some ways makes other parts of the implementation much easier.
I have spent 10 years of my life, on and off, working towards this dream I have. Nevow and LivePage are the latest incarnations of puzzle pieces which fit into the mosaic of this dream. Hopefully someday very soon I will start fitting the pieces together and start to see what a true multiuser web application looks like. We already have a taste for what collaboration can enable; sharing information using weblogs, instant messengers, wikis, file sharing applications, and tools like SubEthaEdit allows to more rapidly fine-tune our idea of what others are thinking. Only a few pieces are missing; a more graphical, spatial ability to arrange information, where information can be picked up easily and carried from place to place in order to sort and categorize it; a more real-time sense of community where presence information gives us a sense of who we are near and allows us to easily communicate with them; and a more immediate programming model where changes can be made to live applications and applied immediately, with results available for all to see. All the pieces are there, they just need to be put together to complete the puzzle.