Nevow 0.3!

Nevow 0.3 is now out! Go get it!

There has been a lot of development over the past five months, and I have been feeling overdue for doing a Nevow release for a while now. So finally today I mustered the strength to write all the ChangeLogs, READMEs, and update the examples and documentation needed for a release. And what a release it is. This release encompasses some pretty major refactorings and additions:

• freeform-patterns: This branch rewrote almost all of the freeform form rendering code (now called webform). It makes use of the Nevow pattern/slot paradigm to allow customization of the automatically generated forms down to the finest detail.


• ubiquitous-context: This branch focused on making the Nevow APIs more consistent by passing a context object to the URL traversal APIs. Nevow keeps track of the objects involved in URL traversal and page rendering using a stack of context objects which you can use to make information available to your components.


• LivePage improvements: LivePage is now cross browser! Mozilla, Firefox, WinIE6, and Safari browsers are all supported. Adding support for browsers such as Opera and Konqueror should be possible as long as they support XmlHttpRequest. Also, some python-side APIs were added which make mutating the in-browser DOM a breeze: set, append, alert, and call. All these python-side objects take stan, so you can pass them XML fragments composed using Python code or extracted from XML/XHTML documents. No JavaScript skills required!


• New Fragment class: Fragment is a Page base class which implements just those APIs required to expose data_* and render_* methods in a stan tree. It's now easy to break out your frequently used components into self-contained classes, instantiate and return them from a Page.render_* method or even another Fragment.render_* method.


• url.URL improvements: Some new APIs make it easy to manage view state using query arguments rather than server-side sessions, which expire and aren't friendly with the back button.


• nevow.canvas: In the tradition of LivePage comes the experimental Canvas module, which gives you a server-side Python API for drawing lines, curves, and text in the browser, with control over line thickness, border color, and opacity. Canvas gives you a socket you can keep open as long as you need to draw animations in the client browser.


• XML Compliance: xmlfile and xmlstring now properly retain doctypes, comments, and xmlns attributes.


• IFoo(ctx) syntax: ctx.locate(IFoo) has been deprecated in favor of the more uniform adaption syntax IFoo(ctx).


• WSGI Implementation: This release contains a simple, incomplete WSGI application implementation. It is now possible to use Nevow to render CGI or WSGI pages without Twisted installed.


• guard improvements: guard.SessionWrapper now works properly when it is installed in locations other than the root.


• Chatola: Chatola, the LivePage demonstration application, has bloomed into a fully operational web-based chat server!


• inevow.IQ: The Interface for Querying is the beginnings of a DOM querying API. Currently, it contains only the patternGenerator, onePattern, and allPatterns APIs.

Visit the web site for more information about Nevow.

Canvas

Dave Hyatt has been writing about the features which have been added to WebKit to support the Dashboard in Tiger, and today he writes about a spookily familiar new feature -- the canvas.

I say it is spooky because a few weeks ago, I had the great idea to expose the Flash MX drawing API to Python on the server side using Nevow. It really was a great idea, too. In less than a week, I had a working implementation and a nice demo application, which I wrote about here.

The original idea was not to specifically have Flash glue, but to devise a server side api for drawing which would use whatever client side drawing technology was available. I originally thought of SVG as the non-flash example, but the WebKit canvas would be excellent glue to write a Nevow canvas implementation with. It would provide some things that Flash currently does not (if only on Safari), such as better integration with actual HTML.

Weird Weather

A few days ago, we woke up to 96 degree weather, with 80% humidity. This is a very rare thing for up north here. It wasn't stifling, although it was much nicer outside than in, so I decided to go for a little swim. It turned into more of a wade than a swim. Lake Superior always stays cold, no matter how warm it is out, and I couldn't bring myself to jump all the way in.

Within an hour, the temperature had dropped to 80 degrees. Within another hour, the temperature was 70. The change was so fast that I went from having all the windows and doors in the house open to closing them all.

Unfortunately, this phenomenon hasn't repeated itself. It has been overcast and in the 60s for a few days now. Today, I am going to drive down to Midland to stay with my Dad for a month, while Ari goes to California to help her parents move. It's amazing what a difference a four hour drive makes. We'll certainly be needing the air conditioning this summer.

Mr. Bald Eagle

A bald eagle lives somewhere around here. Arianna and I have seen him a few times. I saw him once last November, flying over the house on a blustery gust of wind. Ari saw him again in the middle of winter, along with my Dad, again flying directly over the house.

Today, Arianna spotted him again. He was flying low over the lake, being chased by a flock of seagulls. I ran downstairs and grabbed the binoculars, and went out on the breakwall to try to catch a glimpse of him. He was sitting on a sandbar a couple hundred yards north of the house, pecking at something on the ground and looking over his shoulder at the huge crowd of seagulls which were keeping their distance. I suspect he had caught a fish, and the seagulls were pretty pissed off about not getting any.

I wish we could have gotten a picture, but he was too far away. Hopefully Ari's next camera will be telephoto capable so we can get some nice shots of far-away stuff.

Debugging JavaScript

One of the things I hate about web development is the absolute pain in the ass that is JavaScript. It's not a terribly bad language in itself, with a decent anonymous function syntax and prototype-based object system (even if it's completely useless because there is no object database). But the fact that errors often pass silently and the debugging tools available are tedious and a pain to use makes developing with JS almost unbearable. I have often wished for the staple, the crutch of development in tool-poor environments, of the 'print' statement.

Here is how to configure Mozilla so that calls to 'dump()' will show up in the console:

Fire up Mozilla and type 'about:config' in the url bar.

Right click and choose 'New->Boolean'

Type 'browser.dom.window.dump.enabled'

Type 'true'

Start Mozilla from a shell, and you will finally be able to debug complex javascripts by printing things to the console using the 'dump' function.

By the way, to start Mozilla from the Terminal on Mac OS X, execute the binary:

'Mozilla.app/Contents/MacOS/mozilla-bin'

Dark Castle!

The entire Dark Castle, for your viewing pleasure.

Ahh, Dark Castle

Bill writes about both Return to Dark Castle and Zelda today.

There were a few NES games which consumed much of the time I should have been playing outside in my youth. Number one on that list was certainly the original Zelda. As Bill notes, Zelda is interesting historically mostly because of the influence it had on later games, not because of stellar game design. The stilted english translations and inherent limitations of the cartridge format made the game tough to play unless you had access to a walkthrough, which Nintendo Power did a good job of providing every month.

But the amount of time I spent playing Zelda was nothing compared to how much time I wasted playing Dark Castle. I remember it came on two 400k floppies, DC and DC Data, and it took me a while to start it up on the ol' 128 (upgraded to 512k, but no 'e'). Like many early side-scrolling platformers, every time you started the game you had to play from the beginning. I got pretty good at it; I could get the shield 100% of the time, fireball a lot of the time. I could usually make it through Dark Knight 1, and often the game was over in Dark Knight 2. I still remember the first time I beat it, and finally toppled the Dark Knight.

Dark Castle was one of those games that I wished had a level editor. Another game I played a lot around the same time was Pinball Construction Set, and I spent quite a lot of time imagining the levels I would design had a level editor been available for Dark Castle. I remember vividly one time when I sat in the waiting room at the dentist, drawing a Dark Castle level which was a combination of the slide into Trouble 3 and the swinging ropes in Trouble 2.

When Beyond Dark Castle came out, unfortunately the ol' machine wasn't powerful enough to play it. Once or twice a year we would go visit family friends in Lansing who had both an SE/30 and a Mac II (color!). It was the best thing in the world to be able to play Beyond Dark Castle on that SE/30. I never did get enough time to finish it, though.

openit

For a while now I have been using a very useful one-off script I wrote which I call 'openit'. You provide it with the dotted name of any python package or module, and it opens it in your editor. You don't have to worry whether the module is in the stdlib, in your sitepackages, or somewhere else on your pythonpath. I have been using it more and more recently, and the implementation is trivial, so I thought I would share it here:

#!/usr/bin/python

import os, sys

numargs = len(sys.argv)
if numargs == 1 or numargs > 2:
print """Usage: %s modulename""" % sys.argv[0]
sys.exit()

from twisted.python import reflect

obj = reflect.namedAny(sys.argv[1])

fileName = obj.__file__
if fileName[-1] == 'c':
fileName = fileName[:-1]

os.system("open %s" % fileName)

This implementation uses twisted.python because it is convenient, but the implementation of namedAny is short enough that you could really just copy it into the script if you wanted to. It also uses the Mac OS X specific 'open' command, so you will have had to tell the Finder which application you want to edit .py files. If you are on any other OS besides OS X, you can replace the open command with $EDITOR or whatever command you use to edit a file with your editor of choice.

So, how's the weather?

This is a test of using NetNewsWire to post to my blog.

Living back in Michigan has been really exhilarating for me. Every day it's like a new world outside. When we first arrived, it was beautifully warm and the leaves were frighteningly green, quite a change from the endless dreary brown grass of the hills of northern California at the end of summer. Then, fall, with it's insanity of color and endless raking. November was a wild reminder that we are alive and the universe is huge with some pretty crazy storms, power outages, and trees blowing over. Winter -- I think I will remember this winter forever. Being outside in the crisp, dead silence, flakes of frozen ice falling around you...

Then of course there were the snowmobiles. For some reason it seems to be the american way to not enjoy anything. If the air is clean and crisp and cold and silent, americans want to ride around on a noisy, smelly, hot machine. Perhaps it's because people feel powerless in their daily lives that they want to feel power over a machine, and over nature. Another downside of winter this year was that it was overcast for almost two months straight. I think we had maybe five or six sunny days during January and February.

But then, March. This year, it was sunny almost every single day in March. There is nothing quite like sitting in a comfy chair in the sun on a cold day in March. The view is breathtaking and it's always warm in the sun inside. At the end of March this year, we went to Washington DC for PyCon. At first, it was colder than Michigan, and there was no snow. But then at the end of the week, it was in the high 70s. It sure was a nice break from winter, but you know it's going to get hot and muggy there fast.

Yesterday, we had our first really big thunderstorm. Saturday we were in East Jordan and it was beautiful. Sunny, in the 70s at least, sky as clear as a bell. But Sunday the wind picked up and it started to rain on and off. Low, rumbling thunder accompanied far-off lightning flashes. The wind was gusty -- one minute it would be dead calm, the next the trees were flailing in the wind. As the storm moved over us it became completely still, and then something incredibly strange that I have never seen before happened. The water level in the lake dropped dramatically. I went out and walked around on the lake bottom, and Arianna got pictures. I had never seen anything like it before, and probably never will again.

Most people would probably think living in the middle of nowhere was boring. But for me this year, there has been something new and fascinating every single day.

Using TextEdit as your EDITOR

Anyone who has used Unix for a while knows about EDITOR. It's an environment variable you set to tell applications that want you to edit stuff what your preferred editor is. It's usually set to vi by default and it's easy enough to figure out how to set it to emacs or pico. But what if you want to set it to something like TextEdit or SubEtheaEdit?

You can't set it to /Applications/TextEdit.app because to the shell, that looks like a directory. You can't set it to "open -a TextEdit.app" because open will return immediately and the program which is asking you to edit something will think you didn't make any changes. The solution is to set it to the actual executable inside of the bundle:

/Applications/TextEdit.app/Contents/MacOS/TextEdit

If you run the actual executable inside the bundle from the shell, the OS will actually fire up a new copy of that application, blocking the shell until the application quits. Pretty weird, and useful in certain situations.

Setting your editor to TextEdit gives you the benefit of spell checking, services, etc, and generally feels nicer for someone who is more used to the Mac than to Unix.

Running scripts in pythonw from Xcode

The previous tip works fine if your Python script is just a command-line script that doesn't require any interaction with the WindowServer. However, if your script needs to be run with pythonw normally in order to interact with the user in a GUI fashion, it won't work. Xcode won't let you select /usr/bin/pythonw as the executable because it is just a shell script, not a binary executable. To work around this, use /bin/sh as the executable and pass pythonw and your python file as arguments to sh:

Choose Project->New Custom Executable
Type /bin/sh as the Executable Path
Double-click on the new executable
Click the plus button under Arguments and type:
/usr/bin/pythonw /path/to/your/script/here.py

Update: Bob has pointed out in a comment that you can use the executable that the pythonw script runs directly. I have tested it and it seems to work for me. Set the executable path to:

/System/Library/Frameworks/Python.framework/Versions/2.3/Resources/Python.app/Contents/MacOS/Python

And the arguments to the path of your script. Thanks Bob!

Running Python scripts inside Xcode

It's not terribly difficult to tell Xcode to launch a Python process and start your script, but it comes up now and again, so I thought I would spell out the steps required.

Choose File->New Project.
Choose Empty project.
Choose where to save it and what to name it, and click Finish.

Choose Project->New custom executable
Give the executable any name you want (I usually call it 'python') and *type in* the executable path to the python you want to run (/usr/bin/python or /usr/local/bin/python) and click Finish.
You can now use Debug->Run executable (Command-R) to get a Python prompt. Be warned that you can't send Control-D to this process, and clicking the stop task button rather harshly kills the process. This is the one drawback I have found to using this method.

Now, double-click on the 'python' executable you just created in the Executables list. You will be able to set the launch arguments, environment variables, working directory and more.

Press the plus button under Arguments, and type in the fully-qualified path of your main python script.

It is good if your script can run to completion, so hopefully it is just a script that does something and exits, or if it is an application which enters a mainloop and runs forever, it has some way of externally shutting it down. You can kill it with the stop button, but it's not a clean kill at all.

Have you met my Evil Twin?

Well, I finally did it. I re-implemented LivePage in Nevow. It was a piece of cake, really. The hardest part was coming up with the right design for how to allow programmers access to the client object which represents a user's browser. After talking to itamar about it at length, I finally realized that the cred 'mind' concept was the perfect solution. With very little code, a programmer can write a new realm which mediates for the entire application. It can then be the central point for all the clients to communicate with each other, such as in a chat application.

I wrote a simple example chat application called Chatola. You can get the source here:

http://soundfarmer.com/content/code/chatola/

Download all three files and run the tac file using:

twistd -noy chatola.tac

Currently, you will need to have Twisted (http://twistedmatrix.com) and a recent CVS checkout of Quotient (http://divmod.org) installed. Soon, nevow will be released as a standalone package.

CoreAudio from Python

Well, I've only been talking about doing it for 4 years now, but I finally got Python playing some buffers of audio using CoreAudio. It was about a day's worth of hacking, with a few hours on the side nailing down threading issues.

I put the code up here:

http://soundfarmer.com/content/code/coreaudio/

I also put a short description of what it is on the pythonmac wiki:

http://pythonmac.org/wiki/CoreAudio

CoreAudio calls you back in a thread, you see. A thread Python hasn't seen before. Python doesn't like that very much. This isn't the first time I have tackled this issue. CarbonEvents also call you back in a new thread you haven't seen yet. Two years ago when I wrote the CarbonEvent wrappers for Python, there were no convenient ways to deal with this situation. Before you are allowed to request the global interpreter lock, you must have a PyThreadState. You can create a new PyThreadState, but only if you have a PyInterpreterState. And if you are getting called back from a new thread that has never seen Python before, you are pretty much out of luck.
The solution I came up with for CarbonEvents, later on Windows talking to a stupid Asset Management system, and finally for CoreAudio, is to make an initialization call into my module which will eventually be generating callbacks into Python. In this call, I stash the current PyInterpreterState away in a C global. Then, when the new thread comes in and needs to make a PyThreadState you can use the stashed PyInterpreterState.
This time I was trying to use Pyrex to wrap the C calls I needed, partially because I thought it would be fast and easy and partially because I wanted to learn Pyrex. During the build-crash-debug cycle I kept thinking to myself there was an easier way that I had read about a while ago, and I was right: http://python.org/peps/pep-0311.html describes an implementation of an API specifically designed for this situation which is included in Python 2.3.
I'm happy to say it works quite well, and I was able to generate my sine wave in python code. I even went on later and used numarray to wrap the incoming and outgoing buffer for a speed boost.
My favorite thing about Python is that it is trivial for those who aren't afraid of C to get access to literally everything they need, if it's not possible already.