I'm just starting a project that's using JavaServer Pages (not Java Server Pages) aka JSP for doing "templates" -- i.e. making dynamic HTML content. The first thing I'm trying is really quite simple -- create an object and print out one of it's values.

Here's the relevant class:

public class Entry {
  String hour;

  public String getHour () {
    return this.hour;
  }
}

Java/JSP is big on setters and getters, that is, functions named a certain to get access to object attributes. There's plus and minuses to this, but from a conceptual level I thought it was quite clear that we are thinking "Entry objects have an attribute 'hour'", even if we have to use setHour/getHour to access values.

Now, in Cheetah, Django (and probably Rails, though I haven't looked) you'd access the attribute 'hour' in a template language as follows:

e.hour

Simple -- the last thing you want to be doing in a HTML template is dumping tons of programming language code into it, making it hard to read and maintain. Cheetah is quite clever about this: it looks at 'e' and does all sorts of introspection to figure out how to get 'hour', so you don't have to do that yourself.

So how do we do this in JSP (like in 2008)?

e.getHour()

Give me a break. Maybe there's another way to do this? If there's performance reasons to be wedded to this format (though I can't see it since everything is compiled) why not invent a new syntax like

e::hour

Here's a few notes I made while working my way through Django's tutuorial:

Installation

Installation of Django is easy. Just following the instructions on this page:

• http://www.djangoproject.com/download/

It can either be downloaded as a tarball or via SVN. We've chosen the tarball (0.92.2) option.

Nitpicking, the tarball URL doesn't end with the name of the tarball - it's a directory. This is only slightly annoying.

Our environment has as per-user Python installation, so we don't need to use "sudo" to install the Django packages

Testing (Part 1)

We are following the instructions son this page:

• http://www.djangoproject.com/documentation/0.96/tutorial01/

Running a development webserver

Django has options for running under mod_python, WSGI or using a standalone built in webserver. The built-in webserver is documented as not-for-production, but it's good enough to get going so we're going to play with that for now. Eventually we expect to use both the mod_python (because we have it here) and WSGI options (because it's both the way to go and the most efficient).

We immediately ran into issues running the webserver because it's not on the "localhost" - the browser said it couldn't find the server. In our environment we ssh to a linux server and access that from our desktop computer. After a little googling, the trick turns out to be adding a "IP:port" argument to the Python command:

python manage.py runserver 192.168.1.10:8000

Connecting to the DB

Next in the instructions is connecting to the DB. Again, we have issues here - no fault of Django - because of the uniqueness of the local environment. Like Python we run MySQL on a per-user basis so we need to be able to specify a fairly unique setup with a TCP/IP and UNIX domain socket. It's not clear initially how to specify the path for the UNIX domain socket but fortunately the stack traceback and clearly written code come to our rescue here: you use the DATABASE_HOST and the code looks to see if it's a path.

Creating the DBs

The ‘syncdb' asked if I'd like to create a super user for the DB. I said no and everything seems to be working - a bunch of tables show up in MySQL as promised.

Creating an App

An "app" is something that does something or something like that. The command works as promised.

Create the Model and DB tables

Works as promised. There appears to be some underlying cleverness happening as (for example) the "sqlclear" command knows whether the tables are their or not.

The "sqlinitialdata" refers you to another command; this is probably a mismatch between the code and the tutorial.

Interactive Shell

This is cute - running "python manage.py shell" will drop you in the environment that the webserver sees so you can do stuff on the command line to see how it will work.

Stylistically, I differ from Django in that I (almost) never do anything except ‘import x', as I prefer to use the dotpath and not have to guess when I'm skimming code as to where functions are coming from.

As an aside, it would be cool if Python examples did the ‘>>>' as an image or as an CSS trick so they don't get copied when cut-and-pasting examples.

Testing (Part 2)

And we're on to page 2 of the tutorial:

• http://www.djangoproject.com/documentation/tutorial02/

Admin Interface

And we get our first "uh oh" - we didn't create a "superuser account" on way back above, well, just because. Fortunately, Lord Google knows all and we quickly find out that you can run Python commands to do this and we're in business:

$ python manage.py shell
>>> from django.contrib.auth.create_superuser import createsuperuser
>>> createsuperuser()

Thanks to here for the tip (which also looks like interesting reading):

• http://blog.disqus.net/2007/03/11/a-django-primer/

Admining the App

Nice - add an inner class declaration to App "Poll" and it now shows up in the Admin interface and we can do stuff with it. Neato.

I've also just discovered that I don't need to stop and restart the app - it's using reload().

Adding __str__ to the model makes the Admin app useful; if you do something wrong the HTML function is really quite clear.

You can change the way the Admin interface displays Poll by adding a ‘fields' declaration. This is a little bizarre - it's a tuple, of tuples with a dictionary inside. Why not a list of dictionaries?

Bug: if you use the ‘classes/collapse' option of the admin interface and there's a bug while saving an item while the item is collapsed, there's no indication of where the error is.

Changing Templates

The instructions are a little confusing - pay attention - but it works as advertised. There's a command called ‘adminindex' which dumps out template code but I'm somewhat confused by it and I wish there was a little more detail here.

Testing (Part 3)

And we're on to page 2 of the tutorial.

• http://www.djangoproject.com/documentation/tutorial03/

Design your URLs

Django converts URLs coming from user requests into actions by:

• looking at ROOT_URLCONF in settings.py
• loading mysite/urls.py (as defined in the settings)
• sequentially looking at regular expressions
• loading a module - typically a view - and calling a function defined by the matching regular expression. This is expressed as a dotpath - e.g. ‘x.y.z' will load module ‘x.y' and call function ‘z'
  • the function is called with a request object and all the named groups from the regular expression - clever
• also note that:
  
  • you cannot filter on the hostname in the URL (problem?)
  • regular expressions are compiled (and thus are very fast)

The code samples work as advertised. In a later section we'll learn that you can move all the URLs into the app so that you don't have to (a) put all the URL information at the project level (b) decouple the base path of the URL being used.

Write views that do something

This section explains how to returning meaningful results building on the knowledge in section 2.

• the template loading system rocks
• the shortcuts are very handy (i.e. lots of common actions are compressed into single function calls)

Playing with forms (Part 4)

This section is a mess. I'll probably retackle it.

Via Noah Gift at O'Reilly, I've just found out about pyinotify a filesystem event listener (for Linux kernels after 2.6.13). BlogMatrix (and hence Onaswarm, etc.) has great interest in this sort of thing since a lot of our code is built around queuing information through the filesystem.

The main issue I have is that, well, it's a little too complicated for what I would consider to be the common use case. After a little thinking, I've made a Pythonic iterator wrapper which I'm going to submit to Sébastien Martini for inclusion in the code base so that people can write code like this:

import pyinotify
for event in pyinotify.watch("/tmp", "/var/tmp", create = True, delete = True):
   print event

I've attached the code to this posting if anyone would like to give it a try. Hmmm ... the attachment isn't working properly, try this.

PyLinda is a Linda-like implementation (and JavaSpaces-like, which I'm more familiar with) for Python. The BlogMatrix Platform does a lot of it's work by one process dropping stuff on queues and other processes picking that stuff up.

Just saying.

Here's an article on creating a python apps and libraries for distribution using distutils. And from this, here's an article on setuptools, an enhanced version that's been making our lives hell in terms of making totally self contained distributions.

I think it may be doable, but it's quite a time sink. We're leave all our mods in the code base for another day but the core issue is that the Windows "shortcut" is a third-class citizen compared to UNIX's symbolic link. And we use a lot of symbolic links, especially when constructing the "htdocs" directory for Apache. Now, in Python we can get around this (not efficiently, but good enough) but Apache just doesn't have the proper code to do this. The other option would be to run it all under Cygwin, except mod_python isn't really supported and that's a whole other can of worms.

I'm currently linuxless, but since I have access to a Windows XP laptop I thought I'd try a few little experiments. I'm probably not going to pursue this to far, as at this stage in the game it doesn't make a lot of sense to bring another, radically different, platform into the mix.

Running mod_python on Windows (XP) is actually quite simple:

• install Apache (Windows MSI Binary 2.0.X)
  
• install Python; I like ActiveState's version from my Windows versions (Windows MSI 2.4.X)
• install mod_python from here; make sure you pick the correct matching version for Apache and Python!

On Windows I find it fairly useful to install everything the default way. Configuring this is simplicity:

• edit httpd.conf (there's probably a start menu option for this)
  
• add "LoadModule python_module modules/mod_python.so" in the LoadModules section
• restart Apache
• test (the Directory version works out of the box with the appropriate path adjustment)

BlogMatrix uses a lot of shell scripts; mixing this together with Windows Python (not Cygwin Python) is going to be a challenge, as the way paths are handled is different.

eWeek has an article on how Python is being used in enterprise systems:

ITA Software is using the Python language to empower its airline reservation system.

Many computer language purists say that languages such as Java, C++ or C should be used for enterprise applications. However, ITA, a Cambridge, Mass., provider of airline IT software and services, is proving that dynamic languages such as Python can be rock-solid for enterprise work.

[...] Much of the code ITA employs is written in Python, despite skepticism by some that dynamic languages are not ready for prime time. However, people such as Guido van Rossum, the creator of Python, point to the successful use of the language at places such as Google and YouTube, which endure enterprise-scale traffic on a daily basis.

[...] Each server requires a unique set of data feeds, and all the data feeds must be monitored to ensure the data arrives when expected and gets loaded as required, all while ensuring that system performance is not impacted by these data management tasks. ITA uses Python to automate all these tasks to make its systems reliable, easy to maintain and easy to update as its customers' needs change, Kelley said.

Moreover, ITA uses a Python application to monitor its heterogeneous production environment that contains hundreds of servers.

[...] Moreover, when ITA hires new people, the company likes to hire those with Python experience "because we've had a lot of luck with Python people having a lot of core problem-solving and system-building ability," Kelley said.

He said it is pretty easy to find Java or C programmers who are good at line coding but not generally good at problem solving. "It's much more unusual for us to find people who can analyze a problem domain and then implement a solution where they cross a bunch of problem domains," Kelley said. 

BlogMatrix is coded in Python.

Last April, we demoed some of the structured blogging capabilities of BlogMatrix. Part of the demo involved posting an event and having it show up in Google Calendar. We've recently decided to make that code part of the standard "base" installation under a system called continuous export -- whenever you post something, it will show up on other systems as appropiate.

Anyway, I couldn't get it to work so I contacted the Google Calendar Data API support group where a gentleman named Ryan Boyd helped me out. I didn't quite believe his last answer so I decided make a standalone test case that could be safely shared. Of course, after I finished writing it the test case worked perfectly so I ported back that test case code and everything's working just fine.

As a public service, I've included the test case code here. If you're trying to do PUT/POST using Python's urllib2, need to login to Google Calendar, or whatever, this code is well documented internally, is easy to follow, and it works.

This will demo quite nicely -- you can go to upcoming or any other hCalendar supporting tool, import the event and have it published on your Google Calendar. Neat and useful.

Outstanding items:

• making sure when an item is deleted in BlogMatrix, it's deleted on Google Calendar
• implementing Google Account Authentication
• make Google Calendar update (in the user's browser) when the item is added. Currently, one has to press the browser's refresh button which is suboptimal

We have a really neat demo of importing microformats into BlogMatrix. However, we can't show you. Why?

• Apache compiles with its own version of Expat
• mod_python compiles against Apache
• Python compiles with its own version of Expat
• these two different versions of Expat are not compatible
• therefore, all libraries that parse XML cause Apache/mod_python to core dump

We'll have to recompile Apache against Python's Expat and go from there. This will cause a short period of downtime, so we won't be doing this for a little while yet.

All the pages you see here are produced by the Cheetah template language. We're pretty happy with it, though it looks to some degree that the world is standardizing around Django's competing package.

There is one lovely feature of Django that I'd like to see in Cheetah -- the ability to know where you are in a for loop:

forloop.counter	The current iteration of the loop (1-indexed)
forloop.counter0	The current iteration of the loop (0-indexed)
forloop.revcounter	The number of iterations from the end of the loop (1-indexed)
forloop.revcounter0	The number of iterations from the end of the loop (0-indexed)
forloop.first	True if this is the first time through the loop
forloop.last	True if this is the last time through the loop
forloop.parentloop	For nested loops, this is the loop "above" the current one

Cheetah requires the decidely unlovely formulation of this ...

#set $sep = '' 
#for $name in $names 
$sep$name 
#set $sep = ', ' 
#end for

... to make a comma seperated list of names (I know about the other way for trivial lists, this is an example for illustration). I'd much rather do:

#for $name in $names 
$name 
#if $forloop.is_last then "" #else ","#
#end for

Or even (thinking more):

#for $name in $names 
$name
$forloop.if_not_last(",") 
#end for 

At some point we may try template translation tricks, not only (or possibly) to move Cheetah to Django templates (if they are seperable from the platform) but also to covert MovableType and Blogger templates to our internal format.

Python 2.5 is now available. Lots of neat things I'd like to use (especially try/except/finally and with).

We'll probably roll over BlogMatrix (Python 2.4) in about 6-9 months, once the libraries catch up and stabilize.

Here are some useful sites and pages about OpenID:

• OpenID -- the official home page
• spec
  
• OpenID Enabled -- for developers:
• Python
• Ruby
• Perl
• PHP
• DotNet
• Java
• Wikipedia -- this page could use some work
• Identity Servers:
• LiveJournal.com
• MyOpenID.com
• TypeKey
• VeriSign's PIP
• Many more...

IronPython 1.0 is available:

IronPython is a new implementation of the Python programming language running on .NET. It supports an interactive console with fully dynamic compilation. It is well integrated with the rest of the .NET Framework and makes all .NET libraries easily available to Python programmers, while maintaining full compatibility with the Python language.

Jim Hugunin explains the story IronPython here:

I started work on IronPython almost 3 years ago.  My initial motivation for the project was to understand all of the reports that I read on the web claiming that the Common Language Runtime (CLR) was a terrible platform for Python and other dynamic languages.  I was surprised to read these reports because I knew that the JVM was an acceptable platform for these languages.  About 9 years ago I’d built an implementation of Python that ran on the JVM originally called JPython and later shortened to Jython.  This implementation ran a little slower than the native C-based implementation of Python (CPython), but it was easily fast enough and stable enough for production use – testified to by the large number of Java projects that incorporate Jython today.

I wanted to understand how Microsoft could have screwed up so badly that the CLR was a worse platform for dynamic languages than the JVM.  My plan was to take a couple of weeks to build a prototype implementation of Python on the CLR and then to use that work to write a short pithy article called, "Why the CLR is a terrible platform for dynamic languages".  My plans quickly changed as I worked on the prototype, because I found that Python could run extremely well on the CLR – in many cases noticeably faster than the C-based implementation.  For the standard pystone benchmark, IronPython on the CLR was about 1.7x faster than the C-based implementation.

The more time I spent working on IronPython and with the CLR, the more excited I became about its potential to finally deliver on the vision of a single common platform for a broad range of languages.  At that same time, I was invited to come out to Microsoft to present IronPython and to talk with members of the CLR team about technical issues that I was running into.  I had a great time that day working through these issues with a group of really smart people who all had a deep understanding of virtual machines and language implementation.  After much reflection, I decided to join the CLR team at Microsoft where I could work with the platform to make it an even better target for dynamic languages and be able to have interesting technical discussions like that every day.

(Plain-old) Python already works quite well with the Microsoft Platform. However, I expect the best work Microsoft is doing these days revolves around .NET, so then next time I look to do Windows work in Python, I'm definitely going to have a look at IronPython.

Yahoo has added a Python Developer Center to their Developer Network (tip: Slashdot):

Welcome to the Python Developer Center

Python is a dynamic object-oriented programming language that can be used for many kinds of software development. It offers strong support for integration with other languages and tools, comes with extensive standard libraries, and can be learned in a few days. This site is your source for information about using Python with Yahoo! Web Services APIs. Here you'll find:

• HOWTO Articles to help you understand our technologies and how you can use them better with Python.
• Useful Resources on the web where you can find source code and helpful tools.
• Educational sites where you can learn how to program using Python.
• Community Resources where you can join our mailing list and discuss the Yahoo! APIs with us and with other Python developers.

Pretty cool. We originally were using Yahoo Maps when we started adding mapping extensions to the BlogMatrix Platform. However, the maturity wasn't there and due to client demands we migrated to Google. That's not to say we didn't like Yahoo; they're a hell of a lot easier to get an answer out of and they have lots of other great APIs, services and libraries -- such as the Javascript Yahoo UI library, which this site does use -- available.

Update: g*dd*mn*t. no RSS feed. Get it together guys.

Task:

• Improve RDFLib's N3 serializer to make nicer looking output taking advantage of N3's feature set

Details:

• RDFLib's N3 serializer is pretty crappy, producing output like the first attached file. We'd like to see something nicer, with N3 looking like the second attached file.

Required Features:

• @prefix against URIs wherever possible
  
• resonable nesting of subjects as objects as appropriate
• triple-quoted strings as appropriate
• shorthands: a, =, =>
• properly ordered lists (this may be tricky)
• ^^type notation

Notes:

• please contact us if you want to undertake this project
• this must work properly with Unicode-encoded data
• your code must be given to the RDFLib project without encumerance AND released under the Python License.

Bounty Value (Closed):

• $USD 75 via PayPal
• +$USD 25 Danny Ayers

Task:

• create a plug-in for the Universal Feed Parser to read the MovableType Export Format

Input:

• is a stream of bytes 
  
• has an (optional) input charset, defaulting to latin-1
• is in the MovableType Export Format
• may contain comment entries
  

Output:

• a populated Universal Feed Parser object
• oldest entry first, newest last
• comments to immediately follow the appropriate entry in same date order
• comments to be identified using this and somehow marked in the entry
• categories to be identified using this
• all strings to be Unicode and properly decoded from the input source
• convert all unknown keys to lowercase, spaces to underscore and prefix with "mt_"

Testing:

• two input data sets will be provided (see attached documents)

Notes:

• please contact us if you want to undertake this project
• the code should be able to be "plugged in" to the UFP (there's no guarentee they'll add it to source base, so ideally it could be just call the right places in UFP)
• your code must be given to the UFP project without encumerance AND released under the Python License
  
• this spec was revised to make it a UFP "plugin"
• if this works out, there's probably more in the works for WordPress, GreyMatter, ...

Bounty Value (closed):

• USD $50 via PayPal