Boulder Sprint: Adding Oracle support to Django

We had a great turnout on Saturday for the first Boulder Sprint held by the Front Range Pythoneers. Our goal was to provide production-level support of Oracle in Django 1.0. I'm glad to report that we made a strong start on this goal.

Django developer Jacob Kaplan-Moss flew out to Boulder from Lawrence, Kansas, providing us both leadership and guidance into the Django internals. (Next time I hope Jacob doesn't have to fly out the next day on a 6 AM flight!) From Array Biopharma, we had five developers: Ian Kelly, Matt Boersma, Matt Drew, Michelle Cyr, and Mitch Smith. Eric Dobbs, of Bivio, contributed both the space and his seasoned Python skills. And there was me (Jim Baker). Thanks to everyone for your hard work!

We worked on a number of key issues for supporting Oracle:

• Perhaps most important, Jacob split out Oracle-specific functionality into the Oracle backend, allowing for more modularity. Django uses quite portable code, in conjunction with the Python DB2 API, but Oracle has its peculiarities. Being pragmatic, we had to work through that.
• Mapping Django's TextFields to Oracle's CLOBs, not LONGs, which pretty much are deprecated. (Remember Django's origin, we certainly need support for text!) However, supporting CLOBs required some changes: no buffering in the Python layer, just iterate directly over the cursor; explicitly read in data from the LOB reference; prepare the OCI by giving cx_Oracle explicit type information (also necessary for timestamps with greater than one second precision).
• Pagination queries. Django's ORM grew out of supporting PostgreSQL, which has OFFSET and LIMIT clauses, useful for the pagination queries often seen in stateless web apps. Oracle actually has quite good support for this type of queries but this fact is not well-known. And frankly it's a bit clumsy to use, requiring doubly nested subqueries. See Oracle guru Tom Kyte's article in Oracle magazine for more details. I made some progress on this front, but I still need to integrate it into the new django.db.backends.oracle.OracleQuerySet class added by Jacob.
• Test schema support. Oracle uses the concept of "user schema" where other databases might use "database". There's a bit of trickiness in working appropriately with this, especially if there are tablespaces being set up for this test. Eric took the lead on this.
• Mitch Smith wrote two gnarly Oracle-specific queries that have almost got
  introspection and Django's "inspectdb" command working correctly.
• Part of our goal was to get all existing tests to pass from runtests.py, and we're about 70% there.
• Array Biopharma now has their test web app, donuts, running.
  

And this is just what I saw from my side of the conference room table! We do have a photostream for the Boulder Sprint. Check out Matt B. and Michelle contemplating at the Oracle. Or Matt B. and Ian asking rhetorically, "How can we screw up a 3-line function?" There is also a wiki.

For the moment, until we get the work integrated in the Django trunk, you can checkout the Boulder Sprint branch here with Subversion:

svn co http://code.djangoproject.com/svn/django/branches/boulder-oracle-sprint