A brief history of Python Distutils

When I started using Python in September 1998, I pretty quickly noticed there was a problem in the ecosystem: every library that included an extension module had its own little Makefile that cribbed from Python's own Makefile, which was (and still is) installed alongside the standard library. Libraries that did not include extensions generally had a README file that said, "copy foo.py to a directory on sys.path" and left it at that. The audience was pretty clearly fellow Python developers who wanted to use the library and knew exactly what sys.path was. Worst, anyone wanting to build extension modules on Windows was on their own. This was no secret; everyone in the community at the time knew it was a problem, but everybody was too busy with their own stuff to tackle it. (Or they had the sense to stay well away from it.)

So, in a nutshell, I started the distutils project. Well, OK, really I scheduled a session at the 1998 International Python Conference (the precursor to PyCon) called "Building Extensions Considered Painful", with the goal of figuring out what we would do about it. A number of people much smarter and more experienced than me were there, but I only remember Greg Stein and Barry Warsaw. My recollection of what happened in that conference session was:

What I don't recall is how I got roped into writing most of the code. It's entirely possible that I volunteered to do it and nobody stopped me.

About those unit tests

I gather that the lack of unit tests is a frequent complaint for anyone who tries hacking on the distutils. I thought the reason for this was perfectly obvious until I had to explain it to Nick Coghlan in person at PyCon 2013. It's quite simple: distutils predates unit testing. Or at least, unit testing in Python. I'm sure Kent Beck had something working in Smalltalk by then, but I was quite unaware of it.

Anyways, it's pretty clear by looking at history:

However, that does not excuse the lack of automated functional tests. For that, I have no excuse. I vaguely recall either 1) I didn't know how to do it, or 2) I didn't want to write a dedicated testing framework just for distutils. If I was doing it today, I could probably figure something out. But I know now more about programming in general, and about automated testing in particular, than I did then.

Author: Greg Ward
Published on: Mar 28, 2013, 6:08:42 PM
Permalink - Source code