I made my program work, and have to admit that it would have taken a lot longer in any other language (due to either intrinsic or logistical issues with said languages). Though I am still grumpy about codeset conversion:
>>> f = open("/tmp/foo1", "w")
>>> fb = open("/tmp/foo2", "wb")
>>> s = u'\N{ARABIC LETTER KHAH}'
>>> f.write(s)
Traceback (most recent call last):
File "<stdin>", line 1, in ?
UnicodeEncodeError: 'ascii' codec can't encode character u'\u062e' in position 0: ordinal not in range(128)
>>> fb.write(s)
Traceback (most recent call last):
File "<stdin>", line 1, in ?
UnicodeEncodeError: 'ascii' codec can't encode character u'\u062e' in position 0: ordinal not in range(128)
On another note, interesting to read Nat's post on software design. Basically the same contrast drawn in The Inmates are Running the Asylum, between design-driven and feature-driven development.
The complexity that comes in is that feature-driven development exists for real reasons. Two big ones are that people use feature matrixes to decide what to buy, and customers (and users) really do get angry in the face of regressions and changes, even small ones.
Balancing this properly is very, very hard. You could explain the history of GNOME (and probably many other projects) in terms of trying to get the balance right.
Fighting with Python. Has anyone else noticed that it's really hard to figure out how to use the standard library? There aren't type declarations, the API conventions are all over the map, and the docs are often pretty uninformative.
But the thing that really has me upset is this. By default, the standard Python "print" statement will 1) convert to ASCII and 2) throw a fatal exception if the conversion fails. Both 1) and 2) are broken beyond words. On top of that, apparently the default encoding will change from ASCII to locale encoding if stdout isatty(), and not if you say redirect to a file; or something bizarre like that. No! No! No! Cluebat!
To fix it you have to do this:
sys.stdout = codecs.getwriter("utf-8")(sys.__stdout__)
Which of course is impossible to figure out from the docs, though
Google turned it up.
Before stumbling on this I spent a lot of time trying to convert _parse_response() in xmlrpclib.py to handle invalid UTF-8 in the stream. The API docs sound promising (with ignore/replace/xmlreplacecharrefs/etc. error handlers), but heck if I can find the right magic.
Reading this thread on a kernelspace keyring, which I don't pretend to know a lot about (I was just curious from seeing the name of the patch in LWN, and found the thread via google).
However, the thread shows how differently some kernel developers and userspace developers are still thinking about problems. Linus says: "I'm just looking at how _well_ /sbin/hotplug has worked out. I believe that it would have been a disaster done with a binary messaging setup." What??? We didn't get any decent UI for hardware working until we had a binary messaging setup! Of course I realize that a script callout can send the binary message, and so that's fine, but we really do need the real messaging.
Looks like this thread also has the good old "let's have some horrible /proc thing instead of typesafe ABI-stable interfaces with well-defined error handling." Anyhow, don't know how that patch ended up when it was included, so maybe it all turned out well.
Someday I need to write down why I feel grumpy and hostile toward text-based "API", whether we're talking about initscripts, tcl, or /proc. It has to do with things such as 1) reporting errors, 2) ability to write event-driven code, 3) ability to tell when your ABI changes, 4) performance, 5) ability to have multiple threads/processes involved sanely, 6) ability to extend an interface while keeping back compat, 7) ability to keep UI code completely separate from "engine" code, whether "UI" means tty or X, 8) keeping data and configuration strongly separated from code, and 9) probably some other things. Deserves some kind of long essay.
Don't get me wrong, hooks that call out to scripts can be handy. But it's nice if the hooks are invoked by a larger system that's a bit more deterministic and well-defined than scripts usually are.