I got a bunch of really helpful responses from my previous entry and I'm pretty sure that my problems were two-fold: my code was missing some stuff (the code I posted and what I actually had were different--go figure) and my testing program had a bug (or two or three).

Anyhow, my knowledge of file locking on Linux (and possibly other Unixs) is pretty abundant now (or so I think). This helped to fix a problem I had at work as well (not the file locking part, but the thing I needed file locking for helped me at work).

Python has the fcntl module which has a flock and lockf. I'm not entirely sure what the difference is, but there are man pages on both functions as the Python versions call their C counter-parts.

There's a good article on file locking that might be located at /usr/src/linux/Documentation/mandatory.txt (though it wasn't on my system--a Google search helped me out) that's pretty interesting. Also, there's explanations of file locking between processes and the issues involved in Advanced Programming in the Unix Environment and Linux Application Development.

My specific issue was with Exim and PINE. Both have documentation as to how they lock mbox files before playing with them--things a Google search reveals without much effort.

Anyhow, I'm pretty sure my code works now. I'll do some more sophisticated testing this weekend to make sure everything is kosher.

Thanks to Chris, Jason, John and Lance for their help.

AMAZON::0201308215::Linux Application Development AMAZON::0201563177::Advanced Programming in the Unix Environment

I wrote my own web-mail client (bluemail) because I wanted a web-mail client that would work along-side PINE and figured I'd just roll my own in Python. I work on it from time to time--mostly when someone requests new features. Right now I'm trying to implement deleting of items in a given folder.

The problem I'm having is a file locking issue. When the user clicks on the INBOX link, it opens up their inbox file (an mbox file in /var/spool/mail/) as the user. That works super. I parse out the mbox and display a bunch of email one-liners. The problem comes in that I can't seem to open the file as r+, lock it, parse it into email, change some of the email (whether by deleting the email from the mbox file or changing the attributes of the email [i.e. marking the email as having been answered]), then write the changed file back to disk without the possibility of having another process (our MTA, for example) slipping in and changing the file beneath me.

Right now, I open the file like this:

   import fcntl

   f = open("/some/file", "r+")
   fcntl.flock(f, fcntl.LOCK_EX)

That should work fine (as far as my research has told me). I did notice some places that say that LOCK_EX only works when you open the file in "a" or "w" mode, but I've got it opened as "r+" which means I'm going to update it later, so I'm puzzled.

Anyhow, using the code above doesn't seem to lock the file. I can run the above code thereby "locking" the file and then in another process open the file, read the file, and write to it. That indicates to me that it's not really locked.

Any ideas how I can resolve this? I need to be able to open a file for reading and updating and lock it so that other processes can't change the file between my read and update cycles. I'm running the code on a Debian GNU/Linux box with Python 2.3.

I think I may release 1.0.1 this weekend depending on what else is going on life-wise. I tossed a feeler email out to the dev list in regards to what else should be fixed for 1.0.1, but didn't get anything back--which isn't surprising as people are pretty busy these days.

So... to do a release or not--that is the question. Is there anyone reading this blog who has things that they'd like to see for a very minor fix-it point release that's doable by one man who only has a bit of free time (unless they're interested in sending in a patch)?

Last night I did a lot of work on my registry plugin. I'm completely re-writing it from scratch--how it works, what it does, so on and so forth.

One of the things I never bothered to do was separate my blog (what you're looking at now) from my development. So whenever I make a mistake and create a bug, my whole blog goes down. Sometimes, in the case of last night, interesting things happen. I was playing with FileEntry and the generation of the absolute_path which had the effect of mucking up all the categories and permalinks on my site.

Anyhow, so things go up and come down pretty often on this blog since it's very very bleeding edge in terms of the source code and I'm often editing things live.

Having said that, I'm almost done the new registry plugin. I'll install it on Planet PyBlosxom soon and overhaul all the existing registry entries. This version of the registry plugin will make it easier to do a flavour registry as well. It's also a heck of a lot easier to set up and maintain.

I've been asked a couple of times for the templates for the HTML flavour of my blog. So here is a tar.gz file of them and the stylesheet.

A warning to folks who download that file: the templates and stylesheet aren't wildly pretty. I'm not a web-designer--I just pretend to be one.

I'm on the mailing lists for OZ and OE because I own a Sharp Zaurus 5600 and at some point I'll have some time to play with it enough to get it to the point where it helps me in my daily life. I haven't had any time to devote to it in the last four months though, which is unfortunate.

Anyhow, someone on the openzaurus-users mailing list complained that there's no support for the 5600 to which Michael (one of the three and a half people who worked on OZ) responded, expressing a lot of frustration, then said this (and I'm quoting him slightly out of context):

   And just to prevent misunderstandings, I don't want to be thanked for
   all the work... I don't want anything at all except participation. I
   want to work in a team bringing the community forward - I want people to
   realize, they're not helping _us_ to produce a great handheld operating
   system, they're helping _themselves_ and by doing that, helping each
   other. After all, this is what I thought was open source.

On all of the projects that I've been a part of or led, that's exactly what I hoped and prayed would happen. I have a couple of projects right now where there are people interested, but nothing is happening because I'm not driving it [1]. Those projects will die because I'm just one man--I can't drive all projects I'm involved in unless I quit my job (which is the only income stream I have).

I'm not sure that's what open source is. But I wish more community sprung around open source projects. My experience has been that people just want the features they want. They're demand-oriented consumers and it boggles their minds that I haven't coded up their favorite feature. It's so ego-centric. There's a big world out there! 6.5 billion different lists of favorite features!

[1] I'm not even a great driver--I'm ok and I get things moving forward, but I certainly lack the finesse that I've seen other project drivers use. For example, Bruce Perens guiding UserLinux is riveting. I suspect that I drive through sheer force of will rather than sheer leadership--which doesn't really do anyone much good.

I've noticed while working on PyBlosxom that bugs can literally be reported anywhere. Not only that, but people get annoyed (really annoyed) when you don't respond to their bug report which was reported using a method that I never would have dreamed of. We've had reports in all the following places:

• on the pyblosxom-users mailing list (I prefer this one)
• on the pyblosxom-devel mailing list (this one is great too)
• in a comment on the PyBlosxom main site for an article that has to do with a release
• in a comment on the PyBlosxom main site for an article that has nothing to do with a release
• in a comment on some other site (mine, Planet PyBlosxom, ...) in an article that has nothing to do with a release
• in a blog entry on someone else's site--someone I've never previously had contact with and whose blog isn't connected to anyone else's blog
• in the PyBlosxom documentation wiki
• in some other wiki that I've never previously seen
• on IRC in the #pybloscom channel (these are cool, but I'll never hear about the bug unless I'm specifically online which is not usually the case)
• sent in email to me or someone else
• added to the bug tracker on our SourceForge site

It takes a huge amount of time to check all these places--many of which come and go. It really puzzles me why people don't try to be more social and send bug reports via channels that other projects typically use (bug trackers, mailing lists, and IRC). I've been trying to coerce people into a few standard channels, but it's really hard on a project like this and there's a lot of resistance.

Bottom line is that this kind of thing really affects the project's quality. There are bugs out there that I will never trip over that someone else did but because they chose to whisper it into their pillow at night rather than tell someone about it, the bug won't get fixed and will trip someone else up.

I thought blogs were supposed to be social tools to enhance communities and promote discussion.

Anyhow, we can only do the best that we can do.

We've cleaned up a lot code and documentation and fixed a lot of bugs. We've stabilized the API and added static rendering. In short, this should be a pretty solid 1.0 milestone for PyBlosxom.

I encourage people to upgrade. When you untar the distribution tarball, read through INSTALL and README. These files will walk you through installation and upgrading from 0.9.

Download it at https://sourceforge.net/project/showfiles.php?group_id=67445.

Personally, I'm glad we finally pushed out 1.0. I'm not sure it's a perfect release, but it's good enough for a 1.0 milestone and I feel good about releasing it at this point. It's hard to push a project through to a release when there's not much feedback from testing and usage of previous alphas and release candidates. I think at some point you just have to bite the bullet and push it out and see how it does.

Cheers to the PyBlosxom development crew!

I'm releasing pyblosxom 1.0 RC1 right now. I overhauled the callback documentation in the ReadMeForPlugins file and read through other documentation and I think we're good enough for a 1.0 release now.

The RC1 is at http://bluesock.org/~willg/dev/pyblosxom/pyblosxom-1.0-RC1.tar.gz.

Feel free to download it and test it out--it's very much like alpha1 with a few differences:

• I threw in a logdir property into config.py and adjusted plugins that use logfiles to use it
• Documentation fixes

That's about it, I think.

In the tar.gz file is an INSTALL file which details installing PyBlosxom. There is also a README which talks about differences between 0.9 and 1.0 (which will help you upgrade an existing PyBlosxom installation).

Let me know what you think. I plan on doing a 1.0 release in the next few days. Probably Thursday.

We're working to put out PyBlosxom 1.0 which should be coming out very soon now. I'm in the process of overhauling all the documentation and doing some minor tweaks.

If you have comments/feedback/problems, toss me a comment and express them here.