File locking in Python--help! (part 2)

, | Tweet this

I got a bunch of really helpful responses from my previous entry and I'm pretty sure that my problems were two-fold: my code was missing some stuff (the code I posted and what I actually had were different--go figure) and my testing program had a bug (or two or three).

Anyhow, my knowledge of file locking on Linux (and possibly other Unixs) is pretty abundant now (or so I think). This helped to fix a problem I had at work as well (not the file locking part, but the thing I needed file locking for helped me at work).

Python has the fcntl module which has a flock and lockf. I'm not entirely sure what the difference is, but there are man pages on both functions as the Python versions call their C counter-parts.

There's a good article on file locking that might be located at /usr/src/linux/Documentation/mandatory.txt (though it wasn't on my system--a Google search helped me out) that's pretty interesting. Also, there's explanations of file locking between processes and the issues involved in Advanced Programming in the Unix Environment and Linux Application Development.

My specific issue was with Exim and PINE. Both have documentation as to how they lock mbox files before playing with them--things a Google search reveals without much effort.

Anyhow, I'm pretty sure my code works now. I'll do some more sophisticated testing this weekend to make sure everything is kosher.

Thanks to Chris, Jason, John and Lance for their help.

AMAZON::0201308215::Linux Application Development AMAZON::0201563177::Advanced Programming in the Unix Environment

File locking in Python--help!

, | Tweet this

I wrote my own web-mail client (bluemail) because I wanted a web-mail client that would work along-side PINE and figured I'd just roll my own in Python. I work on it from time to time--mostly when someone requests new features. Right now I'm trying to implement deleting of items in a given folder.

The problem I'm having is a file locking issue. When the user clicks on the INBOX link, it opens up their inbox file (an mbox file in /var/spool/mail/) as the user. That works super. I parse out the mbox and display a bunch of email one-liners. The problem comes in that I can't seem to open the file as r+, lock it, parse it into email, change some of the email (whether by deleting the email from the mbox file or changing the attributes of the email [i.e. marking the email as having been answered]), then write the changed file back to disk without the possibility of having another process (our MTA, for example) slipping in and changing the file beneath me.

Right now, I open the file like this:

   import fcntl

   f = open("/some/file", "r+")
   fcntl.flock(f, fcntl.LOCK_EX)

That should work fine (as far as my research has told me). I did notice some places that say that LOCK_EX only works when you open the file in "a" or "w" mode, but I've got it opened as "r+" which means I'm going to update it later, so I'm puzzled.

Anyhow, using the code above doesn't seem to lock the file. I can run the above code thereby "locking" the file and then in another process open the file, read the file, and write to it. That indicates to me that it's not really locked.

Any ideas how I can resolve this? I need to be able to open a file for reading and updating and lock it so that other processes can't change the file between my read and update cycles. I'm running the code on a Debian GNU/Linux box with Python 2.3.

Thinking about releasing a 1.0.1

, | Tweet this

I think I may release 1.0.1 this weekend depending on what else is going on life-wise. I tossed a feeler email out to the dev list in regards to what else should be fixed for 1.0.1, but didn't get anything back--which isn't surprising as people are pretty busy these days.

So... to do a release or not--that is the question. Is there anyone reading this blog who has things that they'd like to see for a very minor fix-it point release that's doable by one man who only has a bit of free time (unless they're interested in sending in a patch)?

Goodbye Audi S4!

, | Tweet this

It's the end of an era of my life, sad to say. I finally sold my car to a dealership for less than I wanted, but at least I can put that issue behind me and work on other things that need attention. I think I'm the last of my crew to sell the car I bought during the dot com hay day.

Now I'm going to spend some time adding folder management features to bluemail (which currently has none) and generally updating the codebase to use fewer deprecated Python things.

Why do talk radio hosts and Times reporters have no talent?

, | Tweet this

Just read this and I think very similarly. I was talking with my roommate Brook about it a couple of weeks ago and his thought (as I understand it) was that Joe Public needs the information in this form because otherwise it's too difficult to consume.

Unfortunately, most of the articles I've read in the last 6 months (anything beyond that is beyond the scope of my memory) don't even have a central theme--they just babble. So here's my thing--I just want facts, thoughts, and opinions; I want them clearly stated and I like them in context. I'm concerned that my fellow Joe Public Americans are getting their news fast-food style with little or no nutritional benefit.

Firefox and Thunderbird

, | Tweet this

I just switched up to Firefox 0.9 and Thunderbird 0.7 and they're really fantastic. I highly encourage switching to Firefox especially.

I keep reading articles about how people should dump Internet Explorer (article 1, article 2, article 3, article 4, ...) and while they all talk about features, defects, bugs, standards support, petty ui issues, and things of that nature, most of them miss the fact that there are still a lot of sites out there that require IE and don't work at all in any of the Gecko-based browsers. Amongst these sites are banks, financial institutions, online stores, and corporate web-sites. While I highly encourage people to make the jump to Firefox, I do understand if you can't because some of the sites you need to access day to day totally suck.

l337 speak

, | Tweet this

I'm on the openzaurus-users list and this guy Eddie sends these emails where his friendly name is:

   eD\\/ARd0 F/\\KEn^M3

I'd post examples, but SF.net has a terrible archive system.

It puzzles me as to why anyone would ever want a friendly name like that. When has l337-speak actually been elite?

New registry plugin soon!

, | Tweet this

Last night I did a lot of work on my registry plugin. I'm completely re-writing it from scratch--how it works, what it does, so on and so forth.

One of the things I never bothered to do was separate my blog (what you're looking at now) from my development. So whenever I make a mistake and create a bug, my whole blog goes down. Sometimes, in the case of last night, interesting things happen. I was playing with FileEntry and the generation of the absolute_path which had the effect of mucking up all the categories and permalinks on my site.

Anyhow, so things go up and come down pretty often on this blog since it's very very bleeding edge in terms of the source code and I'm often editing things live.

Having said that, I'm almost done the new registry plugin. I'll install it on Planet PyBlosxom soon and overhaul all the existing registry entries. This version of the registry plugin will make it easier to do a flavour registry as well. It's also a heck of a lot easier to set up and maintain.

CD Baby

, | Tweet this

I bought a couple of CDs at cdbaby.com and this was in the confirmation email they sent back:

   Your CDs have been gently taken from our CD Baby shelves with
   sterilized contamination-free gloves and placed onto a satin pillow.
   
   A team of 50 employees inspected your CDs and polished them to make
   sure they were in the best possible condition before mailing.
   
   Our packing specialist from Japan lit a candle and a hush fell over
   the crowd as he put your CDs into the finest gold-lined box that
   money can buy.
   
   We all had a wonderful celebration afterwards and the whole party
   marched down the street to the post office where the entire town of
   Portland waved 'Bon Voyage!' to your package, on its way to you, in
   our private CD Baby jet on this day, Wednesday, June 2nd.
   
   I hope you had a wonderful time shopping at CD Baby.  We sure did.
   Your picture is on our wall as "Customer of the Year".  We're all
   exhausted but can't wait for you to come back to CDBABY.COM!!

That's so cool--I wish more places were like this. My experience with them so far has been truly wonderful.