Dev (old posts, page 17)

Compiling the Tiger runtime for SPIM

, | Tweet this

I figured I'd post this because I just spent 24 hours trying to work out the issues.

I'm in a compilers class that's using the Modern Compiler Implementation in ML book (aka the Tiger book) by Andrew Appel. Over the course of the first 12 chapters, you build a compiler for a language called Tiger. Appel has a runtime.c that you can download from his web-site, but if you add functions to the runtime.c (for example, we added a ternary string comparison function which made string relops trivial to implement), then you need to compile runtime.c into assembler that's SPIM-appropriate.

Long story short I've got a Gateway 450 something or other laptop (i686) running Ubuntu Feisty Fawn and gcc version 4.1.2. I used the crosstool scripts from Dan Kegel. I used the demo-mips.sh script, but modified it to this:

#!/bin/sh
set -ex

# Big-endian MIPS

TARBALLS_DIR=$HOME/downloads
RESULT_TOP=$HOME/mipsgcc
export TARBALLS_DIR RESULT_TOP
GCC_LANGUAGES="c"
export GCC_LANGUAGES

# Really, you should do the mkdir before running this,
# and chown /opt/crosstool to yourself so you don't need to run as root.
mkdir -p $RESULT_TOP

# Build the toolchain.  Takes a couple hours and a couple gigabytes.

eval `cat mips.dat gcc-3.4.5-glibc-2.3.5.dat`        sh all.sh --notest

echo Done.

Ubuntu has /bin/sh point to dash--but this causes problems when compiling glibc (see here). So I fixed /bin/sh to point to bash instead of dash. That fixes the "error: missing terminating " character" error.

After you get your cross-compiler working, you can compile your runtime.c into a runtime.s with something like this:

% mips-unknown-linux-gnu-gcc -S -mrnames -mmips-as runtime.c -o runtime.s.raw

After that, you have to remove a few things so that it works in SPIM. Olin Shivers has a page that talks about this some more.

Hopefully I included enough words in here that this pops up in Google and helps future compiler-class takers in the same position I was in.

XPath 2.0 implementations?

, | Tweet this

I've been looking around for implementations of XML Schema and XPath 2.0 so that I can test some things out for a research project I'm working on. Essentially I need to be able to specify a schema in XSD, create an XML document (or two or three) that conform to that schema and then I need to be able to run a series of XPath 2.0 expressions on those XML documents. An implementation that works in Java or Python is cool, but at this point I'd even be willing to learn a new language to get something working.

In Java-land, the only XPath 2.0 implementation I can find is SAXON. However, the "free" version, SAXON-B, isn't "schema-aware" (i.e. it doesn't have XML Schema handling in it). The site says I can get an evaluation license... but I'm not really interested in evaluating SAXON--I just want to run some tests to make sure my paper is accurate.

JAXB has an implementation of XML Schema. JAXP 1.3 is in Java 1.5, JAXP 1.4 is in Java 1.6. Both JAXP 1.3 and 1.4 have an implementation of XPath 1.0, but not 2.0 (which makes sense given that XPath 2.0 is pretty recent).

Jaxen supports XPath 1.0, but not XPath 2.0.

I can't find anything else in Java-land.

In Python-land, there's 4suite which has XPath 1.0 support, but doesn't seem to know anything about XML Schema.

There's lxml which is a binding for libxml2 which has support XML Schema (at least, it has support for XML Schema datatypes), but doesn't have an XPath 2.0 implementation.

There's also Amara, but given the documentation talks about "node sets", I suspect it only implements XPath 1.0. That's all I could find in Python-land.

My research is probably spotty. I'm relying upon Google searches and what I can garner from 20 minutes of skimming the project web-sites. I've also been using some loose heuristics to disambiguificate1 usages of "XPath" with no version number--when someone refers to "XPath" or mentions "node-sets", I assume they're probably talking about XPath 1.0.

Anyhow, any ideas on XPath 2.0 implementations that I missed?

[1] - This is a word we used at ByAllAccounts often because it was amusing. There are other variations, too, such as disambiguificationalize. This sort of word bastardization makes documentation more interesting.

PyBlosxom status: 04/01/2007

, | Tweet this

I saw Titus' Strangling your code and growing your test harness: ... entry and that's the process we've been using for adding a test system to PyBlosxom. We're still on step 2 or 3, though. We have unit tests for a portion of the tools module in and working but that's it. We need a setup/teardown code for testing a request on a basic blog for n variations of "basic blog". Getting there....

Testing in Python

, | Tweet this

A few days ago, I saw on Grig's blog that Titus created a mailing list covering Testing in Python. I joined the list and it's been very educational. So far the posts have been in the "Hi--my name is _____________ (name) and I use ______________ (framework) and ___________ (framework) and ____________ (explanation for which framework gets used for which specific purposes)." Then this spawns a conversation with thoughts and insights and lots of applause. There have also been questions regarding how to use some of the frameworks and which framework would suit some need the best.

While I'm following the mailing list, I'm taking notes so that I can finish up this round of adding testing to PyBlosxom so that I can do a release this week. We're using nose and I'm using a similar structure that Cheesecake uses since I have some familiarity with their project.

It's a neat list so far. I'm glad Titus put it together--it's definitely solving my immediate needs.

Mudos under Ubuntu

, | Tweet this

I installed Mudos on my laptop which is running Ubuntu Edgy. I had two problems compiling the source for v22.2b14.

The first issue is this error when running make:

make: *** No rule to make target `obj/malloc.o', needed by `driver'.  Stop.

The solution, bizarrely, is to just run make again. I don't know what the issue is, but discovered this in the depths of the mailing list archives.

The second issue I get when compiling is this error:

socket_efuns.c: In function 'get_socket_addres':
socket_efuns.c:1198: error: invalid lvalue in unary '&'
make: *** [obj/socket_efuns.o] Error 1

Doing this change fixes the issue:

$ diff socket_efuns.c.orig socket_efuns.c
1198c1198,1199
<     addr_in = &(local ? lpc_socks[fd].l_addr : lpc_socks[fd].r_addr);
---
>     // addr_in = &(local ? lpc_socks[fd].l_addr : lpc_socks[fd].r_addr);
>     addr_in = local ? &lpc_socks[fd].l_addr : &lpc_socks[fd].r_addr;

After that, everything works wonderfully.

PyBlosxom status: 01/01/2007

, | Tweet this

My semester break is drawing short and I didn't get much of what I wanted to get done done. I merged Lance's documentation fixes in, I'm fiddling with a new pyblcmd script, and I'm cleaning up things I started, but never finished.

Ryan is going hog-wild with getting the comments plugin to a better place.

I was thinking of switching the web-site to use MediaWiki and in doing so solve a bunch of issues we have, but I think I'm going to wait on that for a while. In the meantime, I do plan on doing a web-site update and make it look more like some other Python-project web-sites. In the process of doing that, I think I may move it over to be statically rendered.

That's it for this status report--trying to keep it short and sweet while getting across that some things are happening, but it's slow going.

Many thanks to Lance and Ryan and all the other folks who have been putting time and energy into PyBlosxom over the last few months. You guys rock!

Deleting tickets in Trac

, | Tweet this

I'm having problems with spam in my Trac instance which keeps track of my PyBlosxom plugins. It's ... irritating. Anyhow, now that classes are almost over, I decided to poke around and figure out how to delete them rather than mark them as "invalid". Turns out it's really easy:

trac-admin [project-dir] ticket remove [ticket-number]

Cleaning up PyBlosxom (part 2)

, | Tweet this

I got a few comments from people who were interested in how I used Cheesecake to clean up PyBlosxom. I tossed around making it a series of blog entries but then decided to write it up as a technical article: Cleaning Up PyBlosxom Using Cheesecake.

I think it came out nicely, though parts of it are too "wordy". I really liked writing the article using Trac's wiki system. My only problem with Trac's wiki system is that the documentation for writing wiki macros needs a lot of help.

I'm going to update some of the PyBlosxom plugin writing tutorials I wrote a couple of years ago and put them in my trac instance, too. I still have work to do with PyBlosxom, but it's coming along very nicely. I'm working on unit tests and functional tests now.

Update May 11th, 2017: The pycheesecake and my Trac sites are now gone. You can find my article here>.

Cleaning up PyBlosxom

, | Tweet this

I've been following Cheesecake development with the hope that it'd be a tool that would help me fix PyBlosxom's shortcomings, make it easier for people to use, and make it easier for people to develop. It has certainly been all that and more.

I've been going through the various kwalitee measures that Cheesecake does to understand them and I've been making adjustments to PyBlosxom's source code and project infrastructure. I've made a lot of changes so far, discovered some latent bugs and code problems, documented undocumented code, fixed existing documentation, and started writing tests.

It's a huge effort, but it's a lot easier with tools like Cheesecake, nose, pylint, and coverage. Additionally, the blog entries at AgileTesting and articles at Living in an Ivory Basement have been helpful. Also, I've been shamelessly copying the infrastructure of the Cheesecake project. That's made it a lot easier to get my head around various bits.

At the moment, everything is local. I'd start checking things in but SVN at SF is really flakey (though the SF site status page doesn't mention any SVN issues).

I see the next version of PyBlosxom having some bug fixes and new features, but generally it's going to be a release focusing on cleaning up the codebase and the infrastructure.

Out of curiosity, while I'm in the process of cleaning up the codebase is it interesting to anyone for me to write up more detailed blog entries covering up how and what I'm doing? It might help by giving outlining a gameplan for cleaning up other Python-based projects. If that's something you're interested in, toss a comment on the blog or email me at willkg at bluesock dot org.