Aquarionics’ logging system was designed to work against mod_log_sql, a module that, er, logs to an SQL database. This worked until we upgraded to Apache 2, which log_sql didn’t support until recently. Since part of the logging system is the bit of AqCom that shows who linked here recently, I’d rather not convert it to run off plain text files (though I may be converting it to use Sqlite at some point), so I created a perl script that feeds the log into the database in log_sql’s format. It looks like this:

…and is run using this crontab line:

Now, the important thing to remember is that this gets pretty big pretty quickly, since it logs every line. It’s vitally important that you don’t under any circumstances, forget that you commented out this crontab line:

Because otherwise you’ll discover that your daily database dumps start to hit 16Mb each… BZ compressed… 380Mb uncompressed… oh, lets say four months and twelve days later.

For example.

(I ran the above query, or one like it, just before I started this entry. It’s just stopped:

Reformatting for the girlymen who don’t have 2000px wide displays and are reading the RSS feed. See? This is why I want to only do partial content, because that way when I do something like this it only fucks up in IE