Aquarionics

Stephen Fry:

Bertrand Russell, the great philosopher and mathematician, got into terrible trouble by writing quite fearsome articles against the first World War when it began. He got all these letters from people who said, My child is prepared to lay down their life for their country. Dont you think that sacrifice demands some respect?
He wrote this extraordinary essay in which he said, Dont you understand? The sacrifice were asking of our young is not that they die for their country, but that they kill for their country. Thats the sacrifice. To ask a child to kill someone else, whom youve never met. Thats a moral choice, pulling a trigger. Having a bullet hit you is not a moral choice. You dont decide to be killed. Its a terrible thing that happens to you. But killing something is something you do and thats a desperate sacrifice. And were seeing that in the Iraq war. Thats what this poor Lynndie England did, this tragic soldier who was shot smugly smiling next to naked Arab prisoners. Thats the chickens coming home to roost. Its not Americans being asked to die by President Bush. Its Americans being asked to kill and to torture. Not necessarily by name. He doesnt say, I want you to kill this or that one. Of course, politics isnt that simple. Essentially that is what society does. It asks its young to kill, and thats what we all have to live with.

Aquarionics’ logging system was designed to work against mod_log_sql, a module that, er, logs to an SQL database. This worked until we upgraded to Apache 2, which log_sql didn’t support until recently. Since part of the logging system is the bit of AqCom that shows who linked here recently, I’d rather not convert it to run off plain text files (though I may be converting it to use Sqlite at some point), so I created a perl script that feeds the log into the database in log_sql’s format. It looks like this:

#!/usr/bin/perl

use DBD::mysql;

#Database options:
$dbUser = "user";
$dbPass = "password";
$dbName = "epistula";

$database = DBI->connect("dbi:mysql:$dbName:localhost:1114", $dbUser, $dbPass);

#204.95.98.252 - - [24/Dec/2003:15:23:38 +0000] "GET /archive/writing/2003/08/
	19 HTTP/1.0" 200 11873 "-" "msnbot/0.11 (+http://search.msn.com/msnbot.htm)"

while (<>) {
  my ($client, $identuser, $authuser, $date, $method,
      $url, $protocol, $status, $bytes, $referer,$agent) =

/^(S+) (S+) (S+) [(.*?)] "(S+) (.*?) (S+)" (S+) (S+) "(.*?)" "(.*?)"$/;
  # ...
        #$database->quote($thisdir);
        $q = "insert into apachelogs (remote_host, remote_user, request_time,
			request_method, request_uri, request_protocol, status, bytes_sent, referer, agent)
        values
        (".$database->quote($client).", ".$database->quote($authuser).", '".$date."', "
			.$database->quote($method).", ".$database->quote($url).", "
			.$database->quote($protocol).", ".$database->quote($status).", "
			.$database->quote($bytes).", ".$database->quote($referer).", "
			.$database->quote($agent).")";

        #print $database->quote($url)."n";
        my $sth = $database->prepare($q);
        $sth->execute();

}

…and is run using this crontab line:

@reboot tail -f /var/log/apache2/www.aquarionics.com | $EPBIN/apache2db.pl &

Now, the important thing to remember is that this gets pretty big pretty quickly, since it logs every line. It’s vitally important that you don’t under any circumstances, forget that you commented out this crontab line:

@daily echo "delete from apachelogs where time_stamp < `date +%Y%m%d --date '1 month ago'`" | mysql epistula

Because otherwise you’ll discover that your daily database dumps start to hit 16Mb each… BZ compressed… 380Mb uncompressed… oh, lets say four months and twelve days later.

For example.

(I ran the above query, or one like it, just before I started this entry. It’s just stopped:

mysql> delete from apachelogs where time_stamp < 20040825;
Query OK, 913830 rows affected (21 min 44.87 sec)

Reformatting for the girlymen who don’t have 2000px wide displays and are reading the RSS feed. See? This is why I want to only do partial content, because that way when I do something like this it only fucks up in IE

Okay, so my new workplace use Zope, so it’s probably a good idea for me to use that as a framework. I mean, enough people use it for it not to be too bad, right?

Dear god does it suck. I mean, Whales though really thin gauze, it sucks that badly. I got it working on my local network after swearing at it quite loudly. It’s own personal religion of “Not Invented Here” is going to really annoy me, I can tell.

Lets start with the web server. It doesn’t run though Apache without a tremendous amount of futzing around. I like Apache, and run everything else off Apache, so the fact that all my Zope applications will run off a non-standard port (There are fixes for this, yes, but they also require a tremendous amount of futzing around). This is annoying.

Then there is the user system. I have a user account on this machine, you could even talk to it though LDAP. Requiring your own little world to run in – let alone where you don’t tell me the initial username & password, or where the access file is, or anything – is really annoying. Also, it should be noted that in every SINGLE guide I found on how to reset the password manually it missed the fairly basic piece of information that once you have reset the password, you must restart the Zope server otherwise you will spend half an hour working out why it doesn’t take the data. This, admittedly, is partly a Debian Package problem, but since delving has lead me to believe that it used to ask you for a default username and password on configure, and now doesn’t, I’m leaping to the conclusion that whining isn’t going to fix this any.

It should be a default requirement that every Debian package contains a file in /usr/share/docs explaining a) Which configure options were used, b) What you’re going to have to do next to make the package work, and c) Common pitfalls of the above.

Three, the interface. In the last six months or so, the bar for a web-based interface has gone from a minor hurdle to a max-height pole-vault, mostly due to GMail. However, I can state with no fear of derision that <textarea /> is the single most fucking awful method of inputting code since I used a C64 with a broken delete key, and the fact that the physical location of the source files is shielded from me (and will, I suspect, be chowned something silly anyway) means the best way I can see to develop with this pile of shit is to copy/paste from my default text editor, which is just retarded (And yes, that is exactly how I compose blog entries, but I don’t normally need to keep blog entries in CVS). I suspect there is a better way to do this. In fact, I’m sure there must be, or else Zope would be deader than a very dead thing, which brings me to my next point.

The Docs

My first problem with the docs is the inline commenting system. When I’m trying to digest a reasonably complex idea it is of no use whatsoever for the flow of the text to be interrupted constantly by inane slashdotesque comment. On the other hand, without the inline commenting I’d not have known that the introduction to Zope tutorial – which is far better than the one included in the distro – existed, also, I’d have read the chapter on DTML before the chapter on the more advanced, easier to use, designer friendly and new paradigm of Zope Page Templates, which starts off by pointing out how crap everything you read in the previous chapter was. ARGH

Nevertheless, I shall persevere with this Evil, because I’ve got no choice but to know it, in the hope that I will learn my way around it.

Okay, so “find / -iname ZopeZoo” has failed to find any of the files that are part of the tutorial. Where the hell is Zope putting the bloody files? *ARGH*

Open letter from the Computer industry to the Entertainment industry

Look at us: every year, we churn out more computer games than your entire industry is worth. You know how we do it? We like our customers. We don’t treat them like potential criminals, and try to make our products do less

This part, if no other part of that manifesto, is bollocks.

This month I’ve bought five new games (It’s been something of a bad month for my self control). Of these, four of them have had sixty-four-thousand digit CD-Keys that I have to type in, three wouldn’t work in my old CD Drive, and one had a prerequisite of two other games (It was the final NeverWinter Nights expansion pack) which also required sixty-four-thousand digit CD-Keys, and one required signing up on a website also.

All in the name of “Copy Protection”.

In fact, I could bypass all this copy protection by downloading the games from Kazaa or eDonkey or something, because the first thing the Pirates do is remove all the copy protection. So it’s not the people who download the games who end up feeling like they’re being treated like criminals. It’s us, the people who buy the fricken games for £30 a pop.

The only game that didn’t have any kind of copy protection – no magic drivers, hidden sectors, no CD-Keys, no Web registration – was also one of the ones that wouldn’t work in my old CD Drive.

Since it was a 1950s era Monopoly set.

It’s got wooden buildings and everything, it rocks.

Today, I was on the train.

This is not an extraordinary thing, really. I spend between an hour and two hours on trains every week day (depending on which trains I manage to catch). I was using my laptop (Well, work’s laptop) as an MP3 player (Tell laptop not to shut down when the lid closes, turn on iTunes, put laptop in backpack with headphones plugged in, listen, occasionally wishing I could access iTunes with bluetooth so I could do “Next Track” from my phone) and prance merrily on my way.

Except I was on a train, so I was coding (work, regretably. I need to finish this project before I leave) whilst listening to the music, so I nearly didn’t notice that we were passing Hitchin, so I had a few minutes to pack up before we hit Letchworth – my stop.
At this point, for the first time in my history of commuting to and from London, the ticket inspecter tried to make his way down a packed commuter train.

Normally they know better than this, since all the trains from four ‘til six have people sitting in the aisles. This one didn’t, so a couple of minutes fumbling for the ticket and photocard, and we were pulling into Letchworth. Panic!

Shut laptop, shove in bag, get coat, put in earphones, stand up. And the earphones. Come out. Of the Socket meaning that the laptop plays the current track on full volume. Said track happens to be Liposuction by Da Vinci’s Notebook. Not ideal for saving face in a commutor environment. This would have been better if I hadn’t accidentally done that another twice. Argh.

So, in conclusion half a train full of commuters I’ll see every day for the next month or so now think I’m a fool. Still, could have been worse

But the MP3 of the week is neither of those. It’s a band I can find no mention of anywhere on the InterWeb, which is kinda freaky as I have to assume they don’t really exist. So, to introduce you to these people, who are good, the Little Screaming Pilgrims, Better Sideburns

Same deal as ever, they’ll be gone next week.