Aquaintances aqWiki Imported From Epistula Personal Projects

My Terribly Organised Life II

It’s Been a while since the first one, so some updates.

Operating System

Home is Windows duel booting with Ubuntu, mostly in Windows since the main use for the machine is Games and Web, and since Web is Firefox, Windows means I can do everything, Linux means I have to switch to Windows for games, so it remains in Windows by default. Ubuntu is there when I need to isolate myself from the games to get some work done. But for today, there’s Bioshock.

Work is a Macbook, and so runs OS X. I no longer have a Powerbook at home 🙁


On OS X I use BBEdit. Windows I still – after seven years – use EditPlus. Under linux I use Kate. Under all of the above I use vim also.

I will – I maintain – one day learn how to use vim well enough as a multiple document editor to switch to it full time, Today is not that day.


Email is sent from all the servers I get things at to a collection point on Cenote – my hosted machine – and there forwarded to GMail, which is now my primary email client. Between my phone, work, home and cafés, the main downside of web-applications – that you can’t access them offline – simply doesn’t apply to access to my email at the moment, and once they add Google Gears (Offline storage) to GMail, it’ll be – to a large extent – solved.

Personal Organisation

I never used Google’s Calendar site before, but work tend towards it so I linked up my accounts, and it’s become my primary organisation tool.

GCal syncs with iCal both ways thanks to Spaning Sync – Once Leopard is released, Google will probably start supporting the open Apple iCal Server stuff and this can be done nativly – and to Outlook with the Open Source Remote Calendars addon.

My Phone is an O2 XDA (AKA HDC “Wizard” model) running Windows Mobile 2005 – which doesn’t quite sync to mac yet but will sync to my home Windows machine where it can sync calendars from Outlook. This is overly complicated, and I look forward to the day when I can sync all my devices from one machine, preferably a mac. My iPod also contains my calendars and contacts, again thanks to iCal.

Todo lists live on my phone, they sync with nowhere. This doesn’t work well, and I frequently forget to do things.


I have a Global directory, containing Music, Documents and Projects which is rsynced to a server hosted by Bluehost, which gives 100G of space to play with for not much per month. Since I only got this a couple of months ago, I’m still rsyncing data to it, but eventually I will keep data synced across the external drive on my laptop at work (Music), my home machine (All Directories), Laptop (Documents and Projects), and development & hosting servers (Projects). I also need to add the Photos subdir to that, and automate the rsync process (rsync being done with cygterm (In conjunction with Puttycyg to replace Windows’ gods-awful command line).

If I need to access this – or any – external data, I can mount it over SSH as a drive. On linux with vfs, on OS X using macfuse, and with Windows using SFTPDrive, the latter being something I’m pretty sure should have an open source alternative, but apparently doesn’t, presumably because creating windows filesystems is apparently a headache from hell. All of these allow me to mount an external SSH server as if it were a locally connected hard drive, which is awesome.




  • Linky – Open Currently Selected Links In Tabs
  • Wesabe – For uploading my bank statements. I’ll have an article about this soon.
  • Firebug – The main problem I have with this now is that I can’t work without it.
  • IE Tab – Until Firefox finally takes over the world and I every site works in Fx, I have this under Windows.
  • Reload Every – Reload current tab every X seconds. Useful when debugging.
  • FoxMarks – Syncs my bookmarks file so they’re the same from all my various browsers. Handy.


    I still have an Aqwiki install which runs about a half-dozen sites – like – including a private auth-only one. Comics are still being managed by an evolution of the same script I was talking about in 2004, which is now at I moved from Bloglines to Google Reader a while ago – and have plans to resurrect the Aquaintances project to use Google Reader as a feed database (Meaning it will become an offline-reader for Google Reader, as well as it’s own coolness).


    OS X

  • Sticky Windows – Turn windows unto tabs at the side of your screen,
  • SSH Agent


  • Screen – Under which:
    • IRSSI
    • SLRN
  • Mutt


    • Putty – PuTTY makes Windows usable.
    • Ultramon
    • uTorrent
    • AVG
    • Civilization 4
    • Lord of the Rings Online
    • Neverwinter Nights 2
    • City of Heroes
  • Bioshock

    Stuff I’ve never heard of


Aquaintances Imported From Epistula

Aquaintances 2 alpha 000.1

It works.

Well, it takes a list of feeds, downloads them and displays the content, and doesn’t crash anymore.

Now to make it do interesting stuff.

It looks like this currently. It defaults to showing the full article in the main window, which is just temporary (It slows down the browsing too much, but was quicker to implement). The combo-box above the subscription tree will chose from a list of defineable filters (“All unread”, “Just highly rated things” etc). and the progress bar will move prettily.

If you really want to give it a go, the code is here, but the closest thing you’ll get to installation guide is the entry below this one. The only way to get feeds into the system currently is to run the “import OPML” button (The lightbulb on the taskbar, not the file selector. yet) which is hard-coded to import from the file “subs.opml” in the aquaintances2 directory.

Next: Filtering, and cleaning up the code a bit. First: Sleep.

Aquaintances Imported From Epistula Python

Aquaintances 2

Aquaintances 2 is an XML Feed reader for Microsoft Windows, Linux and Apple OSX. It will support baysian filtering of posts – meaning posts you are more interested in will float to the top of your reading lists – as well as regex field matching. It is built in GTK/Python (on top of LibGlade) using the Mozilla Firefox GTK bindings. It parses feeds with the Ultra Liberal Feed Parser, stores them (and most other things) in an SQLite database. It will revolutionise the way you keep track of the world.

Natually, it doesn’t work yet.

This morning I fired up Glade and put down the main interface for the system, which involved a certian amount of farting around with GTK’s box model, and all was fine until I had to attempt to tie the web browser to this.

My preference for this project was to use Mozilla’s “Gecko” rendering engine for the actual displaying of feeds, which was made more difficult by the fact that there are three sets of pages refering to the GTK bindings I was looking for. PyGTKMoz is an aborted attempt to get it working, Mozilla itself has a website on the subject, and PyGTKMoz refers me to PyGTK, which doesn’t mention the bindings at all. Eventually it transpires that the Mozilla bindings are now part of “python-gnome-extras”, which an apt-get installed for me.

That failed to work, because it doesn’t require “mozilla-dev” or “mozilla-firefox-dev” and even when those are installed, I had to manually add /usr/lib/mozilla-firefox add /usr/lib.mozilla to /etc/ so that Python could find it.

Then I had to work out how to get a Widget Glade doesn’t know about into my nice Gladey interface, for which this article was a handy guide, although it’s bitrotted a bit.

Okay, so we have an interface that happily displays a webpage. time to put some actual feed data in it. Importing my OPML file from Bloglines, I wrote a thing to parse the XML (Starting with minidom, which failed because I needed to keep track of recursive tags. so I rebuilt it in SAX (An acronym with too many abbreviations in it) with the standard push/pop method of keeping track of where the hell we are, a method I’ve always found exceedingly ugly, but at least it’s quick…

The tree interface to the subscriptions list was next. The GtkTreeView element caused me a number of problems, partially because I was trying to use it before I really understood it. Also, all the tutorials for using it I found (iki fi, & moeraki were the most useful, as well as the Real Docs) assume you’re building the thing from scratch, rather than editing an already existing TreeView object. Also, there was an interchangability between “TreeStore” and “TreeModel” that was starting to give me a headache. Then, in a blinding flash of light, my screensaver kicked in, englightenment dawned, and I finally grokked it.

So, I have an interface with working buttons that displays the home page of every feed I subscribe to. Except three, which causes a segmentation fault for reasons I don’t understand, but are probably Not My Fault.

Aim is to get it reading and displaying feeds by tonight, with the Bayesian stuff happening tomorrow, at which point I release Real Code and start making it both cool and useable.

2005-05-02: Changed gtkmozbinding instructions, the firefox library appears to crash if you need a plugin

aqcom Aquaintances epistula Imported From Epistula intertwingularity XML


Aquaintances now exports a valid OPML file.

This was far more work than it needed to be, because I have been unable to find a reference for a valid OPML file anywhere, OPML files got imported by Dave’s Wonderful New Toy as “0 feeds added”, which is odd, because they were in exactly the same format as Dave’s old Blogroll before he redesigned Grr.

Paul? Does this lower my Winer Scorecard number?

Oh, yeah, the other thing I’ve done today.

Banners are sticky. That is, it seems a shame to lose all these nice banners I spend ages making, so they now stick to the archive. If I can find my archive of all the ones I did last time I did the rotating banner thing, I’ll put those up too, but right now it’s just this weeks and January’s.

And yes, I’ll explain the “Frowny Lightbulb” thing soon. Promise.

aqcom Aquaintances Imported From Epistula


Some modifications to my blogroll system. The old blogroll was hopelessly out of date. The new one (Which you can get by clicking “Toggle Blogroll” on the front page, right side) is generated on the fly from Aquaintances’ list of subscribed blogs (Actually, from the ‘ukblogs’, ‘geek’ and ‘people’ categories of it. Aquaintances is, as you may remember, my personal RSS reader).

This required something of a rewrite of Aquaintances’ file format (So that it stores the URL of the website as well), which means that the new version will break old configs. Not that anyone beyond me and LoneCat are using it, but hey. When I’ve done the two outstanding major fixes (Supporting last-modified (it does ETags, but not L-M) I’ll give it a proper release, until then the code lives online in The Projects Section

The new Blogroll doesn’t currently support the few websites I read that don’t do RSS (Anna Pickard, Melissa, Murky, Kieron, Adrian, LondonMark & Andrew Swann), of which I will continue on my occasional quest to get Mark & Adrian to use something better, probably design a screen-scraper for Murky’s, hope Blogger gives RSS to free users for the rest 🙂

Aquaintances Computer Games Imported From Epistula Projects

Geek & Games

Lack of entries, you may have noticed.

Apologies for that, but I’m a little harrased. Mostly because of the standard First Month Of Job stuff (That I have to pay rent etc. as well as travelling costs from my non-existant cash flow until I get my first pay-check, not at all helped by the Job Centre taking six weeks to process my claim), but also because of general Stuff Happening. Planning stuff for this weekend (where I go camping with AFP), experimenting with anything new in PHP5 that might help me, MySQL4 (Which has just hit this server, so expect an upgrade of the search facility soonish) as well as theorising interfacing stuff like Trackback and PingBack with The Work Project.

On top of this, I’m redesigning. Well, not so much redesigning as designing. I’m making this masterplan to collate my non-aqcom projects to a new server where they can live without cluttering up AqCom’s navigation. This includes Forever and Threadnaut, at least one of which I’ll have to write now that I’ve mentioned it.

Aquaintances has been upgraded to use the latest version of Mark’s feedParser, meaning that if a feed returns a redirection, the config file will be rewritten to the new address. Aquaintances2 is in planning, and will include a simple NNTP server to access it. Epistula’s Gallery module, which has always been the most hackish of everything – mostly because 90% of it was lifted wholesale from the last version with minor tweaks – is suffering an extensive rewrite. If I can do this without breaking URLs I will do, but it looks like some major issues need to be resolved.

I’m also trying to avoid letting my RSI come back. This is so far involving watching Cowboy Bebop and West Wing episodes.

Countering the Anti-RSI activity, I’ve been playing Syberia, a nice return to the traditional adventure format, including actual character development. Sadly, the ending is somewhat annoying, but I expect the forthcoming Syberia 2 to fix that. I’ve also been watching videos of Star Wars Galaxies, EverQuest 2 and The Sims 2, all of which look very, very cool.

Aquaintances Imported From Epistula

Other Stuff

No entries recently. Nothing happening. Still no interviews. Though I managed to hit DitM with the return post. Talking of which, Aquaintances got updated with what Mark was talking about yesterday when I wrote strip_evil() (raw) and will safely strip (as far as I know) all evil from RSS feeds.

In other news, LoneCat – my other half – has finally finished the first version of Sublime, her diary engine, and therefore her diary is back online. Also my ex-housemate’s diary engine (Adversaria, notable mostly because it impliments the UNIX filesystem and some shell access internally with perl, and is designed to look like a terminal program) is also back online and as sick as ever.

Finally, if you haven’t been following, LondonMark has finished his epic “Case of the Missing …” detective story. If you haven’t been reading it, read it now (The permalinks aren’t, atm, because his archives are broken, but for a limited period it’s all on the front page. If it isn’t, then it starts on the 25th May)

Aquaintances Imported From Epistula

Aquaintances Next

Aquaintances now works, but there are a couple of things it doesn’t handle well, and I’d like to make it handle better.

A downside of storing articles as checksums of the content is that if an article is edited, it immediatly moves to the top of the list. I should probably checksum some other detail, like the GUID if supplied, or the link, but then I wouldn’t detect edits at all. I could generate a checksum of the GUID or link (Not title, Some feeds don’t supply titles or their titles aren’t unique), save it as that, generate the file, check the checksum against the checksum as the existing file (if it exists) and then magically highlight changes. Hmm.

RSS Parser
Either edit “Mark’s” to return all the stuff I want or make my own.
This is where it gets complicated. I want to associate categories to Epistula’s category set, meaning I could associate “Nick’s Virtual Culture”: category and “BB’s Metablogging”: category with my own “Metablog”: category. This would enable me to start on the “crossreferencing theory”: I was talking about, but also mean that if I was in a hurry, I could just see posts on the stuff I’m really interested in.
News Feed
With a little work, I could make it so that I could put a symlink in my newsserver’s directory and read all my feeds by NNTP. That’s one of the nice things about the flat file format :-). With a little more work, I could track those weblogs which allow comments sent as email, or have comment feeds per entry, and turn them into threads on the server.
Aquaintances Imported From Epistula Python

Python Question

I need to save a list into a tab-deliminated text-file. (Mark’s RSSParser sends me the last-modified of the feed as a list which I need to save to the config file so I can recall it when I next run and feed it to the site to make sure I’m not grabbing a duplicate).

The logical way to do this is to serialize the list and save it. For this, I find Pickle which does exactly that. The problem is that the format it saves out to contains line breaks, unideal in a tab-deliminated file.

The question is how do I save (or indeed print anywhere) a string as a raw string? The tutorial says that to output a raw string I prefix it with r, which causes logical problems when I’m trying to output from a variable.

Extra bonus points if you can tell me where I should have found this information.

Aquaintances Imported From Epistula


I’ve written an RSS aggregator. Go me.

As a first time python/PHP project, it’s really simple, Mark’s RSS Parser does all the work, and my junior Python puts it into a directory structure which is then read by a PHP script.

It splits into two parts, reads a list of feeds. The feeds are in a subdirectory of the script dir called “categories” and each text file within that is a category of feeds. For example, in categories/weblog is an entry Mark Pilgrimt (t means “tab”). generate finds that, gets the feed, saves the ETag of the feed – if given – back to the file (so the line becomes Mark Pilgrimt and we don’t keep downloading the whole feed if it hasn’t changed) splits out every item and saves each into cache/weblogs/Mark Pilgrim as a file like this:

Site: dive into mark
Title: The importance of human-readable markup

<p><cite>Slashdot</cite>: <a href="">HTML Rendering crashing IE</a>. Here's a <a href=" new.html">test page</a> that crashes my Internet Explorer 6.0 SP2 with all patches. Thanks to the wonders of integration, [...]

(Note, ESF format rides again :-)) with the file name as the checksum of the item. If a file with that checksum already exists, it won’t overwrite it.

view.php sits on the web-server. When called, it scans the cache directory for categories. When called with a category, it grabs every file from within it and puts the parsed contents into an array indexed by the time the file was created (Remember, the file was created from the item. If the item existed when we read the feed again, it wasn’t overwritten because there was already a file with that checksum) (There is also some fiddling to make sure that two items that arrived at the same second don’t overwrite each other). Said array is then displayed in a neat fashion.

It needs polishing, but it means I now have a LJ-friends-page-style list of every weblog and journal I read that has a feed, allowing me to stay up to date from my own browser 🙂

The demo (Which is my current feed setup) is here until I move it somewhere nicer 🙂