Categories
Apple computing Humour

Hello, Computer?

This is a series zero Apple Watch, charging on my nightstand because I forgot to do so last night. It’ll last a couple of days, but I try to stay in the habit of charging it overnight. Has touch display and voice commands.


This is a Seiko analogue watch, a christmas present from my parents a decade or so ago. It was my daily-wear watch until switched to smart watches.

Notably, it doesn’t respond when you ask it to turn the lights up, or set a timer.


This is Scotty attempting to talk to a PC through a mouse or, as I like to call it, a tableau of my morning.

 

Categories
computing programming

On Medium: How to solve a problem like passwords

I’m going to be occasionally publishing long-form things on Medium as an experiment. Here’s one

Categories
Computer Games computing Python

Sorting Steam Screenshots

So Steam, by default, when asked to take a screenshot will merrily scatter them hither and yon across your hard-drive with unwarranted abandon. Specifically, it’ll put them in the app’s home directory. This isn’t great, because what with cloud-saves and game streaming, I tend to treat the hard-drive with my games on it as transient, and not backed up.

However, Steam also has developed a setting called “Save uncompressed screenshots”, and if you set that and a directory, it’ll put all your screenshots in one place! Hurrah!

Except now all your screenshots are called things like “306760_20160825142347_1.png”. Now, the bit before the underscore is a Steam app ID, so you can look up the game name and file things nicely into the right folders.

So here’s a python script to do that thing.

Categories
computing windows

Backup Awareness Day

Life goes on.

Family stuff’s been on my mind recently, and there’ll be a deep post about the last couple of months at some point. It’s been a while since I did deep personal here, and that’s probably good for my life in general, but I can’t help but realise that I’ve got archives of my life going back 16 years here and 10 on livejournal, but for the last four or so anything of deep import’s gone to Facebook. I mean, I archive my facebook posts to machines I control, but it worries me a bit how much goes up there.

Anyway, Odyssey’s next week, and I’ve been spending evenings reviewing and recoding in prep for that. Work’s ramping up a bit, so that’s a lot of my head, and I was looking forward to spending this evening putting all of those things out of my head for a couple of days and brain-reset with the aid of some mindless pixel shooting.

Mindless pixel shooting was suffering from some performance issues, which I traced to my primary hard drive pegging at 100% with no applications running. A reboot sent windows into a cycle of “Attempting repair of C:\… 18%”… reboot “Diagnosing your PC” … “Attempting repair of C:\”… An hour or so looping through this, and it was obviously not going to work.

Resigned to the death of my hard drive, I dug out an old spare drive and set my old windows test laptop to download the Win10 media to a USB key, and found a Linux-based boot CD with a drive checker on it. Happily ever after, that came back with no problems, and rebooting into Windows worked again. So I’ve installed SSDLife and upgraded the firmware on the drive, and now I just hope it doesn’t happen again…

Well, more than hope, really. My system is designed so that almost everything I care about is on a different drive, the different drive is actually two mirrored hard-drives, the contents of that drive are backed up to Backblaze, and most of the important stuff is also synced with Dropbox. Losing my SSD would mean some hours redownloading a couple of games (though most are on a third drive), a while reinstalling software, a pile of irritation and a significantly less relaxing weekend. But it wouldn’t be a disaster.

I had my disaster about 15 years ago, and a lot of my documents and music went with it, and my setup was tested last year when an errant bit of software deleted the contents of my dropbox (which calmly resynced the now-empty folder to all the machines that once held it).

But yeah, today is National Backup Awareness day. Ideally your backups should consist of something local, something connected, and something nearby. I use an external drive for local, Backblaze connected, and Dropbox sync for nearby.

It’s worth doing.

Categories
computing linux sysadmin

Emergency Server Moves, Automation,

So, I’m having a bad server day.

This is a Monday Morning email if ever there was one
This is a Monday Morning email if ever there was one

It’s my fault, to a large extent. Earlier this year I discovered that my server had been sending daily emails complaining about a problem with one of the mirrored hard-drives, but these were going directly into Spam. I asked my hosting provider (Hetzner) to swap a new drive in, which they did, but the process of copying the old hard-drive mirror to the new one failed because of an error on the old hard drive. Not having the time to devote to fixing it properly, I bodged a fix and moved on with my life.

This is not a good email to receive either, but in addition to the last one, it's worse
This is not a good email to receive either, but in addition to the last one, it’s worse

Earlier this week, that problem developed into a fatal one, it’s getting worse, and so now I have a limited time to move everything off.

 

So now I own a brand new server, continuing with my usual conventions, this is Archipelago.water.gkhs.net.

One of the reasons I didn’t jump to a new server earlier is because I wanted to improve the setup of it. Atoll – the server which is failing – was set up three years ago in a hurry as I attempted to move stuff off of fjord – its predecessor – before the billing period was up. All of it was before I moved back into sysadmin/devops, and it’s a very traditionally set-up server – the kind I refer to at work as “Artisanal”; Hand crafted by craftsmen, unique and unrepeatable because it has no documentation as to *why* any config is as it was. My mission at work is to automate that stuff out of existence, and I wanted to apply some of the stuff I’ve learned in the last few years to my own systems.

Server herding over the last ten years has shifted away from big servers doing multiple jobs, and towards huge servers pretending to be many tiny servers doing single jobs. Initially I thought of setting up the new server as a virtual hosting server, setting up many small servers. I’ve come to the conclusion this is impractical for my use-case, which is that the server tends to host many dozens of tiny websites and services. Dedicating a slice of resource to each does none of them any favours, and increases the admin burden rather than decreases it. Instead, I’ve gone towards a halfway house solution of putting most of the separate servers on Docker.

docker-logo(Docker, for those who haven’t seen it, is basically chroot on steroids, an enclosed mini-OS that shares physical resources with the host, but only has access to the service and disk resources you specifically give it. For example, you can grant it a port to talk to MySQL on the host (or another Docker container), and a directory so it can maintain state between restarts).

Rather than manually set up the server for Docker, and have many automated boxes inside a grand artisanal crafted shelf, I’ve decided to use Ansible to manage the main server setup (and a lot of the DNS, since most of my sites use Amazon’s easily scriptable Route53 DNS server service). I’m still learning Docker, but I’m comfortable in Ansible, so I haven’t gone as far as to orchestrate the Docker setup itself with Ansible, just the install.

All of this is kind of like getting everyone off of the capital of Sokovia before the rockets hit the right altitude. In this metaphor, the unbeatable forces of entropy will be playing the part of Ultron
All of this is kind of like getting everyone off of the capital of Sokovia before the rockets hit the right altitude. In this metaphor, the unbeatable forces of entropy will be playing the part of Ultron

Docker’s given me some fantastic advantages just in the 12 hours or so I’ve spent using it. I’ve long been a user of things like Virtualenv for python to separate out my language and library versions on a per-application basis, and the ability to do that with whole OS packages is very useful. It enables one of the holy grails of PHP development: Hosting multiple sites with different versions of PHP at the same time. So WordPress can be on the brand new PHP7 (you’re using it now, in fact), while a PHPbb install can remain on 5.6 until they get their act together (or I switch it to Discourse or something). For this traditional hosted kind of server, which does many things for many different people, it’s really useful.

Taking Control
Taking Control (Photo by Ady Satria Herzegovina, used with permission)

All this is still an arse, though. Working through today, I’ve got WordPress and the forums moved over, some of the static sites, but still less than 20% of functionality’s been moved. Taking a break now (It’s midnight as I write this) for some gaming and then probably sleep. Maybe I can finish this tomorrow…

 

 

Categories
computing Imported From Epistula internet

A traditional post – Geekcode 2016

A tradition doesn’t stop being a tradition just because it’s lost all relevance to the modern age. That’s what Christmas is all about!

And one I’ve got is that every couple of years I update my Geek Code. The Geek Code is an old and established tradition, a one line incomprehensible block of symbols that represents your opinions and state on many geek things that mattered in 1995 when it was last updated. As such, it’s missed an awful lot of pop-culture advancement, including the bit where the inclusive nature of geek culture took over the world, and then the toxic nature of it burned some of it down. There’s probably a better article about that, somewhere. Anyway, here on the first day of 2016, the traditional geek code block:

-----BEGIN GEEK CODE BLOCK-----
Version: 3.1
GCM$ d s+:+> a C+++(++++)$ ULS++++$ P- L++++$ E- W+++$@ N o+ K w+$ O- M+ V- PS++ PE-(--) Y++(+++)$ PGP+() t 5+ X R++* tv+ b+>++ DI---(++) D++ G++ e+ h--- r++ z? 
------END GEEK CODE BLOCK------

The rules:

  • Done with the most recent 3.1 release of the Geek Code
  • Phrasings are as I understand them as I take the quiz, and formatted according to my whim.
  • I don’t look at previous geek codes before taking the test again.

All the geekcodes, 1998 -> 2016 are here

Observations:

  • It kind of amuses me how I reinterpret the “Dress” section every time, given that my general style hasn’t changed much.
  • My confidence in Computing & Unix varies over time.
  • That you can track down my addiction to Usenet as you can track up the things with “gets paid for this” ($) signifiers amuses me.
  • Watch my opinion of Dilbert do a ski-course impression, as the writer asserts his opinions more.
  • You can basically track my entire relationship history though the (r) column on this graph.
  • I care a lot about security now I’m being paid to.

In the beforetimes:

I’m going to try to resume the weekly posts thing. We’ll see how that goes.

 

Categories
computing linux

Fear and go seek

The common refrain for people advancing the cause that says that encryption should have government back-doors is that only people with something to hide need to encrypt their work. If you have nothing to hide, then you have nothing to fear.

Quite apart from that not being true in the general case, in the specific case it’s bad too, and this is why:

The following things use the same kinds of encryption that the government wants to put back doors into:

* Every website you put your credit card number in to.
* Your online bank access
* The communication between the website you put your credit card number in to and your credit card company.
* Your connection to your email
* The ability for home-workers to log in to their company’s network (VPN)

Companies are legally obliged to keep their company secrets secured, workers are contractually obliged (and in some cases also legally) to keep those secrets to the best of their ability.

These measures would grant the government the ability to read – as they flow through the wire – the items above, and even if you believe the government should be allowed to do this, there’s a wider issue.

Governments have, so far, not demonstrated the ability to keep their own documents secure, which would include the details of back-doors into secure systems, and once widely-used standards to encrypt communication are blown open. Even if somehow governments managed to perfect their own security, the known existence of a back-door would encourage the high number of highly intelligent people that have the required technical skills to try and find it, either for intellectual curiosity, or in order to read your data. Basically, it means that the encryption we rely on every day to make our lives easier and be able to do things over the internet, advancements that make things like personal banking and shopping possible for disabled, busy or just lazy people; suddenly become a lot more risky.

Computer security’s taken a bashing in the last year. Several deep investigations into the publically developed libraries that underpin a lot of internet security have resulted in a number of very public and news-friendly panics, and undermined confidence in them in general (To which the response is: Bugs being found – and fixed – is good. I’m happy that smarter people than me can see the code I rely on, and will publicly say if there’s a problem with it, rather than hope nobody notices), but the fact that ISIS, Apple, Google and other reprobates can encrypt their data so that it can’t be recovered in sub-decade timescales means that your credit card data can be stored safely, that your bank is able to offer your balance to your phone, and that companies – maybe yours – can let you VPN in and work from home once in a while; and crippling it is a very high price to pay.

Categories
Apple computing linux tv windows

Ripping TV Yarns

I’m in the process of ripping some boxsets of DVDs to Plex, and I thought I should probably document the process. The most obvious thing I’m not using here is Handbrake, which works really well for some people, but I am not one of them.

Physical to Digital

MakeMKV turns any DVD or Bluray I’ve thrown at it into an MKV file. The one thing it could do to make my life better would be custom tags on filenames, but the default {Directory}\{DVD Identifier}\title{nn}.mkv  is good enough. {DVD Identifier} is annoyingly unspecific most of the time, and sometimes within disks of the same box set (The thing I’m currently ripping has both WW_S4_T5_D3 and WESTWING_S4_D6 in it, as discs 1 and 6 respectively), so the next stage is to make those directory names consistent. It doesn’t matter what they are, so long as when I “ls” the directory, they are in the right order. Then, I run this:

export COUNT=1; # Start at 1
find . -name \*mkv \ # Find all files ending MKV
	| while read fle;\ # For each of those (as variable $fle)
		do mv $fle $(printf "The_West_Wing-S04E%0.2d.mkv" $COUNT);\ # Increment the filename
		COUNT=$(($COUNT + 1));\ # Add one to the filename count
	done

Note: You’ll need to collapse that back into a single line without the comments for it to work:

export COUNT=1; find . -name \*mkv \| while read fle; do echo mv $fle $(printf "The_West_Wing-S04E%0.2d.mkv" $COUNT);COUNT=$(($COUNT + 1)); done

This gives me a directory of well-named MKV files.

Digital to MP4

Plex is happier with mp4 encoded videos than with MKV files, though, plus they’re smaller without a noticable (to me) drop in quality, so when I’ve got a few series of these built up, I’ll run this overnight:

for fle in mkv/*.mkv; do encode.sh $fle; done

Where encode.sh looks like this:

#!/bin/bash
file=$1
ffmpeg -i $file -codec:v libx264 -profile:v high -preset ultrafast -crf 16 -minrate 30M -maxrate 30M -bufsize 15M -metadata:s:a:0 language=eng -c:a ac3 -b:a 384k -threads 2 ${file%.*}.mp4

Which is a standard ffmpeg encode line, the only real weirdness being the ${file%.*}.mp4 bashism, which basically turns the $file variable from “Foobar.mkv” into “Foobar.mp4” (It will also turn “Foo.bar.mkv” into “Foo.mp4” though, so be careful)

MP4 to Mediacentre

Once that’s finished, I’ll get rid of the mkv files, and send them into Plex. To ensure consistency of my filenames and also get any subtitle files I need, this is done using filebot, like this:

filebot -script fn:amc --output "/media/mediashare" --log-file amc.log --action move --conflict skip -non-strict --def music=y subtitles=en artwork=y --def "seriesFormat=TV Boxsets/{n}/{'S'+s}/{s00e00} - {t}" "animeFormat=Anime/{n}/{fn}" "movieFormat=Movies/{n} {y}/{fn}" "musicFormat=Music/{n}/{fn}" --def plex=localhost .

(Filebot, rename using the (included) automediacentre script. Output to directories below my media drive mount, log to amc.log, move (don’t copy) the files, if it already exists skip it. Don’t do strict checking, download music, search for subtitles, get series artwork, send TV shows to the “TV Boxsets” directory in {Series Name}/S{Series Number}/s{Series number}e{Episode Number} – {Episode Title} format. Anime should go somewhere else, Movies somewhere else, Music somewhere else, then notify plex on the local machine. Do this on the current directory)

Operating System Notes

None of this is OS specific. Filebot, FFMPEG Plex & MakeMKV are available – and work identically – on Windows, Mac & Linux. The various bash scripts could be adapted to powershell, but I’d instead recommend Babun, which is a repackaging of cygwin with a far nicer interface and package management system that’ll give you the basic *nix commandline tools on your windows machine (all of the above up to MP4 to Mediacentre runs on my beast-sized windows gaming rig, to avoid making the puny media centre CPU cry too much)

Categories
Apple

Every Two Years

It’s new phone time again. I’m fairly statically keeping to the iPhone ecosystem for my main phone (I own android devices, but mostly for testing & backup). Having fallen out of “Smartphones” with a disappointing O2 XDA, my first experience of the first iPhone was borrowing one from the CTO of Trutap overnight, and losing it on the train home. It was an expensive day. From then I got an iPhone 3G when it arrived in the UK, switched sides to an HTC Desire when the iPhone 4 didn’t appeal to me, but after falling out with android’s lack of focus, switched back to the iPhone 4S for my next contract.  And here I’ve stayed. Two years ago I bought the iPhone 5S, and just as the battery on that is beginning to get annoying, the 6S is here.

Out with the old…

Out with the old - an iPhone 5S being put back in its box
Out with the old – an iPhone 5S being put back in its box

And in with the new…

in with the new, an iPhone 6S, cherry
in with the new, an iPhone 6S, still in plastic

…and then back in with the old

An iPhone 6S, restoring from backup
An iPhone 6S, restoring from backup
Categories
Apple

Apple, Kremlinology and Technology in action

During the cold war, a starting phrase that begins all the best topical blog posts, when the USA was on Not Speaking terms with the USSR, the only way to work out what the political situation in Russia could be was to examine the official photos of events, and take notes of who was standing closest to the leaders, and whose star appeared to be fading. This gave breath to the word “Kremlinology”, the inference of probable facts from disparate unconnected information. Since Apple are so secretive about their intentions and arcs, media covering them generally make things up based on what seems to be true. I’m going to do that in a bit, but I don’t think my guesses will be better than anyone else’s.

I’m wary, though, of anyone who has financial incentive for Apple’s stock price to change. Think-pieces on how Apple “has to launch a new product this quarter” and is “doomed” (frequently) if it doesn’t follow a specific trend. I’m frequently amazed at how many announced (unlaunched, unspecified, and some times *in research & development*) products get reported as being able to kill Apple. A lot of people stand to make a lot of money on long bets of Apple’s stock price, because – like the London property market – it has gone from a thing that is what it is, to being a Financial Item, and is therefore being played by people who have no interest in the market around it, only in the the number it is tagged with. Apple is one of the few companies in its ballpark that knows the meaning of the word “Doomed” from a business perspective. The company has seen how it happens, and it has recovered.

I’m also ignoring the Android “war” in this. I chose my ecosystem a long time ago, and I’m invested in it. Since my first iPhone in 2008, I’ve sunk hundreds of pounds into apps and daily processes that apply to it. My thoughts on “Apple’s copying what X did Y time ago” are that it’s great that my ecosystem can do some of the things yours can do, equally it’s nice that your ecosystem’s picked up tricks from Apple. Rising tide, all ships, etc. Android’s got the market share – more people own Androids than any other smartphone OS. iOS sells fewer units than Android, but Apple makes more money than HTC. Actually, the only real winner is Samsung, since they manufacture iPhone parts and also the competing Samsung phones. Neither company has refrained from bitchy comments, though. Apple’s new Android app to help you move over to iOS was introduced as “The neighbourly thing to do” in a snide comment, and Samsung buying adwords on “iPhone 6S” with bitchy comments was just as bad. The shit-slinging is just petty.

Having said that, the iPad Pro is a broadside shot against the MS Surface. I’m not a fan of the surface keyboards – I’m told there’s a newer one that might be better, but the travel on the keys has always been something that didn’t work for me. The smart-case with a built-in keyboard looks a lot nicer to type on, though I’d need to try it. The screen looks absolutely glorious, though. It was hilarious to watch them announce a keyboard that was on a direct descending line from the Surface Keyboard and then introduce Microsoft on stage to do the dull office stuff and validate use of the prefix “Pro”.

One of the main problems the iPad seems to have – at least for me – is there’s no compelling upgrade path. My phone is tied to a contract I renew every couple of years anyway, and even my laptop is on a 3/4 year obsolesce path due to hard-drive and memory upgrades, but my only compelling reason to stop using my iPad 2 was the retina screen, and even then it took me until the iPad Air to even consider the upgrade.  I’ve not even looked at Air 2 upgrades, and another “thinner, lighter, faster” speedbump upgrade wouldn’t change it. My iPad is a reading and triage device, primarily existing to watch comics/movies/TV or to triage Omnifocus tasks, email, Teamwork tickets. One of the reasons it’s not a note-taking device in general is that my soft-keyboard skills aren’t that fast, and writing on it – even with a stylus – is a pain. The Pencil and the Pro look really nice, offer a new use-case for the device, and generally turn a bigger/faster/shinier update into a fairly compelling upgrade.

Notably, though, the iPad Pro’s missed Apple’s new interaction method. From the video, they needed to rework how the screen sensors pick things up in order for the Pencil to work properly, and in the 6S they reworked the screen for 3D Touch, and I guess those things just couldn’t be reconciled into the same device yet. It’ll be interesting to see if the Pencil support makes its way down the line to the smaller devices, or is reserved for the iPad Pro line; but dollars to donuts Touch 3D climbs up from the iPhone with the next hardware revisions.

No new iPads this year, though. Apple works in cycles, and for the last few years the iPads have shared a stage with the OS X release with the October announcements, but for the first few (iPad 1 though to iPad 4) the iPads were part of their own annual announcement in Spring. This year, Spring was the Apple Watch big release (having been teased in the autumn), so I’m wondering if iPads are moving to the Spring announcement with the next Watch. The announcement of the new iPad Mini 4 crammed in at the end of the Pro announcement weirds that out a bit, but maybe they wanted to get the new Mini out in time for christmas.

Apple TV looks good. The big thing it does is aggregate all the places you *could* see a film into one searchable archive (so you find Matrix *first* before finding out it’s not on Netflix but is on iTunes). I’m hoping this has some kind of app-interface, so Plex can say “I’ve got it!” or Amazon Prime – very notable by its absence across the entire presentation – can be plugged in quickly. I’m betting not, though.

The iPhone 6S is a nicer phone in the same case with better camera. Sold. The Force Touch / 3D Touch looks interesting, but seems a little mystery-meat in its navigation. How do you know if a thing can be pushed hard on? It’s all very well to talk about “exploration” and “discovery” in apps, but it’s not accessible. Plus, does it really do anything you couldn’t have done with touch-and-hold?

The final big thing was slipped in at the end, though. Apple As A Service, where you pay a monthly subscription and get a new phone every year. Apple’s finding a way around the 24 month phone contract stranglehold, I think, in order to try and consolidate its customers on an annual loop instead of a biannual one. The implementation is kind of sketched-out and clunky on the site right now, it seems to involve a 24 month loan from a 3rd-party bank, with no mention of the upgrade at 12 months bit, and doesn’t seem to answer basic hire/purchase questions such as “Is the phone mine at the end” and “am I still liable for the rest of the 24 months if I lose or break it, even under insurance” but also missing is “is this coming to the UK” so for me, at least, the point’s moot.

As mentioned on twitter, the entire Macintosh line got the same amount of stage time today as the Lisa II. Post PC society, indeed.