Olivia Newton Coleridge

I met Ted Nelson once.

Now, one of the problems with mentioning someone like that on a page that support hyperlinks is; to what do you link his name?  There are many opinions about Ted and they range from rabidly supportive, almost Messianic devotion to the sort of ad-hominem attacks that are usually reserved for the… less popular political figures.  So I linked the three syllables of his name to three different sites that (sort of) span the range.  The first one is the notorious Wired article.

One of the most interesting thing about meeting him was to observe the response of others.  I was there with a couple of colleagues; one of whom was a Huge Nelson Fan (which was why we ended up on Ted’s boat in Sausolito drinking beers) and in him I saw something new.  Here was a manic visionary, someone who would pursue radical ideas beyond their limitations (or alternatively condemn them utterly) who I now saw treating Ted’s slightest opinions as unquestionable axioms.  Whether this was because of a genuine reverence or because this was the sensible way to cope with a maverick radical intellectual, I don’t know.

There’s this effect that hangs around people who are famous, sort of like a distorting field.  If one was wearing augmented-reality glasses, I imagine that someone like Nelson would be surrounded with tags, links to pages written about him and his work, comments about his personality and attitudes.  The real person is underneath, somewhere, their appearance coloured by layer upon layer of report.  But this is all relative, of course.  My wife wouldn’t have the first clue who Ted Nelson was and (if she’d been there) would have seen only a pleasant enough guy, ageing, slightly birdlike in the way he moved and talked.

So I made a concerted effort to dispense with prejudice, and listened to Ted as he was shown the ideas and software we’d brought to California, and as he reacted to them.  And a pattern began to emerge.  That which he was shown “had been anticipated in his work years ago”.  Principles explained to him were “already well understood” in what he’d done.  In the same way as his reputation blocked him from my sight, his fight to defend his work blocked him from seeing anything else.  Everything technical existed in relation to Xanadu.  I felt sad, that someone should be so tied to a venture they were unable to let it go.  I wanted to say to him: let it go.  There were probably good things in it, and bad.  We all write stuff like that, we all end up with projects that don’t fit the real world enough to succeed.  Move on.

Later in the evening, there was a small Generalist Gathering on Ted’s boat for Leap Day[1]  I left before then, taking a cab back into the city.  Reports that reached me later spoke of an assembly of some of the strangest people in the Bay area; those so far along the divorced-from-the-normal spectrum of geekhood that their only relation to the real world and the realms of the physically possible came from association with other geeks who weren’t quite so weird (who in turn related to other lesser geeks, and so on until one approaches normality).  I’m not sure whether Ted was their connection to the world, or they were his.

[1] This is how I remember it, anyway.

I Killed A Risk Today, Oh Boy…

Actually, it was last week, but I tend to let things simmer before I blog them.  It’s more fun that way.

We’re approaching launch of The Mobile Phone Project and, if things go according to at least one of several plans, there’ll be a lot of people eventually hitting the website that promotes it.  I mean a lot of people, and that’s an interesting factor.

Last week, I was indulging in an hour’s break from the head-down code/test/organize/phone/email cycle that’s occupied me since the beginning of the month and trying to look at the big picture; to gain an overview of what needed to be done before we push the metaphorical Go Button[0].  What occurred to me was this: of the software that runs on the phone, there are some areas that are not as elegant as they should be; more complex than feels right.  Not that they’re badly written, or that they’re flaky, or that they’ve caused tests to fail; no.  But the reason they exist is to satisfy certain functional requirements that have been in the project since early on, and I found myself wondering if they’re worth the extra risk.

Look at it this way; this is end-user, consumer technology.  It runs on mobile phones, thus having it work simply, correctly and reliably is key.  The extra bits of software add complexity.  They interact with the phone and each other in ways that are not easy to test exhaustively (think of a network of co-operating processes and you’ll get the idea).  You and I, being programmers, both know, deep down, that’s a source of extra risk.  Now, if one in a hundred users has a problem, and we have a hundred users, that’s not a heavy support issue.  But what if we have a thousand?  Ten thousand?  Per month, or week?  If risks are quantifiable (and a whole industry of risk management tends to argue that they are), these are small percentages.  Yet when I multiply them by the big numbers of users, the end result is worrying.  So I wrote a change request to remove functionality.  To comment out, remove calls, unlink modules and simplify the whole thing.  The code can go into version two, if there’s a good commercial reason for it.

Was it wrong for the code to have ended up in there?  Well, yes, from one point of view, because I’ve just written a change request (which costs effort and therefore money) to take it out.  And no, from another.  Each step towards the code was, at the time, reasonable.  The need for the functionality was established, based on good commercial reasons.  There were discussions about how to achieve it, solutions proposed and adopted, then turned into code.  But at some point along the way, the code became self-sustaining.  What I mean is, there was no questioning as to whether it was doing the right thing, just about how it should work.

I fell victim to this myself last week, in a tiny piece of Python code that handles the upload of data to the website and database, synchronizing the two.  As part of this, images need to be processed in certain ways and, as a good Pythonista, I immediately wrote import PIL and started playing with the Python Imaging Library.  Well, it started out as playing, and ended up as wrestling.  Animated GIFs are a lot more complex than it might appear, especially when transparency and smallest-area-changed optimizations come into play, and the GIF reading code of PIL just… doesn’t… work (with certain well-optimized GIFs).  I spent three hours working through it, patching and testing, before a SIGALRM went off deep in the back of my head and pointed out that I was working hard at the wrong task.  I’d become so caught up in how to solve the issue that was directly in front of me that I’d lost sight of the reason I was trying to achieve it, and thus all the different ways in which it could be done.

Half an hour later, the Python app was invoking command line apps like mogrify and gifsicle, letting them do the clever GIF stuff on temporary files that were sucked in and out of Python as required.

The English saying that best addresses all this is I can’t see the wood for the trees.  Hand me my axe, I’m a lumberjack today.

[0] Actually, I’m seriously considering having a real Go Button.  I mean, making something live should be an event, right?

Ah, You Need The Deluxe Version, Sir!

The amount of music digitally stored in this house grows apace.  The storage devoted to pictures ditto, especially since Small Daughter has taken an interest in photography (and has a pretty good eye for a well-composed shot too).  Thus a young geek’s fancy turns to thoughts of additional disks.

Since I have a brother employed by a computer dealership (actually, they do lots of other things as well, but they deal in components and PCs and all that), one short IM conversation orders me a matched pair of 120Gb drives – slower ones to save money, since they’re not for any use that requires the ultimate in speed and I’m far too old to go through that my-system-is-100ms-faster-than-your-system stuff.  These will form a RAID1 pair.  So, in anticipation of delivery, I ponder the question of where they go – in the XP Pro desktop server, or the Linux server.

Which is when I look up RAID in the XP help for Disk Management.  And discover that (and I quote): You can create mirrored volumes only on computers running Windows 2000 Server, Windows 2000 Advanced Server, or Windows 2000 Datacenter Server.  Hmph[1].  Actually, it’s one of those facts that I knew, somewhere in the depths of the back of my head, but had forgotten until I read it.  And I’m pretty sure I said “Hmph” the first time I saw it.

Now, as I’ve pointed out before, I have no real issue with the Demons That Infest Redmond; they have their products, sometimes I uses them, sometimes I doesn’t.  There is no ideology here, just practicality.  But this sort of markety-feature-bullet-list thing annoys me – is there any good reason why a home user would run one of their expensive Server OSs?  No, not really.  Is there any good reason why a home user would want a decent mirrored or RAID setup?  Well, yes, surely.  Then why the blinkin’ flip doesn’t XP Pro (and this is XP Pro, not the neutered Home version) support mirroring?

Of course, I could do the mirroring in hardware, using SCSI, if I wanted to spend more money and have a more complex setup, sure.  Only I use IDE disks everywhere and they get traded around between machines fairly often.  However, I have a better solution.  Into the Redhat Linux server they will go; software-mirrored and served via Samba.  None of the Windows machines will care, it’s all network storage anyway and I’m left with another fact to consider when balancing the choice of OS for storage servers.

But it still annoys me.  It’s the sort of spread-the-features-over-the-product-range phenomenon you see in cars; do you buy the Ford Escort L, LX, Ghia, Popular?  Each has a different set of bells and whistles carefully selected by marketroids to form a suite of product options to enhance consumer choice.  Or something.  And the open-source approach tends to take the other point of view – you get everything in the one package.  Everything, even stuff you probably won’t ever need.  Hey, you get MySQL and PostgreSQL!  You get four different web server frameworks!  And at least ten different languages whether or not you call Perl a language!

And, naturalmente, you get software RAID.  Para mi, elegir es f├ícil.

[1] The Hmph is at the fact, of course, not the redundant comma before the “or”, or the spelling of “center”.

What’s In The Box?

Productizing technology.

Reading Dave Winer on Longhorn, his phrase “A product with a purpose has a two-sentence description that gets everyone so excited they can’t wait” resonated in my head, the way a bell does in a cathedral, drowning out other considerations until the reverberations die away.  Not that I’m a huge admirer of Mr Winer – in this case I followed a link from Miguel de Icaza’s blog, but I think he hit a nail with that one.

On my business card it says “Technical Director”, which, when you come down to it, is a pretty ambiguous term.  In the case of the Mobile Phone Project (and I promise the two people who emailed me that I’ll post links the day it goes live and start using the real name, okay?) the job is pretty much this: tell the product design guys what’s possible, and guide the technical guys towards making the right product.  In other words, keep the two sides, whose respective worldviews, languages and focus have very little in common, in touch.

Miguel sums up one of Joel Spolsky’s pieces by saying engineering organizations are too much in contact with the technical details of how to make things happen, without looking at where and how the software is going to be consumed, which is a neat way of describing half of the problem I’m employed to solve.

Let’s look at this from the marketing side first; techies are notorious for arguing over details and fine points that the product concept people think are irrelevant.  “All we want is a flying carpet; how it flies is your problem, don’t bother me with detail”.  The extremes of this are, of course, Pointy-Haired-Bossdom in its purest form; the end result of the through process that begins “anything I don’t understand is irrelevant”.  But what a disturbingly large number of geeks don’t seem to get is that the product people often aren’t stupid, or ignorant; they just see the world differently.  And often they are very good at knowing what will appeal, what will have value and what will sell.  I find, myself, that sales are quite important (for boring accountancy reasons with which the Pure Techies need not concern themselves).

So for that side of the business, my job is to interpret technology.  Not to dazzle with science and act the part of the High Priest and Keeper Of Arcane Knowledge but to:

  • explain where it’s necessary to explain
  • summarise where the details aren’t necessary
  • use analogy where it’s appropriate
  • speak the language of those that are listening

The best mental model I’ve found is to see it as translation.

From the techie side; product concept people are notorious for ignoring detail and making outrageous requests for ridiculous requirements.  For that side of my job, I have to translate what the product needs to do into the details of how it can accomplish that.  Often this is iterative:

Concept Guy has a requirement: “It must do full-screen video on a mobile phone handset.”
Technical Girl throws up her hands: “Do these people understand nothing?  You can’t handle that much bandwidth on the phone.”

Actually, what Concept Guy means is that he wants video better than the postage-stamp stuff he’s seen so far.  He’s using the term “full-screen” because to him, it means “decent quality”.  What Technical Girl means is that the phone won’t handle her definition of full-screen, by which she means 30fps.  My job is to find the balancing point, to explain the tradeoffs between bandwidth, CPU power and quality to Concept Guy, so that he can say what will and won’t work, and to give Technical Girl an good, consistent idea of what an end-user wants from a videoplayer, so that she can come up with a way to meet the product needs.  Maybe her ideas will be better than the original concept.

It goes further; as the development continues, inevitably there are technical compromises that must be made as things turn out not to be quite as simple to do as first thought.  And from the product side, once a product begins to be real and people play with prototypes, the idea of what it is that can be sold changes, subtly.  This in turn affects what’s being built.  The trick is to find the cost-effective, commercially-driven way to solve the technical problems.  Feature isn’t working?  Banging your head against a brick wall of code?  Well, let’s take a look at what the feature is for.  Maybe it’s not required in the first version.  Maybe there’s another way altogether to deliver the same value to the user.

Of course, this is self-evident stuff, hardly needing to be written down.  But if that’s true, why is it that so many organizations seem to have these same familiar cultural gaps between the worlds of product and technology?  Is it, perhaps, because the job of translating between them isn’t seen as valuable?  I’ve been making a pretty good living doing it for more than ten years now, and I wouldn’t want to do anything else.

I’m off to get some new business cards printed.  They say “Cultural Ambassador From The World Of The Geeks”.  Or just “Translator”.

What You Mean Isn’t What I Mean

Back from holiday, and dealing with subtle incompatibilities.

I have a new toy, a Netgear MP101.  Like many people who hang around technology, all the music I listen to lives on hard drives these days.  The various PCs around the house all stream mp3s over the cables or WLAN, and all very fine and dandy it is too.  However, I’ve just bought a hi-fi to go in the conservatory.  The shiny new conservatory which must not (by order of those with more domestic clout than I) be sullied with wires, especially of the CAT5 variety and must definitely not contain any sort of computing apparatus.

So I sets out to connect the MP101 to my network.  I tend to use D-Link kit, not after any sort of rigorous testing regime but because I’ve got used to it and it works for me (and for the several clients I’ve installed WLANs for).  I also use WEP as a matter of course, with a nice ASCII key.  So, of course, I dial the same key into the MP101.  And it fails to connect.  I turn off WEP to test; all is well.  Turn it back on; no connection.  Check the keys – identical.  Try a new key.  Still no connection.

At this point I’m sure you’re thinking as I did; Google is your friend, so I did a little research and found nothing about problems with Netgear and DLink kit.  But I did find some vaguely related discussions about the algorithms for generating keys from ASCII and how they’re not standardized…

Surely enough, that was it.  DLink and Netgear each have their own way of deriving a 128-bit key from an ASCII sequence, and they’re not the same.  Luckily enough, the web interface to my DLink DWL2000 lets me see the hex value for the key – punch that into the MP101 and lo!  All connects.  Let joy be unconfined, etc.

As an aside; this is the sort of asinine annoyance that really gets to me when dealing with technology.  It wouldn’t have been that difficult for there to have been a standard, or even for the various vendors to make their kit try more than one algorithm (if key doesn’t work, try another algorithm, then another, etc).  The entire point of having an ASCII key is to make life easier for the poor unfortunate non-technical-specialist who has to buy and install this stuff.  If I had to point a finger at one way in which self-interest makes life worse for geeks, it’d be this sad attempt at vendor lock-in.  On the other hand, never ascribe to malice that which is explained by incompetence[1] – I suspect that it’s actually quick-and-dirty product development that’s to blame.

Then, of course, the firewall on XP SP2 blocks the media server so the MP101 can see the tracks but not play them… but that’s another story, and yet another reason to move the server functionality to a Linux box with RAID drives.  Another project for the long winter evenings that draw ever nearer.  I may even go and think about whilst listening to streamed music…

[1] I attribute this quote to Jerry Pournelle, but others claim different sources.  No matter; it’s a good maxim whoever coined it.