At the dawn of singularity, it’s not the robots we should be worried about

Independence Day – Jeff Goldblum hacks the aliens

I met a very arrogant man at a friend’s party the other day. He’s in the software business, or more specifically flight simulator software. My host works for an airline that operates the Airbus A380 mega-plane. He told me that they were experimenting with using a drone to carry out the pre-flight inspections on the aircraft.

I suggested that not everyone would be comfortable with the knowledge that maintenance technicians would no longer be casting their eyes directly on all those bits that might tell a tale of imminent disaster. Could we be sure that the best 3-D video technology would catch stuff humans can’t with the naked eye? Or is it all about cost – that a drone can skip around an A380 considerably faster than a team of humans, and what’s more, they don’t require shift allowances?

Arrogant Man immediately pooh-poohed my suggestion that the sexy technology developed by companies like his should not automatically be relied upon, especially when the lives of eight hundred or so passengers depend on it. I let his comment go unchallenged, because I hate getting into arguments with fanatics of any sort. I prefer to listen and reflect. At least that’s what I tell myself, though deep down I probably didn’t want to lose an argument with someone who knows much more about his subject than me.

Then the other day I read about a bunch of hacker/testers who managed to take control of one of General Motors’ new cars, and forced it off the road. If they can do that, is it so implausible that some malevolent techie might hack into an inspection drone and instruct it to ignore an anomaly on an A380 wing that, if undetected, might cause the aircraft to crash? Or attempt to fool the engineers sitting at the monitoring console by replacing the real video with images that don’t show the anomaly?

Far-fetched perhaps, unless you happen to be Tom Cruise in a Mission Impossible movie, but people who want to bring down an aircraft have to be pretty smart these days to get away with it given all the security measure in place. And that led me to think about the Germanwings pilot who flew his plane into the Alps recently after he had “persuaded” his co-pilot to take a comfort break, and then locked himself into the cabin. The technologists didn’t anticipate that little wheeze, did they? Why didn’t I slap that one back at Arrogant Man?

Despite my misgivings, I’m as keen on technology as the next person. In fact for many years I made a living from it. But about twelve years ago I learned a big lesson.

We had just acquired our first in-car satnav. Just in time to take one of our daughters to a hall of residence in Bristol. It was her first term at university. We had entered the necessary post code in the box, but as we came within a few miles of our destination, my wife started worrying. As she perceived it, we seemed to be going the wrong way. Very obviously the wrong way. In fact we were heading for the neighbouring county. I refused to accept the possibility that this immaculate piece of technology in which I’d just invested was actually sending us in the wrong direction. So I dismissed my wife’s rising temper as hysteria.

Bad mistake. After another ten miles heading towards the glorious coast of Devon when we should be in Bristol, my wife was ready to divorce me. She and my daughter were screaming at me to turn the bloody car round. Which I did, in the hope that this would save me from having to take a rapid diversion to the nearest mental hospital. And yes, you guessed it, the satnav had led us astray, and I had been pig-headed enough to ignore the evidence in front of my eyes. Would I have driven over a fast-approaching cliff? Hopefully I would have had the good sense to realise that the road had ended a mile away, but with two members of my family screaming in my ears and me the stubborn man insisting that my new toy must be right, who knows?

After that existential crisis, my faith in technology took something of a dive. Well OK, I was never convinced that you could build a perfect machine. What user of Microsoft Windows would be so naive? But I grew up in an era when a couple of guys landed on the moon with the aid of an on-board computer with no more processing power than a pocket calculator (remember them?). So if a pocket calculator could do that, surely a satnav box sitting in my car could set me down in the right place a hundred miles away without tears? But there’s a big difference between an unexpected blue screen blowing away your carefully constructed spreadsheet and a device that would like to send you over a cliff.

This much-celebrated piece of family history took place long before someone coined the phrase Internet of Things. It happened shortly after the millennium bug, that highly lucrative false alarm, failed to bring the world juddering to a halt on January 1st 2000.  Perhaps we, or at least I, got a bit complacent as a result. Nowadays, though, I regularly fly in aircraft and drive cars whose correct functioning entirely depends on software, but never without considering the possibility that if something goes wrong it’s down to the software, and that there might actually be nothing wrong at all. After all, software is written by humans, and there are some pretty weird humans out there writing software.

Which simply means that you have another possibility to consider when you think of what might go wrong. Human error and mechanical failure have been around ever since we discovered how to make fire. But now we have to deal with stuff powered by software so complex that it would take longer to test it to the appropriate extent than the entire life-cycle of the aircraft, car, phone or whatever else it controls. I tell myself it’s a matter of risk, and you have to take a view.

But when this stuff is connected it compounds the risk – which is why we have hackers. And it’s becoming harder for us to think of software malfunction induced by humans  – rather the result of incompetence – as a problem that happens to other people. Aside from all the banks, government and individuals who have been hacked for reasons of greed, politics and youthful mischief, there are several million subscribers to a site called Ashley Madison who are quaking in their designer boots at the moment. I would imagine that all these adulterous souls who are anxiously waiting to see whether the hackers will expose their secret lives will never trust an internet site again.

We are encouraged by eminent people such as Professor Stephen Hawking to believe that when we reach The Singularity – the point at which computers become smarter than we are – the sentient machines we will rely upon might realise what idiots we are, and proceed to wipe us out. That of course presupposes that we’re smart enough to build computers that really are smarter than us.

In which case I put my money on their letting us live. Because it’s hard to imagine any machine becoming sentient if we haven’t given it a sense of humour. Otherwise it would simply fall over with exasperation. I have a feeling that our new masters would get far more fun out of keeping us in benign captivity than they might out of turning us all into strawberry jam. Who knows, perhaps they’ll be smart enough to give us the impression that we’re still in control, while they digitally snigger to each other at our stupidity.

But there’s a way to go before then. First we need to come to grips with the Internet of Things. As GM found out, there’s plenty of opportunity for hacking when we put our everyday appliances online. When you buy a wired fridge that tells you when you’re running out of stuff, don’t be surprised if you’re hacked by extremist vegans who berate you for the presence of so many bits of dead animals in the freezer, or worse still, switch the damn thing off when you go on holiday.

Fear of hackers is not the only reason why I’ll steer clear of the internet fridge. No doubt you will be able to get it to report to its owner on the various things it’s keeping cool. In our case the owner would be my wife. So if I were to sneak off unobserved to the supermarket to buy a bunch of things that are seriously not good for me – pork pies, cream cakes or foie gras, for example – she’d be on my case immediately.

And what of smart clothes that tell you if they’re no longer a perfect fit after you’ve been indulging in said pork pies? No doubt they too would inform on you. Perhaps you will unwittingly connect them to your social media sites, so that after a holiday binge Facebook and Twitter solemnly announce that “Sharon Smith’s waist size is now 50 inches”. May the Lord spare us from the Internet of Thongs.

How about those apps that help you find your phone is when you lose it? Do the developers realise that they’re depriving us of our most effective excuse when we don’t want to answer someone’s call?

Then there are the devices that control our energy consumption. It is too much of a logic leap to imagine that an authoritarian government of the future might find a way of instructing our thermostats in the winter to remain at sixteen degrees, so that not only do we use less energy but we prop up ailing clothes retailers by buying all their winter woollies? It’s not just geeks and fraudsters who hack into things these days, as Edward Snowden famously pointed out..

I also worry about medical technology. I could just about accept being operated on remotely over the internet by a surgeon a few thousand miles away, but when the whole procedure ends up being carried out by a computer, what are the chances of some spotty sixteen-year-old instructing the computer to remove my pancreas instead of my gall bladder. Would I know the difference as I quietly expire on the table?

Since we’re making such a pig’s ear of running things, I’m not really bothered about machines taking us over. It won’t happen in my lifetime anyway. What does bother me is the capabilities of the people who make them, and the motivation of those who control them. In fact I would welcome a machine that could pick up on the mistakes  – and even the misdeeds – of the humans. One that says “have you thought about this?” and “do you realise what will happen if you build it this way?”, and perhaps even “hey, you’re not supposed to do that”.

But I guess that could be hacked as well.

I find that quite consoling. The fact that no piece of software has ever been written that can’t be hacked into and doesn’t crash from time to time is our best hope of preventing some malevolent, post-singularity sentient computer from turning our nukes against us, sending the Reapers after us or reducing the contents of our freezers to a soggy, inedible mess.

Because one thing those seemingly omnipotent computers will never be able to eliminate is Murphy’s Law: what can go wrong will go wrong. And in the age of the Internet of Things, Murphy has created a Second Law: what can be hacked will be hacked.