A presentation to the London Futurists Conference, April 2014
I grew up reading science fiction. This was handy when I got to university, because science fiction is essentially philosophy in fancy dress. But it wasn’t until the year 2000, two decades after I graduated, that I came across Ray Kurzweil, who made me consider the astounding idea that conscious, super-intelligent machines might be created in my lifetime. As a science fiction fan I’d long thought they would arrive one day, but I’d assumed it would be centuries away, long after my death. I’ve been thinking about the implications of that idea ever since, and wondering how to get other people thinking about it too.
Another decade later I semi-retired, which gave me the time to try to do something about it, by writing a novel on the subject. It’s called Pandora’s Brain, and my agent is currently pitching it to publishers.
These are the six questions I’ve been thinking about all that time.
- Can we create a human-level artificial intelligence, an artificial general intelligence (AGI)?
- If so, when?
- Will AGI lead to super-intelligence?
- If super-intelligence arrives, will we like it?
- Can we upload our minds into computers?
- Can we de-risk the arrival of super-intelligence?
Let’s start with some definitions. I take the terms “human-level Artificial Intelligence”, “strong AI”, and “artificial general intelligence” to mean pretty much the same thing: an AI which has all the cognitive abilities that a human has.
Intelligence is essentially about solving various kinds of problems, whereas consciousness is about personal experience. It’s the sense you have of blue, of heat, and of your personal identity as a consistent thing over time. Our intuition is that intelligence and consciousness are correlated, but we don’t know how far.
We can’t define consciousness, but we all know the test. The Turing Test is a brilliant idea from a brilliant man. I think the people who deny its validity are wrong: it is after all how we determine whether other people are conscious persons or not.
The potential development of greatest interest to me is Super-intelligence. If and when an AGI is built and then becomes a super-intelligence everything will change, and the world will become either wonderful or disastrous. The arrival of super-intelligence will be the most important development since the invention of agriculture. It will quite literally determine the future of humanity. It could happen slowly, or very quickly, in the form of an intelligence explosion. And the incredible things is that people in this room today may witness it.
I don’t use the word Singularity because it means many different things to different people, and also because it has acquired unfortunate pseudo-religious overtones.
1. Can we create a human-level artificial intelligence, an artificial general intelligence (AGI)?
Let’s not kid ourselves: it is very, very hard. Imagine giving every inhabitant of New York City 1,000 strings. Each string is attached to another inhabitant, and carries up to 200 signals every second, travelling at 300 metres a second. Now multiply that city by ten thousand. That is a reasonable model of a single human brain – the amazing 3 1/2 pounds of gloop inside your skull.
And that’s just the hardware. If consciousness was simply an emergent property of sufficiently complicated information processing then the internet should have woken up by now – as it does in Robert Sawyer’s Wake, Watch and Wonder trilogy. In addition to the hardware we also need the software – the wiring diagram, and instructions for how to initiate the firing.
There are three ways to get at this. The first is Reverse Engineering, in which an existing brain is analysed either destructively – by cutting it into very thin slices – or non-destructively, by increasingly advanced scanning technologies. This is what the EU-funded Human Brain Project led by Henry Markram is doing, and they have just announced an important collaboration with the American version, President Obama’s BRAIN project.
The second approach is Incremental Development, where you assemble the best existing AI systems that do search, natural language processing, deep learning and so on, you feed them more and more data, and you use trial and error to improve them. This is what Google is doing, it’s what IBM is doing with Watson, and it is happening in many other labs around the world.
An important feature of both these approaches is that you do not need to know exactly how a human brain produces a mind in order to make your own artificial one. Some people, including Marvin Minsky and Gary Marcus, claim that you can’t manufacture a mind unless you know how one works beforehand. I think this is an unproven assertion.
The third approach does involve understanding how the brain works. You develop a comprehensive theory of mind and then build your brain. This looks even harder.
Will any of these approaches work? I can’t tell you for certain, and I don’t believe anyone else can either. To coin a phrase, the science of brain building is most definitely not settled.
Here are some of the reasons that people give for why it might fail. Maybe brain builders have to capture time-series data, or model the brain in more granular detail than the neuronal level. But these requirements would simply delay AGI rather than prevent it forever. The third and fourth reasons seem implausible to me.
So I’ve yet to hear a good reason why we can’t build an AGI, and there certainly is a lot of money being spent trying.
2. If so, when?
The whole project, both hardware and software, is driven by Moore’s Law – the observation that the performance of computers doubles every 18 months. I don’t think we should be too upset by the fact that Gordon Moore himself is sceptical about AGI being created any time soon.
The only thing we know about forecasts is that they are wrong; we just don’t know by how much, or in which direction. History’s most famous exaggerated forecast was by Robert Malthus, and of course there have also been famous under-forecasts, such as Alexander Bell’s suggestion that there might one day be a telephone in every city, and Thomas Watson’s comment that the global market for computers might be as big as five. As if to underline how unreliable this whole forecasting business is, neither of these men actually said what has been attributed to them.
People argue about how much longer Moore’s Law will continue. Well we know the brain processes information at the exaflop scale – that’s 1 with 18 noughts after it – we will soon have exaflop scale computing. The Very Large Array telescope system, with dishes in Australia and South Africa, and processing at Jodrell Bank near Manchester, will operate at that scale. According to one report it will process half as much data in four years time as the entire internet does today. That’s a nice illustration of the power of exponentials.
Quantum computing has long been seen as a potential saviour for Moore’s Law. There are some important caveats, but it’s looking increasingly likely that D-Wave is manufacturing genuine quantum computers.
Ray Kurzweil has been saying for a long time now that 2029 is the due date for AGI, with uploading coming along 16 years later. A poll by the Oxford Institute for the Future of Humanity found that half the people working in the field think it will be before 2050. We should have observable progress by 2025, and a lot more people will have woken up to the likelihood of it happening soon. That increase in awareness will itself be very important, as I’ll argue shortly.
3. Will AGI lead to super-intelligence?
Almost certainly, yes. An AGI can be expanded, speeded up and improved in ways that a human brain cannot. What we don’t know is whether it will be fast or slow. The novelist and computer scientist Ramez Naam has recently argued that it will be slow, taking years rather than days or weeks. But he doesn’t really know, and neither does anyone else.
4. If super-intelligence arrives, will we like it?
The impact will certainly be dramatic. The arrival of super-intelligence will determine the future of humanity. The range of possible outcomes is extreme, ranging from the sublime:
- We upload and become godlike
- They help us, solving all our material and psychological problems
…to the disappointing:
- They leave
- We become Pets
… to the pretty grim:
- We become farm animals
- We become zoo animals
… to the really bad:
- We become slaves
- We wilt
… to the absolutely horrifying:
- Humane extermination
- Brutal extermination
- Eternal torture
It amazes me how many people claim to know what the outcome will be. Some argue that a super-intelligence will be more civilised than us, and therefore benign. But that is mere supposition. Maybe a super-intelligence will think that we are a potential threat and decide to get its retaliation in first with a devastating pre-emptive strike. Or maybe, in Eliezer Yudkovsky’s classic phrase, it will neither like us nor dislike us, but will simply have better uses for the atoms that we are made up of.
Other people argue that a super-intelligence will necessarily be damaging to us unless we take steps to prevent that. This too is mere supposition.
The truth is, we simply do not know what will happen, however brilliant the advocates of particular outcomes may be.
One thing we do know is that we cannot stop the march towards super-intelligence. The advantage of owning one is simply too great for any business, any government, and above all, any army. Relinquishment will not work.
It is therefore amazing how few people are thinking seriously about these risks. One of the leading organisations engaged in the task, the Machine Intelligence Research Institute, is located exactly where you would expect to find it, in Northern California. Surprisingly, the two other leading ones are in the UK – one at Oxford and the other at Cambridge.
Almost by definition, we can neither predict nor pre-determine the goals of a super-intelligence. The whole point of Asimov’s stories was that his Three Laws wouldn’t work. How could we hope to programme a system of ethical rules into an AGI when we’re no closer to agreement on ethics than were the ancient Greeks?
Personally I think our best bet may be an Oracle AI, or an AI in a box – meaning an AI which is unable to influence the outside world directly. Some people think this is impossible, but there are some talented groups working on it.
5. Can we upload our minds into computers?
The most optimistic scenarios is that we upload our minds into computers, merge with the super-intelligence, and zoom off to explore the universe together, like gods. Well, let’s hope so!
But uploading isn’t just an optimistic scenario. It may be the only way that we humans can enjoy the full benefits of super-intelligence. Unless we merge with it, we are simply creating our successors and then probably fading into oblivion – or worse.
Uploading certainly won’t be easy. Reproducing an existing mind without destroying it will require advanced nanotechnology for starters.
But is uploading even possible – philosophically, never mind technologically? Some people argue that uploading doesn’t preserve you as a person – it merely copies you. In which case it would be better to call it sideloading. Maybe beaming Captain Kirk up is actually killing him here and giving life to a copy of him over there.
This debate actually goes back to the ancient Greeks, who called it the Ship of Theseus problem. For my money the debate is more-or-less settled by the thought experiment where your carbon brain is gradually replaced, neuron by neuron, with silicon. The resulting entity would certainly insist it was you, and it is impossible to say at what point during the process it would have stopped being you.
But this leaves us with some paradoxes. If a mind is successfully uploaded without destroying the original brain you end up with two people, both claiming to be the same person. Which one gets the kids, the house, and the record collection? There are serious questions here, but I think there are satisfactory answers.
To sum up so far, we don’t know whether we can create an artificial general intelligence and then a super-intelligence. The science is not settled. But they do seem plausible developments, which may arrive relatively soon. What we can say is that if it is possible, super-intelligence is very, very risky.
6. Can we de-risk the arrival of super-intelligence?
Let’s review what could go wrong. If it turns out that AGI is impossible, or doesn’t lead to super-intelligence, well there will be a lot of disappointed geeks – including me, despite my concerns about super-intelligence – but the human race will carry on. The other two scenarios are the ones we really need to avoid. We don’t want the super-intelligence to harm us either deliberately or accidentally. And if the only way for us humans to fully the enjoy the benefits of super-intelligence is to merge with it, it would also be tragic if unloading turns out to be impossible. Or if it takes so long to become affordable that a whole generation dies while waiting. How awful to be one of the last mortal humans, when you know that indefinite life lies just beyond your reach!
So I believe that we need to launch two new Apollo Projects. We already have one in effect that is trying to create an AGI and then a super-intelligence. The first additional one is obvious: we need to work on making the arrival of the super-intelligence beneficial for humans. This is known as Friendly AI, or FAI.
The second Apollo Project is to abolish death, or rather to make it optional. The gap between the creation of the first AGI and the development of uploading is likely to be years. During those years, millions of humans will die – 150,000 every single day, just like today. To bridge that gap we must achieve longevity escape velocity and develop brain preservation techniques (cryonics or brain plastination or both). We should be doing this anyway, but we should especially be doing it considering that uploading may be – in historical terms – just around the corner.
These are expensive projects. Some of the money will come from very rich individuals and businesses. Google’s Calico and Craig Venter’s Human Longevity Institute are exciting ventures. But most of it must come from governments. Governments will only provide it if people demand it. People will only demand it if they understand it. They will only understand it if they talk about it.
So we need to get people talking about super-intelligence. Not as some kind of rapture for nerds, but as a future which is distinctly possible, but highly uncertain, and full of risk. A future that we need as many smart people as possible to be thinking about and working on. Super-intelligence may well be the future of humanity – and it is a future we need to prepare for.