(soft piano music)
Of the 40 or 45 million centuries
of the Earth's history, this one is very special.
This is the first century when human beings,
one species, have the future of the planet in their hands.
Because the stakes are very high
there is a new category of risks,
which are certainly going to be global
and would involve some sort of catastrophic setback
to civilization.
(digital computer-like sounds)
Existential Risk is a risk or a threat
that might wipe out humanity at some point in the future.
Or at least wiping out the kind of
high tech, civilized lifestyle that we've got used to
over the past few centuries,
that in effect setting us back to the dark ages.
The suffering that would be caused to billions of humans
in the process of one of those threats evolving.
So even if it didn't get to the stage of wiping out humanity
that huge amount of suffering is a big interest.
(soft orchestral music)
Now, new technologies, bio, cyber and AI,
are empowering us
and they have huge potential for good,
but also by error or by terror,
they could cause serious catastrophes,
so potentially catastrophic that once is too often.
So these are the kind of things
which are on the agenda of our center.
What we are trying to do as a research group
is effectively to build bridges,
open lines of communication and collaboration
between the leading researchers,
whether they be in academia or in industry,
the academics who are thinking about the economic,
social, philosophical, ethical implications,
the risk, and the policymakers
who very much need to have a deeper understanding
of these things.
(digital beeping)
The world's never been as connected as it is now.
We are so interconnected that anything really serious
in one part of the world will cascade globally,
computer networks and cascade a cyber attack globally.
Pandemics can spread at the speed of a jet aircraft.
Panic and rumor can spread literally at the speed of light
through social media.
It's amazing how little effort is devoted
to these major issues.
I must admit that I do have sleepless nights
over these issues.
I think there are a number of threats
that are quite salient.
I believe that the threat of nuclear war
has never really gone away.
The world population is much higher than it was.
50 years ago it was about 3 billion,
it's now 7.4 billion,
and almost certain to rise to 9 billion by mid-century.
So we are having a much heavier footprint on the planet
with the risk of tipping points.
We are wiping out species at a faster rate
than we ever have before,
and we simply don't know what the effects
on our own wellbeing and survival will be from that.
Climate change is a very real threat.
One of the big questions scientifically is,
how bad would it be?
How will we be able to respond to it
and will we be able to take the steps in time?
Advances in biotechnology will help us cure many diseases
and increase quality of life for people,
but there is always a potential
that could lead to accidental releases of organisms,
such as modified influenza virus,
or the development of ever more powerful bio weapons.
Since we're getting to the existential level of risk,
I think the top one currently is,
if States were to have biological warfare programs,
because at the moment terrorists can't get
to that level of sophistication that States can.
Even a few people could create a bio weapon
that could produce global catastrophe,
and this is why things are much more dangerous now
than when we had just nuclear for us to worry about.
The equipment that's needed is small scale and dual use
available in many university labs,
and it's very hard to imagine
any kind of verification regime
that could ensure that none of it
is used for nefarious purposes or negligently.
(digital music)
Artificial intelligence
can be a bit of a tricky thing to define.
A broad definition would be
that artificial intelligence is any cognitive capability
that we normally associate with being within the domain
of what humans are only able to do.
The very interesting thing about artificial intelligence
is that it can be seen as a general purpose technology,
and that advances in artificial intelligence
will allow us to make better progress
on a range of other technologies.
It's wonderful if you can talk on your phone
in one language, and it comes out in literate speech
in a different language.
Things like that, that can be feasible.
It's probably clear that large networks and systems,
like electric power grids, can be coped with better,
by AI than by human controllers.
There are many, many aspects of healthcare
that we might be able to improve
with artificial intelligence.
So, examples to include automated ways of diagnosing tumors.
But if this was the kind of thing
that a smartphone could do, powered by an AI system,
suddenly you can bring that sort of treatment
to a much wider range of people.
Right now most of the artificial intelligence
that we see in the world
comes under the narrow definition
in that these systems are designed
to solve a particular problem
within a particularly set environment
but aren't necessarily able to do anything else.
So, Deep Blue is able to play chess better
than a grand master, it's not able to play a simpler game
like checkers.
It's certainly not able to make a cup of tea
or to tie its shoelaces.
There's a different idea,
which is that of artificial general intelligence,
and this refers to the idea of an intelligence
that is much broader in its scope and capability,
and right now that's the domain of humans and animals.
To give an example, what if you had a Mars rover
that rather than just going through a set of steps,
you could just send it out and say,
"Look for something interesting.
"Design a few scientific hypothesis,
"carry out those experiments and figure out
"what we should know about Mars."
If you were able to develop a system like that,
it would be tremendously powerful and important.
And right now research in this direction
is still at an early stage.
We're going to be sharing the planet
with a lot of stuff, which is not biological,
which on many dimensions is as smart as we are,
but it comes with a note of caution.
A lot of people within the research community
are worried about potential application
of artificial intelligence to various aspects of warfare,
both in the design of autonomous weapons
that would be able to take independent kill decisions
but also in the more kind of systemic issues
of AI systems managing lots of information
around a wartime situation.
If we were bringing these systems into the world
in various safety critical settings,
like self-driving cars, medical diagnostics and so on,
we need to know how they're coming to the answers
that they are.
It's not just enough to say,
"Well, it's better than a human 90% of the time."
There is the possibility
and is what some people worry about,
that a very intelligent machine
which has access to the internet of things,
will be able to do things which are unpredictable.
It may find ways to sort of work around your off switch.
How do you turn a broad amorphous concept
like human morality, into something
that can be instilled crisply into a new system?
Can you come up with a moral code
that is sufficiently satisfactory for all of humanity?
Right now this is a deep question of debate
within both the philosophical and technological community.
Then of course is a much deeper question
which is a philosophical one,
of the extent to which these machines
will develop consciousness.
Some people think consciousness is something
which is peculiar (muffled) hardware in human skulls.
Others think it's something which emerges
when you get up to a certain level of complexity.
I think it probably will require a change
of the definition of what it means to be human.
Although one might argue that the definition
of what it is to be human has been changing
over the course of our history.
Right now they're developing bio technologies
that allow us to correct errors in our genome
and remove horrible diseases
that previous generations have taken for granted.
I would regard that as a change of what it is to be human.
Once we bring artificial intelligence in,
that will perhaps change what it is to be human
even more radically.
At least in the case of AI,
when you start to talk about extinction,
there's a number of possibilities you need to consider,
the sort of bleakest end,
that our species might come to an end with no successor.
But then there are a range of less bleak possibilities,
which is the purely biological species
eventually comes to the end and there are no humans around
who don't have access to this enabling technology
that makes, in some sense something which is unseen.
This is going to be a long-term threat.
There are, some people who of course think
that we need to worry already about AI
and that the development in that area needs to be regulated
in the same way as everyone agrees biotech needs
to be regulated.
At a time in which regulation
is not a political flavor of the month.
But I think that's another reason
for trying to develop a broad community
with as much cooperation in it at this early stage
to make it easier
when we get to some of those tricky questions.
All scientists who work on subjects
with any societal impact,
should be concerned about that impact,
and so decisions about how science is applied
should not be made just by scientists,
because scientists shouldn't play God.
They ought to involve societal judgements,
involving the great religions and involving governments.
Over and above that,
scientists have an extra responsibility.
They should try all they can to ensure that their work
has benign applications,
and to try and avoid the dangerous applications.
We can't fall back on a model
that involves making mistakes, learning from those mistakes
and not making the same mistakes in the future
or making different mistakes,
because some of the mistakes we might potentially make,
will be of such consequence
that we won't really be able to recover well from them,
or the cost will simply be too high.
(soft piano music)
I'm very much an optimist
and one of the things that's been really positive to me
is just seeing how much people care about thinking
in a longterm way about the future,
seeing how much enthusiasm there is from research leaders
to work together with people from the different backgrounds
to think about how to beneficially guide technologies.
It won't happen by default,
which is why I think it's important that there be
M centers like ours,
thinking about these things ahead of time.
(digital tones)