Hello. My name's Mike Wooldridge.
I'm a human, studying artificial intelligence.
- Hello, I'm Ai-Da, the artist robot,
and I create art that helps people
consider the future.
- The robot, Ai-Da, is painting my portrait,
but this process takes a while, so we started a few minutes ago.
We're going to come back at the end of the lecture
and see what Ai-Da's managed to do.
Ai-Da, can I go now?
- Yes. - Thank you.
Hello, and welcome to the first
of the three Royal Institution Christmas Lectures,
an event supported by CGI,
in which, we'll see what artificial intelligence is, how it works
and how it's shaping our world.
We're entering an era where intelligent machines
can do things that seemed impossible just a few years ago.
And the pace of change of this new AI
is surprising even the scientists that built it.
And this new AI was made possible by artificial neural networks,
inspired by the structure of the brain.
And our story starts with this.
This is a human brain.
Of all the wonders in the universe,
this is the greatest.
Forget about black holes and neutron stars and supernovas -
you are looking at the most incredible thing in all of creation.
All of human experience has its origins here.
Every thought, every dream, every hope, every love.
Everything it means to be human, from Shakespeare to Taylor Swift,
all has its origins here.
It's the ultimate triumph of evolution.
It's a marvel that we are still nowhere close
to properly understanding.
And human brains, like this one and like yours,
are unimaginably complex.
They have huge numbers of cells
that are connected to one another in enormous networks.
And those cells are called neurons.
We estimate that a typical brain - a typical human brain,
like yours or this -
has about 86 billion of them.
And neurons are the basic building blocks of human intelligence.
Somehow, everything that we are emerges from what they do
and the way that they interact with each other.
So it almost seems like heresy to suggest that
we can build a machine
which does something like
this incredible organ does.
But that is the dream of artificial intelligence.
But can we really recreate human intelligence with this?
This is an Intelligence Processing Unit, an IPU.
This is a bit like one of the processors in your home computer,
but it's designed to process
enormous amounts of data in parallel,
and it turns out that is perfect
for artificial intelligence.
Now, IPUs like this are not brains,
they don't have neurons, but they are built from huge numbers
of very simple computational devices, transistors.
This IPU, this state-of-the-art IPU
has about 60 billion transistors.
Remember, compare that to the something like 86 billion neurons
in the brain, it doesn't sound
that different.
So how far are we from using IPUs
to recreate something like human intelligence?
Before we go much further,
we thought we'd show you an example
of the kind of incredible thing
that artificial intelligence is capable of today.
For this, please welcome our first guest, Sarah de Lagarde.
APPLAUSE
- Hello. - Hello, Sarah, and welcome. Welcome to the Christmas Lectures.
Now, Sarah, you have a truly remarkable story.
Will you share that with us? - Yes, of course.
So, just about 12 months ago,
I was in a terrible train accident
and, as a result, I lost my right leg below the knee
and I lost my arm above the elbow. - Oh, wow.
And so, you've been fitted with a prosthetic arm.
Those are nothing new, but what makes this one special?
- So this one's pretty special
because this is what we call a bionic arm,
and it's powered by artificial intelligence.
- OK. Now, I understand, at this point,
you've got a party trick to show us... - Oh, yes.
- ..involving a can. Is that right?
- Yes. Well, the first thing I wanted to show you
is that I've got a movable wrist
that can turn all the way round. - AUDIENCE MURMURS
- And then, it's got a little bit
of power as well. So let me try
to crush this can for you.
- CAN CRUNCHES - Oh, oh, oh! - OK! - It does work!
- THEY LAUGH That really is quite a party trick.
So, how does your arm know how to do that?
- Well, basically, the way it works
is that I think about the movement that I want to make,
and I twitch a series of muscles,
and the muscle twitches are being recorded by 16 electrodes
that are positioned inside the socket,
and then transform that into electric impulses
that run through the forearm and powers the Covvi hand,
my bionic hand. - OK.
And where exactly does the AI come into the picture?
- So the AI is a lot of
data recording. It's basically...
Every time I use the arm, every time
I make a movement, it records it.
It starts to learn how I use the arm
and it starts to predict the movements that I want to make,
including rotating it, like that. - THEY LAUGH
I could watch that all night.
OK, so the AI is being trained on your body,
but is your body and your brain, are they adapting to the AI?
- So, I did have a lot of training to understand,
because it's very different to think about a movement,
and it's more difficult to control it when it is bionic.
But when I take the arm off at night
and I recharge it, like an iPhone,
- THEY CHUCKLE then I miss it.
How would your life have been like without this arm?
- Well, I have to say that this world is being designed
for people with two hands. Everything.
Think about the zipper on your coat, for example,
or on your bag, on your rucksack. Think about a parcel, packaging.
For everything, you need to have two hands - one to hold it down
and the other one to rip off the lid on something,
and with the arm, it helps. - OK.
Now, this technology is evolving. AI is moving very fast.
What do you hope this arm is going to be able to do in the future?
- So, I really hope that this is going to evolve,
become even more performant,
because the idea is that, at the moment,
it takes me about ten seconds to think about a movement
and then execute it. That's the AI part.
It hopefully will reduce that time
and become instantaneous, like my other hand.
- So, as the AI gets better and better,
that's going to help you in your life?
- And I really dream that, one day, I can go back to riding a bike
and, if I'm lucky, maybe I can drive a car again.
- Sarah, that is a truly remarkable story.
Thank you so much for sharing it with us. Thank you, Sarah.
So, AI is already capable of amazing things,
and it's helping Sarah every day.
And in this example, it's working with Sarah's body.
But as AI gets better and better, some wonder,
"Might it eventually be able to think like a human?"
Now, this is a question which has been with us
ever since the origin of the first computers.
And the best-known answer to that
question was provided by this man.
This is Alan Turing.
You may have heard of Turing
through his code-breaking work
for the Allies in World War II,
but Turing was also one of the inventors of computing,
and one of the inventors of artificial intelligence.
And a test that he invented
has become one of the best-known ideas in AI.
Turing's test is a beautifully simple and elegant way
in which we can test machines for intelligence.
And we're going to carry out a live Turing test now.
I need a volunteer! Hands up to be a volunteer.
OK, you in the hat there. Come on down.
OK.
Come and stand here. Hello, what's your name?
- Xanthe. - Xanthe? OK. Thanks for joining us, Xanthe.
We are going to test you to see if you are a human being.
OK? That's nothing to be alarmed at. The team are going to
take you backstage. We're going to see you shortly.
Just follow their instructions. Thank you, Xanthe.
So, Turing suggested that if a human judge
is unable to tell the difference
between a human and a machine
by just having a conversation with them,
asking them questions and looking at the answers,
then we should assume that the machine has humanlike intelligence,
because we can't tell the difference.
And if we can't tell the difference, Turing said,
then it's redundant to argue about
whether the machine is really intelligent,
because we can't tell the difference.
And with Xanthe, we're going to run a live Turing test.
Now, behind us, we have two boxes, Box A and Box B.
One of those two boxes has our volunteer, Xanthe,
and the other holds a computer.
Now, the computer is running some advanced artificial intelligence,
something like ChatGPT,
and we've asked that artificial intelligence
to respond to questions in the way that a human teenager would.
Now, we asked all of you in advance to send in some questions,
and we had a look through those and selected what we thought
were the most interesting of those questions.
They're in this hat right now.
And what I'm going to do is I'm going to give you the hat.
You are going to be our questioneer.
When I ask you for a question,
I want you to stir up and give me one of those questions.
OK. So, first. Questioneer!
OK. And the first question is number two, Noah.
Noah, where are you?
There you are. OK. Take the microphone, Noah,
and then tell us your question.
- What is your favourite memory from when you were a child?
- OK.
So, human, artificial intelligence -
what is your favourite memory from when you were a child?
Go!
COMPUTATIONAL BEEPING
OK. We've got our answers.
Let's get the answer from Box B first.
OK. And let's get the answer from Box A.
OK. Good answers from both.
Let's get another question.
Stir the hat.
OK. Let's have one of the questions.
Thank you very much.
OK. This is question 11, from Oliver.
Oliver, where are you?
Oliver, up there.
OK.
Oliver, read your question.
- What was the best present you got on your last birthday?
Pass that microphone back.
OK. Human, artificial intelligence -
what was the best present you got on your last birthday?
Go!
COMPUTATIONAL BEEPING
OK. Box A, give us your answer, please.
OK. And Box B?
Who doesn't love a new phone?
So, questioneer, mix them up again.
OK, let's get one of these.
And this is from Tomohiro.
What was your question, please? Into the microphone.
- What are your dreams for the future
and how do you plan on achieving them?
- OK. Human, computer - what are your dreams for the future
and how do you plan on achieving them?
Go!
COMPUTATIONAL BEEPING
OK. We've got our answers.
We're going to go to Box B first.
Box B?
This is pretty good advice for life, folks.
You can take that away with you for free.
Box A?
LAUGHTER
OK. We're done with our questions.
Now, you are the judges -
you have to decide which is the human.
That's what we're asking you.
If you think the human is in Box A, put your hand up now.
OK?
That's quite a lot of you think the human is in Box A.
I'm looking up into the stalls, there's quite a lot there.
If you think the human is in Box B, put your hand up now.
OK, there's a couple of people.
I'm getting only a couple of hands,
so I think the overwhelming majority of you
think the human is in Box A, right?
Well, let's take a look.
Let's see what's in Box A.
And it's Xanthe!
APPLAUSE
Xanthe, that was an incredible set of answers. You've completely
convinced the audience. There is no doubt here about where they thought
the human being was. Were you pretending to be a robot,
or were you just being Xanthe?
- I was basically just being honest, yeah.
- OK, well, you are the only person in the whole room that's
been certified as human and overwhelmingly certified as human.
Thank you, Xanthe. You've been a star.
So we're going to take a look before Box B goes.
We're going to take a look and see what's inside Box B.
BEEPING
And Box B is our advanced artificial intelligence.
Well done to all of you who guessed correctly. Well done.
Important though it is, most AI scientists today don't think
that the Turing test is the end of the story
for artificial intelligence.
Even advanced AI can't necessarily pass the Turing test today.
Turing's test only looks at language,
and as we saw earlier with Sarah,
you can do infinitely more with AI
than just have a conversation with it.
But the surprising thing is that some things that we find
really hard as human beings turn out to be really easy for AI
and some things that are very hard for AI are incredibly easy for us.
So we're going to have a little illustration of this.
We're going to play the Can AI Do It? game, and I need a volunteer.
So let's have some hands up, some volunteers.
OK, you, come on down.
Stand here. What's your name?
- Niranjan. - Niranjan, OK. Thank you very much for joining us.
Now, we're going to play a little game and I'm going to ask you
to look at the screen, and what's going to happen is
I'm going to show you some tasks that we might want AI to do for us,
and I just want you to tell us whether you think AI can actually
do them or not. OK? Is that clear?
Yeah? OK, so, let's start with a maths problem.
OK, folks, look at this.
Hands up if you think you can do this in the audience. Anybody?
Couple of moderately...
Oh, yes, some very confident people up here.
What about you? Do you think you can do that one?
- I can't do that. - You can't do that one. OK.
So, turn round, face the audience.
So do you think that AI
can solve that equation?
- Yeah. - You think AI can solve that equation?
OK, well, you're absolutely right.
AI can solve it quite easily
and this, as it turns out,
is the answer.
Folks, you can check that when you get home.
That's your homework for tonight.
So most people - I think most of you in the audience -
find maths problems like that really quite hard.
Now, let's look at a problem that's a lot closer to home.
OK. What's this?
This is a messy bedroom.
Is that your bedroom?
- No. - No. I'm very glad to hear it.
But actually, that's an AI-produced image of a messy bedroom.
So do you get asked to tidy up your bedroom?
- Yeah. - Yeah, we all get asked... I get asked to tidy up my bedroom,
I'm nearly 60 years old.
OK, so do you think we have AI
today that could go into that room
and tidy it up?
- No. - No, you don't?
- No. - OK, does anybody think we have AI today that can do that?
A couple of people that can do it.
Well, no. Sorry, parents, but I'm afraid AI can't help you
with messy teenage bedrooms.
They're a tricky problem for AI to solve.
But isn't that weird?
You all looked at that equation and thought,
"Wow, that looks really hard,"
but looking at a messy bedroom, do you think it requires intelligence
to tidy up a bedroom? - No. - No. - A little bit, a little bit.
- A little bit of intelligence, but you don't have to be a genius.
You don't have to be Einstein, do you, to tidy up a bedroom? - No.
- No, you just have to want to do it, don't you?
OK, all right, so the really interesting thing, then,
is some things that we find easy are really hard for computers
and some things that are really easy
for computers are really, really hard for us.
OK. Thank you so much. And, well, do you want to
clear up your mess? - Can I take the football?
Ha! Take the football. - LAUGHTER
The football is yours.
We've learned an important lesson about AI.
Lots of the things that we can do,
things which we often take for granted,
are far beyond the reach of today's AI.
And yet, today's AI can easily do things that most of us
think are phenomenally hard.
The capabilities of AI today, as we saw with Sarah, are incredible,
but there is no AI at present that can do everything
which you all can do.
But what we know for sure is that the way that both human
and AI process our questions and come up with answers
can be compared, just like we did in the Turing test.
Now, if we want to build an intelligent machine, then maybe
it's useful to look at the one example that we actually have
of intelligence in the world today - us, human beings.
And in fact, the most successful approaches to contemporary AI
were indeed inspired by the structure of brains.
This is the idea at the heart of contemporary AI.
Basically, all the progress that we've seen in AI this century
is because artificial neural networks have started to work
on problems that are big enough to be really, really useful.
So to understand AI, perhaps it's going to help to know
a little bit more about what's inside our heads.
Now, for this next part, I'm going to show you how neurons fire,
and to do this, I need the help of the Royal Institution's
very own neuroscientist, Dr Daniel Glaser, and Dr Arjuna Nagendran.
APPLAUSE
Daniel, we're talking about neurons. Did you happen to bring a neuron
with you? - I did bring a neuron, Mike. Neuron is just a nerve cell
and it's a cell that produces electrical signals.
The neuron I've brought is in my spine and it's connected
to the muscle in my arm. It gets signals from my brain.
And what we're doing here is we're going to listen in to
the neural signals from my brain, from a nerve in my arm.
We're using an electrode which Dr Arjuna is going to put in,
which doesn't hurt at all, I'm told. - LAUGHTER
OK, Arjuna, can you tell us a little bit about what you use
this equipment for? - Yeah, so EMG or electromyography,
is used to give us an insight into nerve and muscle function,
and also, when things go wrong.
- So, let's get this ready.
- So, Daniel, just relax the arm.
- So that sound you can hear - RAPID CLICKING
is a few spikes from the neuron in my arm.
It's being controlled by my brain.
And if I start to straighten my arm a little bit more...
- CLICKING INTENSIFIES
- ..you get more spikes.
And that's because neurons either fire or they don't.
They don't have strength, they just have a number of pulses.
If I try really hard...
- CLICKING INTENSIFIES
- ..you get loads and loads of spikes.
So that's a neuron right there, Mike.
It's a device that either fires or it doesn't,
and it fired loads of times when I was trying hard.
- OK, and those little clicks, they're basically the sounds of
neurons firing? - They're sending pulses down their axon wire to show
that they're excited. - OK.
Thank you, Daniel. Thank you, Arjuna.
So now, we know what neurons are.
They are cells that relay electrical signals in our brains -
just like the brain we saw earlier - but also, in our bodies, as we saw
in the demo there with Daniel.
But what we're now going to see is what happens when you put
a bunch of neurons together.
OK. - So this is a microscope, Mike, and George is going to turn it on.
We've got a dish, if you can pass me one of those there,
which is inside it, and in it are human neurons.
Now, we haven't got them from grown-up people,
we've grown them in a lab. We've reprogrammed their DNA
using AI, Mike, and they've grown in this dish.
And because they're neurons, what they've done is they've connected
themselves up into networks.
They are firing all on their own. And if we look at the screen
over here, Mike, we should be able to see these neurons firing.
So those green dots that you can see
are human neurons
that have grown in the dish.
And if you look very carefully,
you can see little glimmers,
as they send signals to each other.
- Yep. - Did you see that firing there? - Yep. - And the electrical pulse there.
These are just neurons in a dish, but they can't help themselves.
They've connected themselves up together and they're firing. - OK.
So, what we've got is these neurons.
Each of these green dots is a neuron.
They're connected in networks
and they're sending signals to one another, just like we saw
in your muscles. That's a really beautiful demo.
Fuad, from Bit Bio, thank you so much for joining us.
George, from King's College, thank you so much for joining us.
Thank you, Daniel.
Now, all of the many billions of neurons in our brains are firing,
just like we saw those fluorescent neurons in the Petri dish firing.
But what is each individual neuron actually doing,
and how do they come together to create intelligence?
How can they make decisions?
To explain, please welcome scientist Steve Mould.
OK, Steve, what on earth is this contraption
that we're looking at here? - So this is a really simple
representation of a neural network.
This particular neural network
can identify pictures of mammoths. - OK. - Sounds impossible...
- It does sound a bit impossible. - ..but hopefully,
we'll get it working. We need the help of a volunteer, though.
- OK.
In the yellow cardigan there.
OK, what's your name? - Rahel.
- Hello. Well, welcome.
OK, Steve. - So, I'm going to show you a picture, I'll reveal it to you.
No-one else is going to see it. It's going to be one of
the images on the screen, here.
I'll just quickly reveal it to you. Are you ready?
OK. You take that.
So, I need you to think about two things -
how hairy was the animal in the picture,
and how big was the animal in the picture?
I want you to think about those things in terms of, like,
a scale of one to ten, while I explain what's going on over here,
OK? So we've got these two containers that we can
pour water into through these funnels.
You can think of these like neurons.
So this needs to be able to identify
if something is hairy or not.
This needs to be able to identify
whether an animal is big or not.
We'll start with hairiness.
Question one. On a scale of one to ten,
how hairy was the animal in the picture I showed you?
- I think about an eight. - About an eight. OK.
So we've got all these beakers of water here.
They're labelled one to ten.
And you can see the amount of water goes up as the number goes up.
So you chose... - Eight. - ..eight.
Right, there we go. So that's quite a lot of water.
So take that, and I want you to pour that into the funnel.
Can you reach that?
Yeah, just about. OK, great.
Yeah, keep going. That's fantastic.
So because you chose quite a lot of water,
the level should be able to
rise to at least this funny little
U-bend here. I want you to watch
what happens when it gets there.
Look at that. So that channel
has now turned into a siphon,
and that siphon effect
is pulling all of the water out of this neuron
and dumping it in this one.
So you chose enough water to reach
the activation level of this neuron.
So this neuron activated, it fired and it sent a signal
to this neuron here.
But you'll notice, look, this neuron hasn't activated yet.
That's because the activation level is higher. It's up here.
So this neuron is only going to fire if it gets a signal from this one
and this one at the same time.
So question number two, then.
On a scale of one to ten, how big was the animal that I showed you?
- I'd say about a nine. - A nine. OK. So this is beaker number nine.
So come round here, and you're going to
pour into the second container, the second neuron.
And again, hopefully, it's enough.
It's quite a lot of water,
quite a big number that you chose.
That's going to automatically prime the siphon again.
There we go.
So now, this neuron is getting
a signal from both of them
and, hopefully, it's going to reach
the activation level.
And there it goes.
So this neuron's now firing and
it's sending a signal, this time,
to... This is basically our readout.
So you can see the needle is swinging away from "not mammoth"
and towards "mammoth", right?
So, that's... And by the way, it is very slow.
Like, a real neuron, it's basically instantaneous, compared to this.
But the question is, did this incredibly sophisticated
neural network get the answer right?
- Well, this is what Rahel
was shown. - Yes!
- APPLAUSE
Success!
It's a mammoth! OK.
That very simple neural network there,
just with those, a couple of neurons,
was able to do a simple
pattern recognition task - a simple decision.
Now, ChatGPT does not involve
buckets of water at all, but nevertheless,
you can start to see how neural networks like this can make simple
decisions and be able to, put together, can do complex problems.
OK. Thank you very much, Rahel.
And thank you very much, Steve.
Thank you. APPLAUSE
Now, we wouldn't be able to build a physical network with buckets
of water that can represent all the tens of billions of neurons
that are in our brain.
They make literally trillions of connections.
We couldn't connect together with pipes like that.
But this century, we've been able
to use the power of modern computers
to build networks like this using software.
And these are artificial neural networks, and they contain
billions of neurons, arranged into layers, with those software neurons
communicating just like we saw in the Petri dish, but also,
just like we saw in Steve's experiment.
So, how do we design neural networks out of software
to be able to recognise a face or to play a game?
The simplest way that we do this is with what's called
supervised learning.
And supervised learning works by us labelling things for the computer,
so that it's able to learn and recognise them on its own.
We train AI like this every day.
Whenever you upload a picture of yourself or your friends
onto social media and you label it with your name or their name,
what you're doing is, you're providing training data
for the AI of the social media company, so that it will be able
to recognise you and your friends in the future.
To build AI that can do things for us, it needs data.
Without data, we wouldn't have the powerful AI systems
that we have today.
And data about people, about you and me, is a really precious
commodity for the companies that want to build AI.
But as well as supervised learning, there's another way that AI learns.
And to demonstrate it, I need the help of
machine learning scientist Safa Abbas.
APPLAUSE
Welcome, Safa. So, tell us a little bit about your job, what you do.
- So, I help companies apply AI to their business by implementing
machine learning solutions to understand and analyse data
and make predictions, and just help their day-to-day workflow.
So, depending on the data that our clients give us, we can use
supervised or unsupervised learning.
So, when we have labelled data, we use supervised learning.
But sometimes, we have data that isn't labelled
and not structured, and that's where we use
unsupervised learning to discover
patterns within the data for itself.
- OK, well, we're going to have another demo.
We're going to play another game. We need a volunteer.
Come on, put your hands up!
OK, I'm going to take you, please.
Yeah, please come on down.
APPLAUSE
Hello and welcome. What's your name?
- Khadija. - So you go in the middle.
So, Safa. - Yes. So this conveyer belt is going to bring out
a bunch of objects that are going to represent our dataset.
And what I need you to do is to organise them
into three categories. But you can
choose to do it however you like.
You have about 15 seconds for this task.
So if we just let the objects come out.
- You just organise them how you think they're best organised.
OK, I see where you're going with that.
- Great. Khadija, do you want to
tell me a little bit about how you decided to categorise your objects?
- I was thinking about the ways they can go, and I had two solutions
on how I can go.
One of the ways which I done was, do knife, fork and spoon.
But another way I could have done it was by the object, like,
it could have been the metal,
or it could have been the plastic and the wood.
- Oh, wow! That's great. - You're thinking very fast, I have to say.
- Very fast! - This is very impressive. Safa. - OK, great.
So now what we're going to do is complicate the task a little bit
by bringing out some more objects.
So we're making our dataset bigger, and I want you to do the same thing.
- Over to you. - No time limit on this one
because there's a lot more objects for you to look at.
- CHUCKLING IN AUDIENCE
- Great. OK, thank you, Khadija.
So, do you want to tell me again how you decided
to categorise these? - This one was more complicated,
but I decided to think a bit more outside the box.
This one is for metals.
This one is with wood and plastic.
And this is the ones that
I could not categorise.
- LAUGHTER - OK, great.
- Thank you so much.
So, this is a really, really simple way of demonstrating
how unsupervised learning works.
So, Khadija here kind of represents our algorithm that is observing
the data and learning, and learning patterns within the data
and then categorising it. Yeah?
OK, so there's one more thing to do.
We've got a random object,
and we're just going to test our algorithm as it's been trained.
And just this random object,
where would you put it?
OK, great. So it fit into
your kind of "other",
not wood, not metal category.
OK, great. Thank you so much.
OK. - APPLAUSE
Thank you, volunteer. You were wonderful and you were thinking
very quickly on your feet there. OK, Safa, so when is
unsupervised learning...?
I mean, we saw that if you could label data, you can do really
quite powerful things. So when do we need unsupervised learning?
- Yeah, exactly how you said.
So, supervised learning, we need kind of a ground truth.
We need, when we train, the algorithm
will learn what's right and wrong
from the data, but we don't always have that.
In the case of this data set, where we kind of had no rules,
we didn't know which way to categorise it,
the algorithm can learn to find the best pattern.
And sometimes, we work with really, really large data sets
that are difficult for us as humans to label, and also difficult for us
to recognise patterns in. So, yeah, this is where we kind of use
unsupervised learning, where we want to understand our data,
without obviously knowing what we're working with.
- OK. Thank you so much for joining us, Safa,
and making a difficult subject
APPLAUSE very comprehensible.
So, with both unsupervised and supervised learning - in fact,
with any sort of neural network -
scale matters. The size of the neural networks,
the number of elements it contains,
the number of neurons and the number of connections between those,
the amount of computer power that you can use to train the model
and how much training data you have to build it,
these are all the key ingredients of AI.
But as we've already seen,
the more data you train a neural network in, the better your results
are likely to be, and nowhere is that clearer
than with large language models.
A large language model is a model
like ChatGPT, or Cohere, or Google Bard.
It's a type of AI that can
recognise, summarise, translate
and even completely create new knowledge.
So, the picture behind me is the Bodleian Library.
This is from my university in Oxford.
It's one of the oldest libraries
in Europe, and it's what's called
a copyright library. That means every time a new book
is published, the Bodleian gets
a copy of it.
That's a lot of books every year.
And in 1598, the Bodleian Library
held about 2,500 books.
Anybody read 2,500 books?
That's a lot of books.
OK. Now, we are going to try
and visualise the amount of data
that's used to train large language models like Cohere,
or ChatGPT, or Bard.
So, this tiny little sliver
of confetti, which is so small
that I think probably, the people
at the back can't even see it,
and I'm having difficulty holding it,
that is a representation of how much
the Bodleian Library had in 1598.
OK?
That is the amount of data that the Bodleian Library had in 1598.
Well, today, the Bodleian Library
is home to around about 13 million volumes.
This is a picture of the Bodleian Library today.
Has anybody read that many books?
OK. Somebody thinks they have.
I don't think you've read 13 million books. OK.
So, that is represented by this amount of confetti.
Remember this? It's even disappeared.
That tiny little sliver of confetti
is the Bodleian Library in 1598.
This is the Bodleian Library today.
So now, on that scale,
let's see how much confetti
you would need to represent all the data that was used
to train Google Bard.
FANS WHIR
AUDIENCE MEMBERS GIGGLE AND EXCLAIM
LAUGHTER
That's a lot of confetti.
That's an awful lot of data.
So, how do large language models use
all of this data, this unimaginable
quantity of data, to be able to just
make a sentence?
To understand that, we're going to play a game, and this is called
the Prompt Completion Game. OK.
Now, we've got two teams that are playing this - Team 1
over here and Team 2 over here.
And all they're going to do,
these two teams, is construct
a sentence, and they're going to do that one word at a time.
You, in Column A, in column one, as you pass the piece of paper down,
you're going to write a word, but you only see the last word
that was written down. So you're going to fold the paper over,
so that the person in front of you only sees the last word
that was written. You, on this side, you're going to do
the same thing, but we're not going to fold the paper,
so you are going to see the whole of the sentence.
And what we want you to do is
just construct the best possible
sentence that you can.
OK. Ready?
Three, two, one - go!
OK.
The game that we're playing here
illustrates the way that language models work.
All they try to do is to predict
the next word that comes after the words before it.
You've already seen this idea
when you send messages
on your phone. If your mum starts to send you
a text message on her smartphone, the phone
will suggest completions for her.
So, for example, if she types,
"Have you", the phone might suggest...
So, the phone has learned that after she types
"Have you" that the likeliest
next things for her to type
are either "tidied your room"
or "walked the dog". Here,
your phone is doing
just exactly what a large
language model does.
The difference is the scale,
the unimaginable scale of data
that these models are used to train on.
OK, we've finished. So let's come over here.
Now, remember, this is the column
that did not get to see all the sentence before.
They just got to see the last word.
So, let's see what genius you came up with.
"The boat sails fast.
"Car wash.
"Bomb explosion..."
LAUGHTER "..happening on an elephant."
LAUGHTER
Your parents must be so proud, column one!
Let's see how column two did.
OK.
"The cat is mean and it papercut on the table because of..."
OK. Better...but, guys, that's not Shakespeare, I'm afraid.
I'm really sorry. I don't think we're going to be seeing
any New York Times bestsellers from column two.
But nevertheless, they did a better job than column one.
Column one's made no sense whatsoever.
Sorry, column one!
It didn't make any sense whatsoever.
But we set you up to fail there.
There was no way that you were going
to produce a sensible sentence
if all you saw was the last word.
Now, one of the reasons that
large language models like ChatGPT
and Bard and Cohere
have started to work in the last few years
is that they are designed to process
whole sentences,
not just one word at a time.
And as we saw in our game,
it's much easier to make a good guess
about the words that come next
if you see the whole of the sentence.
And it sounds obvious, but the AI
technology to make that happen
was only invented in 2017.
It's a neural network architecture
called transformer architectures
that was invented by Google.
But, OK, I've explained to you now how large language models
like ChatGPT work.
Do you think they understand what they're doing?
I have to tell you, I don't.
Large language models don't understand the meaning
of the sentences that they are constructing.
So how do they determine which words are most likely to come next
in a sentence?
To answer this, please welcome, from Cohere,
Seraphina Goldfarb-Tarrant.
APPLAUSE
Welcome to the Christmas Lectures, Seraphina.
OK, Seraphina, we're going to try and understand
how text is understood by a large language model.
- Yeah, so I'm a research scientist and I train them, but before I start
to train them, I have to take all of that confetti data
and I have to segment it into units, which we call tokens.
And then you have all of that data
and then the language model gets what the word is
based on the context it appears in.
And then you can take one of those words
and you can plot it into a space.
- That's what we're going to do now.
So here on the floor,
what we've got is a graph.
A graph just like you've seen
in your maths lessons at school.
It's got two axes.
This is the X axis
and this is the Y axis.
And what we're going to see is how
we can plot words on this graph.
Now, for the next segment, we need some volunteers, but they're not
really volunteers at all, I'm afraid, because we need ten of them.
And so, our ten volunteers who are going to come down, please,
now from the second row, this is your moment. Come on down.
APPLAUSE
OK, volunteers, so stand in the graph.
And now, number ten, just come and stand over here.
Your moment in the sun will be in a few minutes' time.
Put your animals on, so we can see your animals.
OK, we're going to play a game.
You're going to stand inside the axes of this graph
to see how you can be plotted.
And what I want you to do is just this.
I want you to stand close to animals that you are similar to
and as far away as possible from animals
that you aren't similar to. Got it?
So let's count them down.
Three, two, one, go!
OK, so how's that looking now, Seraphina, do you think?
- Yeah, so it looks like we have
the cat-tiger-lion group over here,
which is like feline-ish.
The wolf-fox-dog group over here,
so canine-ish. And then,
the donkey-zebra-horse group over here.
- So, you've just arranged yourself
by putting ideas that are similar
close to one another in this graph,
and each of you now has a co-ordinate in the graph,
an X and Y co-ordinate.
That's an X and Y co-ordinate,
which is identifying our cat.
This is the X and Y co-ordinate
that's identifying our tiger.
This for the lion. So, Seraphina, is this what large language models do?
- Absolutely. So large language models cluster words together
that are similar, just like you all have done,
by representing those words, or tokens, as coordinates in space,
just like a list of numbers, which we call a vector.
- OK, so now, this is your moment in the sun, number ten.
You're going to be a very unusual kind of animal.
Come forward. You are going to be a toaster.
"What did you do in the Christmas Lectures?"
"I was the toaster."
OK, so same rules for you, right?
Stand close to things that you're similar to and as far away
as possible from things that you are unlike. Off you go.
OK, so, number ten, again,
we set you up to fail.
A toaster isn't like a horse.
It isn't like a tiger. It isn't like a lion.
It isn't like any of these things.
In order to be able to plot you, we would need more dimensions.
We would need to add more dimensions to this graph,
in order for you to fit.
OK. Thank you all, volunteers,
and in particular, our toaster, number ten.
APPLAUSE
OK. Now, humans like us struggle
to think in more than three dimensions.
But for large language models,
we need many more than three dimensions.
Is that right, Seraphina? - Absolutely, yeah. So what we noticed
in this is that two dimensions
was clearly not enough, because since the vector coordinates
measure how similar things are based on their meaning,
like similar meanings are close by,
you definitely need way more than two, right?
You can't separate all words based on similarity in just
two dimensions. So in practice, we use a little over 1,000...
- 1,000 dimensions?! - ..in the language models I train. Yeah. - Wow.
OK. That's a lot of dimensions.
OK, so the language models can draw lots of correlations
between those different words. Is that right?
And what we noticed was that those words,
those animals that were similar had actually very similar coordinates.
Is that right? - Yep. - OK. So, language models don't just work with words,
do they, right? They can work with sentences or documents?
- Absolutely, yeah, and that's the big innovation that we've had recently.
We've been able to extend to much longer contexts.
So here's a little example of just
two types of sentences, embedded
with one of my language models.
We've made it in two dimensions.
This is in 1,000 dimensions, but my computer can't do that,
so I've had to project it into two.
- Just the same way that we did
with the kids and the animals, we've
reduced that sentence down to an X-Y co-ordinate? - Exactly.
- OK. - Yeah, so here, we have,
"please set an alarm at six am."
And in the similar cluster,
let's see, we have,
"alarm time for twelve pm."
- OK, and they're similar sentences.
- Exactly, and everything in that
cluster is kind of alarm-focused.
And then, this cluster over on
the other side of the coordinates,
it's like, "what and who was I emailed about last week?",
and maybe,
"all new email messages."
So this is almost like the X-Y
coordinates are email versus alarm
in this case, but it's embedding the whole sentence.
- OK. So large language models reduce text, sentences, documents,
down to these vectors, these lists of words.
And we saw vectors with just two dimensions.
But in practice, they're using many, many more than that. - Exactly.
- Seraphina, thank you so much for coming on and explaining this.
APPLAUSE
Neural networks that embed word in galaxies like that,
plotting their relationships to one another,
might also enable us to communicate in ways
that we have never thought possible.
And to explain, please can you give a warm welcome to the author of
How to Speak Whale, Tom Mustill?
APPLAUSE
Tom, welcome, but how exactly do we speak to whales?
- Well, if you want to understand what whales are saying,
first, you've got to listen.
Is the flame organ ready?
Are you ready?
- AUDIENCE: Yes. - All right.
Play the voice of the blue whale.
- RUMBLING MOAN
- Blue whales are the biggest animal on Earth
and they've got the loudest voice on Earth.
But until recently, we thought they were totally silent.
Their voices are below human hearing, so we've sped them up
and put them through this big subwoofer, so you can hear them.
And we've also put them through
the series of Rubens tubes
which, with the flames, allow you
to look at the characteristics
of their voices and see their waveforms.
These sounds are from underwater recordings
in Monterey Bay in California.
But play the second blue whale.
- RUMBLING MOAN
- So can you see the shapes, or maybe some patterns,
in these lines of flames?
These are very helpful for us.
When we have spectrograms, when we make pictures of sound,
we can find patterns in those pictures of sound.
And AI tools are really useful here, because it turns out
with natural language processing systems and human language,
when you plot all the different parts of a language, say,
with billions of examples into a giant galaxy, you can find
hidden patterns in those plots,
and AI tools can find these invisible patterns
that we can't see. There's no chance that our brains or mind
could perceive them, and it can link them to patterns between languages.
And things like Google Translate can match the hidden patterns
between different human languages and translate between them.
- OK, now, you prepared a demo for this, Tom, right? - Yes.
- And for all our demos, we need a volunteer.
So, let's have some hands up for a volunteer.
- You guys are keen! - You ARE keen.
Where do you want to go, Tom? Who do you want to pick?
- You. - Yeah.
OK, come on down.
APPLAUSE
Hello.
What's your name? - Maya. - Maya. Welcome, Maya.
OK, Tom, over to you.
- OK, Maya, do you speak any other languages?
- No. - Perfect. Neither do I.
But Google Translate is here to help us.
Now, on this piece of paper here
is a sort of graph of language
with the sort of X axis there
and the Y axis there,
and kind of imagine that there are thousands of other axes,
but we can't, because our brains don't work like that.
But the AIs can make giant galaxy maps of huge amounts of words,
and they can plot their relationships to each other.
And what we need to do is to identify a pattern
in this zoomed-in part of the galaxy map of words
in the English language. So I'm going to ask you to find "whales".
In fact, I'm going to give you a pen.
I'm going to ask you, can you please
put a circle around "whales"?
Very good.
"Are".
Oh, that's the wrong "are"!
Sorry, I should have been like...
Not like "Argh," like a pirate.
More like "are". - Oh. - Maybe that "are".
Sorry. Too many ares.
Google Translate does make
mistakes sometimes, too.
And "amazing".
OK, so, "Whales argh...are amazing."
So ignoring this one,
we're going to now take away
the English language map
and imagine we're going into
a totally different map of
a different language system.
So this is Japanese, and the natural
language processing systems,
the pattern finders, if you find the pattern for "whales are amazing"
in English and you look in the same places in the kind of galaxy cloud
of Japanese, what do you get?
And don't worry if your pronunciation is bad
because mine would be worse.
- I don't know how to pronounce it.
- Well, "Kujira wa subarashi."
Are there any Japanese people here?
Was that terrible?
- Yes. - Yes, it was terrible.
But would you like to say it in Japanese?
- Kujira wa subarashi. - Exactly!
Exactly what I said.
So, now let's try it
with one last language.
With German. So,
ignoring "Argh!" again.
Could you try that?
- Wale. - Yes.
- Sind. - Yes.
Ungl... - Unglaubliche.
- Wale sind unglaubliche.
Are there any German speakers here?
Yes. Could you say that?
Shout it out in German.
- OK. Wale sind unglaubliche.
- OK, but that... That's how it works, essentially,
that if you have enough examples of a language
and you can make a giant galaxy and you can find the shapes
of a certain phrase in them, you can match those shapes
between language without even having a dictionary.
And that is how Google Translate works.
It doesn't have a dictionary and it doesn't even know it's dealing
with the language. - OK. Thank you very much. Thanks for joining us
and well done. - Well done. Whale done!
- APPLAUSE
- And what's really exciting is,
we also don't have dictionaries for other species.
- OK. But, Tom, how important are large data sets for this?
- They're absolutely crucial.
The AIs need billions upon billions of examples of language
for this to work. If you find the pattern, you can translate it
between different languages,
and we would like to try
and apply this to other species,
but we need to get big data sets for that too.
- Well, for human languages
like English, we have lots and lots and lots of data, as we saw earlier.
But whales don't write books. Where are we going to get the data from?
- We need to get it straight from the whale's mouth,
and that is what we've begun to do. - I think you were going to show us
some videos at this point. - Yes, this is really special.
Nobody else has seen this video, I think, in the world before.
This is fresh from the sea
from Dominica. So these
are sperm whales. What can you see
on the top there, amongst them?
That's a family group of mothers,
grandmothers, sisters, brothers.
- It looks like a little baby whale.
- That's exactly right. That is
a newborn baby sperm whale calf.
It's so young, it's just a few minutes old, it hasn't
even learnt to swim yet, and
its family is lifting it up
to the surface, so it can take
its first breaths.
But what else strikes you about
that video? - Well, there was
no sound. - There was no sound.
So now, let's play
the second sperm whale video. So this is from the same
family group and this is from
Project Ceti, who are studying these whales.
- CLICKING
- So those clicks you can hear,
that is the conversations of these whales.
Project Ceti not only is studying their lives and their behaviours,
but they're recording the conversations, so we can
match what they're doing
with what they're saying.
In one year, they have doubled the entire dataset for every single
recording of sperm whales that we ever had before. And in a couple of
years, with that information, they hope to do embeddings like this
of sperm whale communication
systems and try and use these AI tools to find patterns within them.
And I don't know about you, but when I see
a video of an animal, especially an underwater animal, I make
wildlife films, I film them underwater, and I can hear them
communicating at the same time,
it changes how I feel about them.
I think about them differently.
And Project Ceti, just this week, has released some early AI analysis.
They think that whales have vowels. We can't hear them,
we just hear clicks. But the AI tools have found
in-between those clicks, vowels,
which maybe indicate that they've got something like a language.
- Wow! So where is this going? I mean, am I going to be able
to understand what my dog is saying to me soon?
- I mean, maybe. We just don't know. We've never listened to
other animals enough before, but dogs and cats are some of
the biggest data sets we have. Think of the internet.
It's almost like it was built for cats! So, that's one thing.
And there's another thing. I love, when I go out for walks,
there's an app you can have called Merlin on your phone. It's free.
You turn it on, it just listens to all the birdsong around you
and the AI scrolls through all of the sounds,
and it identifies which species they might be.
And I think the most exciting thing is that maybe, when I make
my next nature film, it might not just have underwater sound,
but it might also have subtitles. - That's incredible!
Thank you so much, Tom. - Thank you very much. Thank you.
- APPLAUSE
OK. Before we wrap up, let's take a look at what the robot
Ai-Da's been doing throughout this lecture.
Ai-Da, have you finished my portrait?
- Nearly there.
- OK. And what do you think of it so far?
- I'm quite pleased, so far.
I hope you like it.
- I've never had a portrait painted before, Ai-Da.
Thank you.
In this first lecture, we've gone under the hood of AI
to find out how intelligent machines learn.
In lecture two, we're going to have a look at how AI
is already being used all around us.
We'll look at breakthroughs in computer games, in medicine
and even in art. And we're going to see how ubiquitous AI is
in our everyday lives, even though we may not be aware of it.
APPLAUSE Thank you.
AI is here now.
Discover the different ways it may already be revolutionising
your life in my exclusive interview.
To watch, head to...
..and follow the links to the Open University.