(pensive music)
Hello!
Finding a quicker route to your destination.
Would you like me to reply to the email?
How can I help you?
The role of artificial intelligence seems to be
pervading many, many aspects of life, some that are
quite surprising and raise some concerns.
User recognized.
Hello, Dave.
As AI is being broadly deployed in the world,
the kinds of problems that we are dealing with are changing.
Finding recommendations
based on your viewing history.
We're increasingly finding these situations where
AI systems have to make these decisions
that in our eyes have a significant moral component.
Isolating tumor.
We really need to think about
how to encode those priorities in the algorithm itself.
(pensive music)
Most of the time, despite what we like to think,
we make our moral judgments based on intuition and emotion.
We get tired and we lose cognitive resources
and emotional resources, so, by the end of a day,
you often can't make the decisions
as well as when you started.
So, as technology started to develop
and it started to become possible to start thinking about
something else making a moral judgment,
it was a very natural switch to think
how could we use the developing technology
to make us make better moral judgments.
(pensive music)
I think there is a single morality
that applies to all humans.
One, reducing harm to others,
and respecting other people's rights.
So, you wanna reduce the amount of harm
that people suffer in their lives,
but you wanna do it in a way that doesn't
violate the rights of individuals along the path.
My response is no, I don't think there is,
but it doesn't bother me because everyone has some sense
of what morality for them is.
You define it.
Morality is a very personal thing.
Our project is dedicated to developing a moral
artificial intelligence with a small I rather than a big I,
so we're not trying to create something
that is equal to a human.
It is nowhere near ready for real practice.
We're starting to think about it now
because it'll take a long time to think about it.
The actual implementation will be decades away.
We're trying to create something
that can help solve problems.
What we are putting in there is what humans
think is morality, not what the machine thinks is morality.
Whatever the machine thinks is morality
is what we defined for it.
We want the computer to reflect human morality in general
rather than a particular individual.
And that's what our program allows us to do
because we can take the data from large numbers of humans,
we can develop and algorithm that predicts
what most humans would say.
An example that we have focused on is kidney exchanges.
Sometimes when you're distributing kidneys
to potential recipients, you have to decide
which one gets it, 'cause you've got one kidney
and lots of potential recipients.
So, should you give it to younger people
rather than older people?
What if the person has been on the waiting list
longer than another person?
People's intuitions obviously disagree.
Some people would, for example, say that you should
take into account whether a patient has dependents
like small children that they're taking care of,
and other people would say that you
should not take that into account.
What if they're responsible to a certain extent
for their own kidney problems 'cause they did something
that helped cause those problems.
There are lots of different features
that people might take to be morally relevant.
And yet, when people think about those,
they might forget one of those.
They might misunderstand one of those.
Why does it matter whether someone's younger or older?
Is that just all about life expectancy?
And they might get confused by the multitude
of different morally relevant features.
The machine can give you a better sense
of which judgments humans would make
if they considered all the morally relevant features.
We are not trying to take our favorite moral theory,
build it into a machine, have it apply to problems
and have everybody agree with us.
That's not the goal at all.
Our project looks at survey data about what
a wide variety of people
take to be morally relevant features,
and then develops an algorithm to determine
how those different features interact.
And then the algorithm predicts
which judgments humans would make.
We don't want the computer
to do the same thing in every circumstance,
we want it to be sensitive to those particular variations.
The data shows that most people,
they've got biases that are gonna affect their daily lives
even though they're not aware of them
and even though they think they're wrong.
So, it's not unusual to think
that these hospital administrators would have biases
that are affecting their decisions.
It happens to all of us, we're all human,
and that computer can help us figure out
when it's happening and then correct for it.
These committees are very busy so,
they'll supposedly have to make 10 different decisions
in a meeting and they're 10 people,
and they've got an hour.
You get all of this information
from all these different people,
what their medical data is, who might be matched with who,
and now you have to sort through this
enormous number of people and figure out
what is the best way to match them all together.
Even if what best means is just maximizing
the number of people that get a kidney,
this quickly becomes very overwhelming.
(pensive music)
So, this is where algorithms come in,
because they don't mind searching through
a large space of possible alternatives.
Then the computer says, according to the values
that you've expressed in the past,
we think you oughta do this, given your values,
corrected for biases and ignorance and confusion.
Then they can say, oh, of those 10 cases,
there are only two
where we're disagreeing with the computer.
The human still has to make the decision,
but the computer can say this is the decision
that you would make given your values
and the things that you, yourself,
take to be morally relevant.
Given them interacting in the way that you, yourself,
take to be appropriate, and getting rid of
all those biases that you, yourself,
take to be features that should not
figure into your moral judgments.
(pensive music)
There need to be multiple different ways
that the AI can give you feedback.
So, one should be, are you being consistent
with your own choices, your own behavior,
and the other should be are you being consistent
with certain groups that you might care about.
And, one might be your local group,
another might be your profession,
another might be the population as a whole.
And then if the judgment you think is right or wrong
disagrees with the computer, now you've got a problem,
but if it agrees you feel okay,
the computer has helped me confirm,
I'm gonna be more confident.
(pensive music)
One upside is that it's going to improve
human moral judgment.
I've been on these hospital ethics committees before
and you reach a consensus but you still think
I don't know, it was a tough case.
Other upsides is, if there are fewer mistakes then
the people who need the kidneys and who deserve the kidneys
are gonna be more likely to get them.
You're also going to have people
who maybe they don't deserve it as much now
but they've got a special condition which makes it
very unlikely they'll get matched in the future.
The goal is to help the committee
make the judgments according to the features
that they, themselves take to be morally relevant.
(pensive music)
No, we're not building a HAL.
Notice that HAL went against
what Dave thought was the right thing.
What are you talking about, HAL?
Now we've gotta ask, wait a minute,
why do we trust Dave?
We have certain empathy for Dave,
'cause we're both humans, but humans make mistakes too.
It can only be attributable to human error.
So, let's say there are two Daves,
one where Dave is making a moral judgment
about what ought to be done,
that almost all other humans would agree with.
Hello, HAL, do you read me?
Well, then, we don't want the computer
to go against it, but if Dave is making a decision
that's gonna be good for Dave
but contrary to everybody else,
then maybe we want HAL to stop him.
So, what's the decision there?
We want a HAL that is going to reflect
what most or almost all humans
would think is the morally right thing to do.
So, most AI systems today, even the very successful ones,
are still very narrow, in that,
they focus on a very particular domain.
You might think about AlphaGo.
AlphaGo is exceptionally good at playing Go,
but it cannot play Chess at all.
Because we are making an artificial intelligence
with a small I, we're training this intelligence,
so, it's always gonna have the same goal.
If you're using machine learning
and deep learning techniques,
and you've specified as its goal that it should
develop an algorithm that best mimics human morality,
then, that makes it unlikely to all of a sudden
start making up its own morality.
(pensive music)
Four years ago the idea of self-driving cars
still sounded a little nuts,
and now everyone accepts it as an inevitability.
So, I think before the idea
of a moral artificial intelligence sounded nuts
and now people, I think, will be more open to it.
With enough work on the AI, if it works on a hospital board,
then in theory we should be able to find
ways to design an AI that could be
generalized to other situations, as well.
I think there's a lot of opportunity here
to do good for the world.
I think there are real things to worry about.
Technological unemployment, autonomous weapons systems,
large scale surveillance,
these are all issues tied to AI.
In large part, that is up to us, as humanity,
to determine how we're gonna use this technology.
Look at all the things that computers
have been able to do in the last 10 years, 20 years.
So yes, I'm an optimist about what
they'll be able to accomplish in years to come.
These programs will not reduce
our understanding of morality,
instead, it will improve and enhance
our understanding of morality,
and also enhance the judgments that we make.
(pensive music)