1
00:00:00,000 --> 00:00:02,667
(pensive music)

2
00:00:06,850 --> 00:00:08,240
Hello!

3
00:00:08,240 --> 00:00:10,623
Finding a quicker route to your destination.

4
00:00:11,614 --> 00:00:13,420
Would you like me to reply to the email?

5
00:00:13,420 --> 00:00:14,500
How can I help you?

6
00:00:14,500 --> 00:00:15,333
How can I help you?

7
00:00:15,333 --> 00:00:16,166
How can I help you?

8
00:00:16,166 --> 00:00:17,321
How can I help you?

9
00:00:17,321 --> 00:00:20,490
The role of artificial intelligence seems to be

10
00:00:20,490 --> 00:00:24,230
pervading many, many aspects of life, some that are

11
00:00:24,230 --> 00:00:27,540
quite surprising and raise some concerns.

12
00:00:27,540 --> 00:00:28,820
User recognized.

13
00:00:28,820 --> 00:00:29,980
Hello, Dave.

14
00:00:29,980 --> 00:00:33,140
As AI is being broadly deployed in the world,

15
00:00:33,140 --> 00:00:36,753
the kinds of problems that we are dealing with are changing.

16
00:00:36,753 --> 00:00:38,670
Finding recommendations

17
00:00:38,670 --> 00:00:40,430
based on your viewing history.

18
00:00:40,430 --> 00:00:42,850
We're increasingly finding these situations where

19
00:00:42,850 --> 00:00:44,980
AI systems have to make these decisions

20
00:00:44,980 --> 00:00:48,130
that in our eyes have a significant moral component.

21
00:00:48,130 --> 00:00:49,750
Isolating tumor.

22
00:00:49,750 --> 00:00:51,250
We really need to think about

23
00:00:51,250 --> 00:00:55,653
how to encode those priorities in the algorithm itself.

24
00:00:55,653 --> 00:00:58,320
(pensive music)

25
00:01:08,810 --> 00:01:11,660
Most of the time, despite what we like to think,

26
00:01:11,660 --> 00:01:14,850
we make our moral judgments based on intuition and emotion.

27
00:01:14,850 --> 00:01:17,130
We get tired and we lose cognitive resources

28
00:01:17,130 --> 00:01:19,940
and emotional resources, so, by the end of a day,

29
00:01:19,940 --> 00:01:21,570
you often can't make the decisions

30
00:01:21,570 --> 00:01:22,970
as well as when you started.

31
00:01:23,940 --> 00:01:26,210
So, as technology started to develop

32
00:01:26,210 --> 00:01:28,500
and it started to become possible to start thinking about

33
00:01:28,500 --> 00:01:30,640
something else making a moral judgment,

34
00:01:30,640 --> 00:01:33,220
it was a very natural switch to think

35
00:01:33,220 --> 00:01:35,440
how could we use the developing technology

36
00:01:35,440 --> 00:01:38,055
to make us make better moral judgments.

37
00:01:38,055 --> 00:01:40,722
(pensive music)

38
00:01:43,070 --> 00:01:45,420
I think there is a single morality

39
00:01:45,420 --> 00:01:47,500
that applies to all humans.

40
00:01:47,500 --> 00:01:49,240
One, reducing harm to others,

41
00:01:49,240 --> 00:01:51,440
and respecting other people's rights.

42
00:01:51,440 --> 00:01:53,370
So, you wanna reduce the amount of harm

43
00:01:53,370 --> 00:01:55,230
that people suffer in their lives,

44
00:01:55,230 --> 00:01:57,670
but you wanna do it in a way that doesn't

45
00:01:57,670 --> 00:02:01,660
violate the rights of individuals along the path.

46
00:02:01,660 --> 00:02:05,260
My response is no, I don't think there is,

47
00:02:05,260 --> 00:02:07,830
but it doesn't bother me because everyone has some sense

48
00:02:07,830 --> 00:02:09,860
of what morality for them is.

49
00:02:09,860 --> 00:02:11,230
You define it.

50
00:02:11,230 --> 00:02:13,713
Morality is a very personal thing.

51
00:02:15,860 --> 00:02:18,500
Our project is dedicated to developing a moral

52
00:02:18,500 --> 00:02:21,450
artificial intelligence with a small I rather than a big I,

53
00:02:21,450 --> 00:02:23,490
so we're not trying to create something

54
00:02:23,490 --> 00:02:24,750
that is equal to a human.

55
00:02:24,750 --> 00:02:27,740
It is nowhere near ready for real practice.

56
00:02:27,740 --> 00:02:29,930
We're starting to think about it now

57
00:02:29,930 --> 00:02:32,820
because it'll take a long time to think about it.

58
00:02:32,820 --> 00:02:35,840
The actual implementation will be decades away.

59
00:02:35,840 --> 00:02:36,820
We're trying to create something

60
00:02:36,820 --> 00:02:38,150
that can help solve problems.

61
00:02:38,150 --> 00:02:40,070
What we are putting in there is what humans

62
00:02:40,070 --> 00:02:43,310
think is morality, not what the machine thinks is morality.

63
00:02:43,310 --> 00:02:44,700
Whatever the machine thinks is morality

64
00:02:44,700 --> 00:02:46,560
is what we defined for it.

65
00:02:46,560 --> 00:02:51,560
We want the computer to reflect human morality in general

66
00:02:51,560 --> 00:02:53,520
rather than a particular individual.

67
00:02:53,520 --> 00:02:55,500
And that's what our program allows us to do

68
00:02:55,500 --> 00:02:59,430
because we can take the data from large numbers of humans,

69
00:02:59,430 --> 00:03:01,590
we can develop and algorithm that predicts

70
00:03:01,590 --> 00:03:03,200
what most humans would say.

71
00:03:03,200 --> 00:03:07,460
An example that we have focused on is kidney exchanges.

72
00:03:07,460 --> 00:03:10,040
Sometimes when you're distributing kidneys

73
00:03:10,040 --> 00:03:12,547
to potential recipients, you have to decide

74
00:03:12,547 --> 00:03:15,450
which one gets it, 'cause you've got one kidney

75
00:03:15,450 --> 00:03:17,760
and lots of potential recipients.

76
00:03:17,760 --> 00:03:20,680
So, should you give it to younger people

77
00:03:20,680 --> 00:03:22,200
rather than older people?

78
00:03:22,200 --> 00:03:24,640
What if the person has been on the waiting list

79
00:03:24,640 --> 00:03:26,150
longer than another person?

80
00:03:26,150 --> 00:03:29,170
People's intuitions obviously disagree.

81
00:03:29,170 --> 00:03:32,750
Some people would, for example, say that you should

82
00:03:32,750 --> 00:03:36,050
take into account whether a patient has dependents

83
00:03:36,050 --> 00:03:38,030
like small children that they're taking care of,

84
00:03:38,030 --> 00:03:39,390
and other people would say that you

85
00:03:39,390 --> 00:03:40,930
should not take that into account.

86
00:03:40,930 --> 00:03:42,870
What if they're responsible to a certain extent

87
00:03:42,870 --> 00:03:44,830
for their own kidney problems 'cause they did something

88
00:03:44,830 --> 00:03:46,630
that helped cause those problems.

89
00:03:46,630 --> 00:03:48,050
There are lots of different features

90
00:03:48,050 --> 00:03:50,960
that people might take to be morally relevant.

91
00:03:50,960 --> 00:03:53,500
And yet, when people think about those,

92
00:03:53,500 --> 00:03:54,840
they might forget one of those.

93
00:03:54,840 --> 00:03:57,510
They might misunderstand one of those.

94
00:03:57,510 --> 00:04:00,100
Why does it matter whether someone's younger or older?

95
00:04:00,100 --> 00:04:02,410
Is that just all about life expectancy?

96
00:04:02,410 --> 00:04:04,651
And they might get confused by the multitude

97
00:04:04,651 --> 00:04:07,552
of different morally relevant features.

98
00:04:07,552 --> 00:04:11,091
The machine can give you a better sense

99
00:04:11,091 --> 00:04:14,127
of which judgments humans would make

100
00:04:14,127 --> 00:04:17,840
if they considered all the morally relevant features.

101
00:04:17,840 --> 00:04:20,950
We are not trying to take our favorite moral theory,

102
00:04:20,950 --> 00:04:24,300
build it into a machine, have it apply to problems

103
00:04:24,300 --> 00:04:25,880
and have everybody agree with us.

104
00:04:25,880 --> 00:04:27,610
That's not the goal at all.

105
00:04:27,610 --> 00:04:32,138
Our project looks at survey data about what

106
00:04:32,138 --> 00:04:34,230
a wide variety of people

107
00:04:34,230 --> 00:04:37,100
take to be morally relevant features,

108
00:04:37,100 --> 00:04:41,270
and then develops an algorithm to determine

109
00:04:41,270 --> 00:04:43,500
how those different features interact.

110
00:04:43,500 --> 00:04:45,410
And then the algorithm predicts

111
00:04:45,410 --> 00:04:48,240
which judgments humans would make.

112
00:04:48,240 --> 00:04:49,300
We don't want the computer

113
00:04:49,300 --> 00:04:51,670
to do the same thing in every circumstance,

114
00:04:51,670 --> 00:04:54,963
we want it to be sensitive to those particular variations.

115
00:04:56,540 --> 00:04:58,550
The data shows that most people,

116
00:04:58,550 --> 00:05:01,210
they've got biases that are gonna affect their daily lives

117
00:05:01,210 --> 00:05:02,810
even though they're not aware of them

118
00:05:02,810 --> 00:05:04,570
and even though they think they're wrong.

119
00:05:04,570 --> 00:05:07,020
So, it's not unusual to think

120
00:05:07,020 --> 00:05:09,460
that these hospital administrators would have biases

121
00:05:09,460 --> 00:05:11,710
that are affecting their decisions.

122
00:05:11,710 --> 00:05:14,120
It happens to all of us, we're all human,

123
00:05:14,120 --> 00:05:16,500
and that computer can help us figure out

124
00:05:16,500 --> 00:05:18,783
when it's happening and then correct for it.

125
00:05:20,330 --> 00:05:23,439
These committees are very busy so,

126
00:05:23,439 --> 00:05:25,640
they'll supposedly have to make 10 different decisions

127
00:05:25,640 --> 00:05:27,520
in a meeting and they're 10 people,

128
00:05:27,520 --> 00:05:28,810
and they've got an hour.

129
00:05:28,810 --> 00:05:30,420
You get all of this information

130
00:05:30,420 --> 00:05:31,960
from all these different people,

131
00:05:31,960 --> 00:05:35,620
what their medical data is, who might be matched with who,

132
00:05:35,620 --> 00:05:38,041
and now you have to sort through this

133
00:05:38,041 --> 00:05:39,790
enormous number of people and figure out

134
00:05:39,790 --> 00:05:42,430
what is the best way to match them all together.

135
00:05:42,430 --> 00:05:44,610
Even if what best means is just maximizing

136
00:05:44,610 --> 00:05:46,280
the number of people that get a kidney,

137
00:05:46,280 --> 00:05:48,982
this quickly becomes very overwhelming.

138
00:05:48,982 --> 00:05:50,540
(pensive music)

139
00:05:50,540 --> 00:05:52,070
So, this is where algorithms come in,

140
00:05:52,070 --> 00:05:54,370
because they don't mind searching through

141
00:05:54,370 --> 00:05:57,040
a large space of possible alternatives.

142
00:05:57,040 --> 00:05:59,550
Then the computer says, according to the values

143
00:05:59,550 --> 00:06:01,380
that you've expressed in the past,

144
00:06:01,380 --> 00:06:03,149
we think you oughta do this, given your values,

145
00:06:03,149 --> 00:06:07,090
corrected for biases and ignorance and confusion.

146
00:06:07,090 --> 00:06:09,360
Then they can say, oh, of those 10 cases,

147
00:06:09,360 --> 00:06:10,193
there are only two

148
00:06:10,193 --> 00:06:12,562
where we're disagreeing with the computer.

149
00:06:12,562 --> 00:06:13,920
The human still has to make the decision,

150
00:06:13,920 --> 00:06:16,450
but the computer can say this is the decision

151
00:06:16,450 --> 00:06:19,040
that you would make given your values

152
00:06:19,040 --> 00:06:20,390
and the things that you, yourself,

153
00:06:20,390 --> 00:06:22,050
take to be morally relevant.

154
00:06:22,050 --> 00:06:24,720
Given them interacting in the way that you, yourself,

155
00:06:24,720 --> 00:06:26,970
take to be appropriate, and getting rid of

156
00:06:26,970 --> 00:06:29,033
all those biases that you, yourself,

157
00:06:29,033 --> 00:06:31,090
take to be features that should not

158
00:06:31,090 --> 00:06:33,415
figure into your moral judgments.

159
00:06:33,415 --> 00:06:35,670
(pensive music)

160
00:06:35,670 --> 00:06:38,260
There need to be multiple different ways

161
00:06:38,260 --> 00:06:40,170
that the AI can give you feedback.

162
00:06:40,170 --> 00:06:42,550
So, one should be, are you being consistent

163
00:06:42,550 --> 00:06:45,080
with your own choices, your own behavior,

164
00:06:45,080 --> 00:06:47,130
and the other should be are you being consistent

165
00:06:47,130 --> 00:06:49,030
with certain groups that you might care about.

166
00:06:49,030 --> 00:06:50,550
And, one might be your local group,

167
00:06:50,550 --> 00:06:52,150
another might be your profession,

168
00:06:52,150 --> 00:06:54,903
another might be the population as a whole.

169
00:06:56,120 --> 00:06:59,090
And then if the judgment you think is right or wrong

170
00:06:59,090 --> 00:07:02,730
disagrees with the computer, now you've got a problem,

171
00:07:02,730 --> 00:07:05,360
but if it agrees you feel okay,

172
00:07:05,360 --> 00:07:06,900
the computer has helped me confirm,

173
00:07:06,900 --> 00:07:08,300
I'm gonna be more confident.

174
00:07:09,545 --> 00:07:12,212
(pensive music)

175
00:07:14,970 --> 00:07:17,630
One upside is that it's going to improve

176
00:07:17,630 --> 00:07:19,110
human moral judgment.

177
00:07:19,110 --> 00:07:21,360
I've been on these hospital ethics committees before

178
00:07:21,360 --> 00:07:23,980
and you reach a consensus but you still think

179
00:07:23,980 --> 00:07:26,690
I don't know, it was a tough case.

180
00:07:26,690 --> 00:07:30,990
Other upsides is, if there are fewer mistakes then

181
00:07:30,990 --> 00:07:35,240
the people who need the kidneys and who deserve the kidneys

182
00:07:35,240 --> 00:07:36,800
are gonna be more likely to get them.

183
00:07:36,800 --> 00:07:38,630
You're also going to have people

184
00:07:38,630 --> 00:07:41,360
who maybe they don't deserve it as much now

185
00:07:41,360 --> 00:07:43,520
but they've got a special condition which makes it

186
00:07:43,520 --> 00:07:46,090
very unlikely they'll get matched in the future.

187
00:07:46,090 --> 00:07:48,540
The goal is to help the committee

188
00:07:48,540 --> 00:07:51,370
make the judgments according to the features

189
00:07:51,370 --> 00:07:54,495
that they, themselves take to be morally relevant.

190
00:07:54,495 --> 00:07:57,162
(pensive music)

191
00:07:59,060 --> 00:08:00,890
No, we're not building a HAL.

192
00:08:00,890 --> 00:08:02,740
Notice that HAL went against

193
00:08:02,740 --> 00:08:05,639
what Dave thought was the right thing.

194
00:08:05,639 --> 00:08:07,290
What are you talking about, HAL?

195
00:08:07,290 --> 00:08:09,780
Now we've gotta ask, wait a minute,

196
00:08:09,780 --> 00:08:12,440
why do we trust Dave?

197
00:08:12,440 --> 00:08:13,730
We have certain empathy for Dave,

198
00:08:13,730 --> 00:08:17,160
'cause we're both humans, but humans make mistakes too.

199
00:08:17,160 --> 00:08:20,980
It can only be attributable to human error.

200
00:08:20,980 --> 00:08:22,560
So, let's say there are two Daves,

201
00:08:22,560 --> 00:08:26,180
one where Dave is making a moral judgment

202
00:08:26,180 --> 00:08:28,010
about what ought to be done,

203
00:08:28,010 --> 00:08:31,010
that almost all other humans would agree with.

204
00:08:31,010 --> 00:08:32,570
Hello, HAL, do you read me?

205
00:08:32,570 --> 00:08:34,260
Well, then, we don't want the computer

206
00:08:34,260 --> 00:08:37,350
to go against it, but if Dave is making a decision

207
00:08:37,350 --> 00:08:38,550
that's gonna be good for Dave

208
00:08:38,550 --> 00:08:40,600
but contrary to everybody else,

209
00:08:40,600 --> 00:08:42,440
then maybe we want HAL to stop him.

210
00:08:42,440 --> 00:08:44,170
So, what's the decision there?

211
00:08:44,170 --> 00:08:46,970
We want a HAL that is going to reflect

212
00:08:46,970 --> 00:08:49,560
what most or almost all humans

213
00:08:49,560 --> 00:08:51,560
would think is the morally right thing to do.

214
00:08:51,560 --> 00:08:54,183
So, most AI systems today, even the very successful ones,

215
00:08:54,183 --> 00:08:56,420
are still very narrow, in that,

216
00:08:56,420 --> 00:08:58,700
they focus on a very particular domain.

217
00:08:58,700 --> 00:09:00,200
You might think about AlphaGo.

218
00:09:01,185 --> 00:09:03,400
AlphaGo is exceptionally good at playing Go,

219
00:09:03,400 --> 00:09:05,410
but it cannot play Chess at all.

220
00:09:05,410 --> 00:09:07,580
Because we are making an artificial intelligence

221
00:09:07,580 --> 00:09:10,550
with a small I, we're training this intelligence,

222
00:09:10,550 --> 00:09:12,810
so, it's always gonna have the same goal.

223
00:09:12,810 --> 00:09:14,810
If you're using machine learning

224
00:09:14,810 --> 00:09:16,830
and deep learning techniques,

225
00:09:16,830 --> 00:09:19,760
and you've specified as its goal that it should

226
00:09:19,760 --> 00:09:23,942
develop an algorithm that best mimics human morality,

227
00:09:23,942 --> 00:09:28,520
then, that makes it unlikely to all of a sudden

228
00:09:28,520 --> 00:09:31,166
start making up its own morality.

229
00:09:31,166 --> 00:09:33,833
(pensive music)

230
00:09:35,600 --> 00:09:38,300
Four years ago the idea of self-driving cars

231
00:09:38,300 --> 00:09:40,000
still sounded a little nuts,

232
00:09:40,000 --> 00:09:43,530
and now everyone accepts it as an inevitability.

233
00:09:43,530 --> 00:09:45,770
So, I think before the idea

234
00:09:45,770 --> 00:09:48,200
of a moral artificial intelligence sounded nuts

235
00:09:48,200 --> 00:09:51,490
and now people, I think, will be more open to it.

236
00:09:51,490 --> 00:09:55,920
With enough work on the AI, if it works on a hospital board,

237
00:09:55,920 --> 00:09:57,770
then in theory we should be able to find

238
00:09:57,770 --> 00:10:00,830
ways to design an AI that could be

239
00:10:00,830 --> 00:10:03,260
generalized to other situations, as well.

240
00:10:03,260 --> 00:10:05,150
I think there's a lot of opportunity here

241
00:10:05,150 --> 00:10:06,430
to do good for the world.

242
00:10:06,430 --> 00:10:09,690
I think there are real things to worry about.

243
00:10:09,690 --> 00:10:12,939
Technological unemployment, autonomous weapons systems,

244
00:10:12,939 --> 00:10:15,360
large scale surveillance,

245
00:10:15,360 --> 00:10:18,370
these are all issues tied to AI.

246
00:10:18,370 --> 00:10:21,490
In large part, that is up to us, as humanity,

247
00:10:21,490 --> 00:10:24,020
to determine how we're gonna use this technology.

248
00:10:24,020 --> 00:10:25,610
Look at all the things that computers

249
00:10:25,610 --> 00:10:28,800
have been able to do in the last 10 years, 20 years.

250
00:10:28,800 --> 00:10:30,990
So yes, I'm an optimist about what

251
00:10:30,990 --> 00:10:33,683
they'll be able to accomplish in years to come.

252
00:10:35,140 --> 00:10:37,680
These programs will not reduce

253
00:10:37,680 --> 00:10:39,220
our understanding of morality,

254
00:10:39,220 --> 00:10:41,849
instead, it will improve and enhance

255
00:10:41,849 --> 00:10:44,200
our understanding of morality,

256
00:10:44,200 --> 00:10:46,910
and also enhance the judgments that we make.

257
00:10:46,910 --> 00:10:49,660
(pensive music)