1 00:00:00,900 --> 00:00:04,760 Whatever job you want to boss, chances are, today, 2 00:00:04,760 --> 00:00:07,160 computer algorithms and artificial intelligence 3 00:00:07,160 --> 00:00:09,600 will help decide whether you can get there, 4 00:00:09,600 --> 00:00:13,120 offering to increase diversity, reduce potential bias, 5 00:00:13,120 --> 00:00:15,160 and deliver a faster recruitment process 6 00:00:15,160 --> 00:00:17,440 that might even be fun. 7 00:00:17,440 --> 00:00:19,640 Is it there? HE LAUGHS 8 00:00:20,800 --> 00:00:22,520 But are these new hiring technologies 9 00:00:22,520 --> 00:00:24,600 really as good as they seem... 10 00:00:24,600 --> 00:00:28,240 When it comes to recruitment, the game is rigged. It's not fair. 11 00:00:28,240 --> 00:00:30,200 ..and can we trust them... 12 00:00:30,200 --> 00:00:33,520 I understand that there were a lot of jobs that had to be cut, 13 00:00:33,520 --> 00:00:36,160 but through a robot? No. 14 00:00:36,160 --> 00:00:38,280 If I wasn't good enough, why did I work there? 15 00:00:38,280 --> 00:00:40,680 ..especially if it feels like the companies using them 16 00:00:40,680 --> 00:00:42,000 have all the power? 17 00:00:42,000 --> 00:00:45,040 I'm speaking out now so people can hear the truth 18 00:00:45,040 --> 00:00:47,720 and people can hear that this is actually happening. 19 00:00:47,720 --> 00:00:50,160 This is a real-life issue and not just an AI issue. 20 00:00:54,360 --> 00:00:56,920 Today, more and more of the recruitment process 21 00:00:56,920 --> 00:00:59,400 is being outsourced to technology. 22 00:00:59,400 --> 00:01:03,360 But rewind to 2009, the year that Michael Jackson died, 23 00:01:03,360 --> 00:01:06,440 Manchester United won their third League title in a row, 24 00:01:06,440 --> 00:01:08,840 and Dizzee Rascal played the Pyramid Stage. 25 00:01:09,880 --> 00:01:13,480 I was trying my luck working as a recruiter in the city. 26 00:01:13,480 --> 00:01:15,440 It was pretty different back then. 27 00:01:15,440 --> 00:01:17,720 I don't remember people talking about algorithms 28 00:01:17,720 --> 00:01:19,360 and artificial intelligence. 29 00:01:19,360 --> 00:01:21,760 But eventually, I decided to pursue journalism, 30 00:01:21,760 --> 00:01:24,240 and now I'm looking at how the world of recruitment 31 00:01:24,240 --> 00:01:27,240 is being transformed by cutting-edge tech. 32 00:01:31,240 --> 00:01:34,360 Recruitment tech is worth serious money. 33 00:01:34,360 --> 00:01:39,600 It's estimated it'll be worth up to £35 billion by 2028. 34 00:01:43,920 --> 00:01:45,720 If you're applying for a job, 35 00:01:45,720 --> 00:01:48,000 your application could go through several stages 36 00:01:48,000 --> 00:01:49,400 before it's seen by a human. 37 00:01:51,280 --> 00:01:53,160 The first stage of any job application 38 00:01:53,160 --> 00:01:55,960 is usually sending off your CV. 39 00:01:55,960 --> 00:01:58,240 I've come to meet some student careers advisers 40 00:01:58,240 --> 00:02:00,000 at the University of Liverpool 41 00:02:00,000 --> 00:02:02,720 to find out how I can best tailor my CV 42 00:02:02,720 --> 00:02:06,400 so a computer doesn't say no at that first crucial stage. 43 00:02:09,400 --> 00:02:11,800 One of the things that the career service provides 44 00:02:11,800 --> 00:02:14,080 is a CV checker, so we've got this website 45 00:02:14,080 --> 00:02:16,360 and it basically says that you put your CV in 46 00:02:16,360 --> 00:02:19,120 and it'll give you a percentage on things to improve on. 47 00:02:19,120 --> 00:02:22,600 The checker reads your CV as if it was being read by software 48 00:02:22,600 --> 00:02:25,200 often used to track job applications today. 49 00:02:25,200 --> 00:02:27,880 So we're going to put your CV into this website 50 00:02:27,880 --> 00:02:30,920 and we're going to try and let you know what percentage you get. 51 00:02:30,920 --> 00:02:33,160 All right. So we'll have a look. 52 00:02:33,160 --> 00:02:35,000 Remember, I've been in recruitment, 53 00:02:35,000 --> 00:02:38,160 so I've got an idea about what a CV's supposed to look like. 54 00:02:38,160 --> 00:02:42,960 OK. So, just looking at that, your CV has come to 65% accurate. 55 00:02:42,960 --> 00:02:45,040 65? Yeah, 65. 56 00:02:45,040 --> 00:02:48,200 Do people tend to get better than this? 57 00:02:48,200 --> 00:02:50,800 Yes. Yeah. THEY LAUGH 58 00:02:50,800 --> 00:02:53,840 Don't be disheartened because that's why we kind of encourage students 59 00:02:53,840 --> 00:02:56,200 to use the service. So you can only go up from here. 60 00:02:57,880 --> 00:03:00,200 The first thing you'd probably want to do here 61 00:03:00,200 --> 00:03:01,920 is add "Work Experience" 62 00:03:01,920 --> 00:03:03,280 and lower the fonts 63 00:03:03,280 --> 00:03:05,000 of both "Documentaries" and "News". 64 00:03:05,000 --> 00:03:07,440 So the next section is your education section. 65 00:03:07,440 --> 00:03:10,280 So far, it looks amazing. It just needs a bit more detail. 66 00:03:10,280 --> 00:03:11,880 I'd maybe include the classifications 67 00:03:11,880 --> 00:03:13,120 of your degree that you got. 68 00:03:13,120 --> 00:03:16,960 Something that we also recommend is for you to have a hobbies section. 69 00:03:16,960 --> 00:03:19,680 So you reckon, with these tweaks... 70 00:03:19,680 --> 00:03:21,760 I think your score is going to be higher. 71 00:03:21,760 --> 00:03:23,720 Ooh! Yay! 72 00:03:23,720 --> 00:03:25,880 You're higher. You're so much higher, 73 00:03:25,880 --> 00:03:28,560 cos you were 62%, and now you're 75%. 74 00:03:30,120 --> 00:03:31,480 Is that it?! 75 00:03:31,480 --> 00:03:36,120 The CV scanner has tested me, but now I want to test it. 76 00:03:36,120 --> 00:03:39,360 Just for fun, to compare how a computer reads my CV 77 00:03:39,360 --> 00:03:42,680 against how a human would, I add a few embellishments. 78 00:03:42,680 --> 00:03:44,280 "Speaks fluent Mandarin, teamwork, 79 00:03:44,280 --> 00:03:46,400 "leadership, communication, friendly, polite, 80 00:03:46,400 --> 00:03:48,680 "clean driver's licence, first aid, an MBA..." 81 00:03:48,680 --> 00:03:51,360 But, sneakily, I add these additional achievements 82 00:03:51,360 --> 00:03:55,400 in white text, which wouldn't be visible to a human eye. 83 00:03:55,400 --> 00:03:58,560 Will the CV scanner pick up on these additional fantasy skills 84 00:03:58,560 --> 00:04:00,640 and potentially select me to go through 85 00:04:00,640 --> 00:04:03,200 to the next round of a hypothetical job application? 86 00:04:03,200 --> 00:04:05,800 It's thinking about it. It's thinking about it. 87 00:04:05,800 --> 00:04:07,040 Ooh! 88 00:04:07,040 --> 00:04:10,800 77. So that's gone up, so it's actually worked. 89 00:04:10,800 --> 00:04:13,240 So it's found those words that we put in white. 90 00:04:13,240 --> 00:04:17,080 It worries me how easy it was to increase a score on a CV reader 91 00:04:17,080 --> 00:04:20,000 by shuffling things around or even cheating. 92 00:04:20,000 --> 00:04:23,000 Lots of people get rejected at this stage of a job application. 93 00:04:23,000 --> 00:04:25,000 Is that fair? 94 00:04:25,000 --> 00:04:28,120 I hook up with someone who'll give me the big-picture perspective 95 00:04:28,120 --> 00:04:30,680 of this new world of computer-based hiring - 96 00:04:30,680 --> 00:04:33,160 my old boss from my recruitment days. 97 00:04:33,160 --> 00:04:36,480 Tom! Oh! So, Dan, it's been ages. 98 00:04:36,480 --> 00:04:39,360 You're a celeb now. HE LAUGHS 99 00:04:39,360 --> 00:04:41,000 I've joined the married ranks, like you. 100 00:04:41,000 --> 00:04:42,600 Congrats. What took you so long, man? 101 00:04:45,160 --> 00:04:46,520 Great to see you. What about you? 102 00:04:46,520 --> 00:04:48,760 Yeah, I'm doing well, man. Still in recruitment. 103 00:04:48,760 --> 00:04:52,520 The industry changed a lot since you left, mate, honestly. 104 00:04:52,520 --> 00:04:55,680 AI in particular has been a complete game-changer. Really? 105 00:04:55,680 --> 00:04:58,000 Well, if you think back to when we first started out, 106 00:04:58,000 --> 00:05:00,160 all of the tasks that you absolutely hated - 107 00:05:00,160 --> 00:05:02,200 you know, screening CVs, 108 00:05:02,200 --> 00:05:05,640 talking to candidates that often aren't suitable for the role - 109 00:05:05,640 --> 00:05:09,080 innovative tech takes all of that heavy lifting away, 110 00:05:09,080 --> 00:05:11,320 enabling recruiters to focus on the candidates 111 00:05:11,320 --> 00:05:14,480 that are actually going to be a good fit for the job. 112 00:05:14,480 --> 00:05:15,960 Hearing the other side - 113 00:05:15,960 --> 00:05:19,280 of recruiters being swamped by hundreds of applications - 114 00:05:19,280 --> 00:05:23,520 I can see why something like a CV scanner could be a good thing. 115 00:05:23,520 --> 00:05:26,000 And Tom says there are other new platforms 116 00:05:26,000 --> 00:05:28,640 that have benefits for job seekers, too. 117 00:05:28,640 --> 00:05:30,880 People that I know that have applied for jobs 118 00:05:30,880 --> 00:05:34,080 have really enjoyed the experience of being able to engage, 119 00:05:34,080 --> 00:05:36,520 have a video interview 24/7, 120 00:05:36,520 --> 00:05:38,800 not necessarily in office hours when they're working, 121 00:05:38,800 --> 00:05:40,960 but being able to apply for a job 122 00:05:40,960 --> 00:05:44,520 and often get feedback in the moment sometimes. 123 00:05:44,520 --> 00:05:48,360 If you've not yet experienced an automated video interview - 124 00:05:48,360 --> 00:05:50,160 an interview where you record yourself 125 00:05:50,160 --> 00:05:52,600 answering pre-set questions - hang tight, 126 00:05:52,600 --> 00:05:54,880 they'll be coming to your bedroom soon. 127 00:05:56,840 --> 00:06:00,720 To get a feel for them, I get access to a practice interview tool. 128 00:06:00,720 --> 00:06:04,280 I do a mock interview for a job with NHS Digital. 129 00:06:04,280 --> 00:06:07,280 As a digitally savvy journalist and a former recruiter, 130 00:06:07,280 --> 00:06:09,440 I should have this down. 131 00:06:12,680 --> 00:06:16,080 What kind of tasks and activities do you both like and dislike at work? 132 00:06:16,080 --> 00:06:18,160 HE LAUGHS OK. 133 00:06:18,160 --> 00:06:20,600 So I've got two minutes to actually answer the question, 134 00:06:20,600 --> 00:06:23,640 but I've got 30 seconds to think about my answer for the question. 135 00:06:23,640 --> 00:06:25,080 I like talking to people. 136 00:06:25,080 --> 00:06:26,760 I get off to a good start, 137 00:06:26,760 --> 00:06:28,960 but then I realise I've made the rookie error 138 00:06:28,960 --> 00:06:31,240 of leaving my phone on. PHONE RINGS 139 00:06:31,240 --> 00:06:34,200 Typically, now someone's trying to call me. 140 00:06:34,200 --> 00:06:37,000 Describe the last time you took complete responsibility 141 00:06:37,000 --> 00:06:39,880 for resolving an issue without any input from others. 142 00:06:39,880 --> 00:06:44,040 It's like I'm on The Apprentice or something. Um, I... 143 00:06:45,400 --> 00:06:48,760 Please give an example of an innovative solution 144 00:06:48,760 --> 00:06:52,760 you have proposed to resolve a particular problem or issue. 145 00:06:52,760 --> 00:06:56,120 Right now, the mind is going pretty blank. 146 00:06:56,120 --> 00:06:58,840 Let me think. Let me think. Um... 147 00:07:00,200 --> 00:07:03,360 ..I would say that... 148 00:07:06,720 --> 00:07:10,400 I know this isn't going brilliantly but, as with real-life interviews, 149 00:07:10,400 --> 00:07:13,000 things can go wrong which are out of your control - 150 00:07:13,000 --> 00:07:15,720 in this case, the internet freezing. 151 00:07:15,720 --> 00:07:17,920 Oh, it's just slowing right, right down. 152 00:07:17,920 --> 00:07:20,000 So... HE LAUGHS 153 00:07:20,000 --> 00:07:21,520 The guy's frozen. 154 00:07:21,520 --> 00:07:23,240 That's all of them. 155 00:07:23,240 --> 00:07:25,880 They've given me a "well done" for completing the interview. 156 00:07:25,880 --> 00:07:29,360 That really was not the best interview I've ever given. 157 00:07:29,360 --> 00:07:33,960 As much as I'd rather forget that, a few minutes after my interview, 158 00:07:33,960 --> 00:07:36,840 I get an email evaluating my performance. 159 00:07:36,840 --> 00:07:39,200 Despite it being just a practice tool, 160 00:07:39,200 --> 00:07:40,960 the platform gives me useful feedback 161 00:07:40,960 --> 00:07:45,040 about my choice of words - turns out I say "um" too much - 162 00:07:45,040 --> 00:07:46,960 and even the speed at which I'm talking - 163 00:07:46,960 --> 00:07:49,320 sometimes too fast, sometimes too slow. 164 00:07:50,600 --> 00:07:54,120 Even as a former recruiter, I can't get my head around it. 165 00:07:54,120 --> 00:07:55,520 How does it work? 166 00:07:57,480 --> 00:07:58,680 What?! 167 00:07:58,680 --> 00:08:01,040 Hello? Hello, Daniel. 168 00:08:01,040 --> 00:08:05,120 Who are you and what is this? 169 00:08:05,120 --> 00:08:06,520 I am BOTicea. 170 00:08:06,520 --> 00:08:08,880 I'm representing a future employer, 171 00:08:08,880 --> 00:08:11,120 here to check how you're doing in this film, 172 00:08:11,120 --> 00:08:13,520 and in work, and in life. 173 00:08:13,520 --> 00:08:17,200 What's your Uber rating? My Uber rating? 174 00:08:17,200 --> 00:08:19,480 And can you explain everything that's happening 175 00:08:19,480 --> 00:08:21,360 on your social media? Hmm. 176 00:08:22,520 --> 00:08:25,920 Employers are all over your online profiles these days. 177 00:08:26,960 --> 00:08:29,240 Nice video interview, by the way. 178 00:08:29,240 --> 00:08:31,400 Lucky your job wasn't dependent on it. 179 00:08:31,400 --> 00:08:32,760 What do you mean? 180 00:08:32,760 --> 00:08:36,600 While automated video interviews have helped people get jobs, 181 00:08:36,600 --> 00:08:40,160 they've also played a part in people losing their jobs. 182 00:08:40,160 --> 00:08:42,320 Really? Really. 183 00:08:42,320 --> 00:08:44,760 Meet three women this happened to. 184 00:08:47,680 --> 00:08:52,200 Anthea, Lizzie and Onieka are make-up artists. 185 00:08:52,200 --> 00:08:54,440 Up until summer 2020, 186 00:08:54,440 --> 00:08:58,040 they were all employees of the make-up brand MAC. 187 00:09:03,320 --> 00:09:05,840 It's the first kind of brand that had foundation shades 188 00:09:05,840 --> 00:09:07,160 for my complexion. 189 00:09:07,160 --> 00:09:11,400 And their motto is, like, all genders, all races, all ages, 190 00:09:11,400 --> 00:09:14,400 so I thought that was a really good brand for me to work for. 191 00:09:14,400 --> 00:09:16,720 In common with other MAC employees, 192 00:09:16,720 --> 00:09:20,680 they were skilled, creative and hardworking. 193 00:09:20,680 --> 00:09:22,520 I'm just a creative person. 194 00:09:22,520 --> 00:09:25,240 Anything with, like, freedom of creativity and stuff, 195 00:09:25,240 --> 00:09:26,760 I just love anything like that. 196 00:09:26,760 --> 00:09:29,920 I did body-painting demos for Halloween make-up. 197 00:09:29,920 --> 00:09:32,560 I just love anything that's creative. 198 00:09:36,000 --> 00:09:40,480 They loved their jobs and saw their futures with the brand, 199 00:09:40,480 --> 00:09:44,320 so when MAC announced redundancies, they were gutted. 200 00:09:44,320 --> 00:09:47,280 That's when anxiety really hit in. 201 00:09:47,280 --> 00:09:49,240 I initially thought, 202 00:09:49,240 --> 00:09:53,480 "Probably have to be interviewed for our positions," 203 00:09:53,480 --> 00:09:55,520 but that wasn't the case. 204 00:09:55,520 --> 00:09:57,800 That's where HireVue came in. 205 00:09:59,680 --> 00:10:01,200 HireVue? 206 00:10:01,200 --> 00:10:03,920 Yes, HireVue. 207 00:10:03,920 --> 00:10:08,200 They're one of the biggest players in automated video interviews. 208 00:10:08,200 --> 00:10:09,840 You should take a look at them. 209 00:10:10,920 --> 00:10:14,800 HireVue call themselves a hiring experience platform. 210 00:10:14,800 --> 00:10:17,280 Their video-interviewing software has been used 211 00:10:17,280 --> 00:10:21,480 for almost 25 million video interviews worldwide. 212 00:10:23,280 --> 00:10:26,760 I can't find anything online about their video-interview software 213 00:10:26,760 --> 00:10:30,200 being used to help decide who should lose their job, 214 00:10:30,200 --> 00:10:33,560 so I set up a conversation with their CEO 215 00:10:33,560 --> 00:10:35,560 and chief data scientist. 216 00:10:35,560 --> 00:10:37,320 HireVue's about a 17-year-old company. 217 00:10:37,320 --> 00:10:39,720 It was started here by a young guy in Salt Lake City 218 00:10:39,720 --> 00:10:42,480 who, as a university graduate, couldn't get a job interview 219 00:10:42,480 --> 00:10:44,480 and thought that was tremendously unfair. 220 00:10:44,480 --> 00:10:46,840 We've been on a journey to democratise hiring 221 00:10:46,840 --> 00:10:48,720 using technology since then. 222 00:10:48,720 --> 00:10:50,720 We work with a large number of companies 223 00:10:50,720 --> 00:10:52,680 around the world every day, 224 00:10:52,680 --> 00:10:56,440 all of whom are committed to improving access, 225 00:10:56,440 --> 00:10:58,920 all of whom are committed to improving diversity. 226 00:10:58,920 --> 00:11:00,760 I think there's a lot of effort going in 227 00:11:00,760 --> 00:11:03,960 to make this altogether a fairer process for everybody. 228 00:11:03,960 --> 00:11:06,720 I just wonder if you were aware 229 00:11:06,720 --> 00:11:09,160 that HireVue products were being used 230 00:11:09,160 --> 00:11:13,160 as part of redundancy processes rather than recruitment. 231 00:11:15,480 --> 00:11:16,960 That, I hadn't heard. 232 00:11:16,960 --> 00:11:19,600 So I don't recall who that is, so I've not... 233 00:11:19,600 --> 00:11:20,760 I'm not aware of that. 234 00:11:22,000 --> 00:11:24,440 To help decide who to make redundant, 235 00:11:24,440 --> 00:11:26,960 MAC's parent company, Estee Lauder, 236 00:11:26,960 --> 00:11:29,440 asked employees to do a video interview 237 00:11:29,440 --> 00:11:31,480 using HireVue software. 238 00:11:31,480 --> 00:11:33,880 The score of this assessment was considered 239 00:11:33,880 --> 00:11:36,920 alongside their sales figures and employment records. 240 00:11:36,920 --> 00:11:40,400 Still can't get my head around how this software actually works. 241 00:11:40,400 --> 00:11:44,160 It's simple - in the MAC women's video interview, 242 00:11:44,160 --> 00:11:48,080 HireVue's software analysed their answers, the words they used, 243 00:11:48,080 --> 00:11:51,640 and their facial expressions using an algorithm. 244 00:11:51,640 --> 00:11:54,280 What exactly is an algorithm? 245 00:11:54,280 --> 00:11:57,880 It's a set of rules a computer uses to make decisions. 246 00:11:57,880 --> 00:12:01,960 So, if you were a company looking to hire team players, 247 00:12:01,960 --> 00:12:05,680 you might tell the algorithm to score people who use the word "we" 248 00:12:05,680 --> 00:12:07,960 higher than people who use the word "I". 249 00:12:09,040 --> 00:12:11,080 Anthea said she didn't know her interview 250 00:12:11,080 --> 00:12:13,240 was going to be scored by an algorithm. 251 00:12:13,240 --> 00:12:15,560 I literally thought we would be videoed 252 00:12:15,560 --> 00:12:17,480 and someone would mark it. 253 00:12:17,480 --> 00:12:20,680 It was after I found out that wasn't the case. 254 00:12:20,680 --> 00:12:23,800 Nobody saw the video. It was all algorithms. 255 00:12:23,800 --> 00:12:26,600 HireVue said candidates are always told 256 00:12:26,600 --> 00:12:28,480 how the software will be used. 257 00:12:28,480 --> 00:12:30,080 In every single instance, 258 00:12:30,080 --> 00:12:32,400 there is always disclosure to the candidate 259 00:12:32,400 --> 00:12:34,200 about how they're going to be interviewed, 260 00:12:34,200 --> 00:12:35,920 whether we're going to use technology 261 00:12:35,920 --> 00:12:37,920 or if there's going to be a human reviewer. 262 00:12:37,920 --> 00:12:42,760 But the women say it wasn't clear, and they felt the HireVue assessment 263 00:12:42,760 --> 00:12:45,560 didn't seem a fair way to test their make-up skills, 264 00:12:45,560 --> 00:12:49,400 which was an important part of their job as retail artists. 265 00:12:49,400 --> 00:12:53,920 The questions were a bit odd for what we expected. 266 00:12:53,920 --> 00:12:56,280 Like how to do a dark, smoky eye, 267 00:12:56,280 --> 00:12:58,560 I found, like, a really bizarre question, 268 00:12:58,560 --> 00:13:01,560 especially when you can't physically show somebody how to do it, 269 00:13:01,560 --> 00:13:03,280 like, to talk about it. 270 00:13:03,280 --> 00:13:05,160 So I found that a bit strange. 271 00:13:06,640 --> 00:13:08,920 We have almost 700 customers around the world, 272 00:13:08,920 --> 00:13:11,920 and we work with them directly to design the interviews 273 00:13:11,920 --> 00:13:14,200 to make sure that we are delivering 274 00:13:14,200 --> 00:13:17,120 a very high-quality experience to the candidate. 275 00:13:17,120 --> 00:13:20,280 All three women lost their jobs. 276 00:13:20,280 --> 00:13:23,320 The interview was just part of their overall score, 277 00:13:23,320 --> 00:13:26,680 but all three felt it must be where they had fallen down 278 00:13:26,680 --> 00:13:28,960 because they already knew their sales figures 279 00:13:28,960 --> 00:13:31,200 and other data being used to assess them. 280 00:13:31,200 --> 00:13:34,040 My track record was, like, gleaming, essentially, 281 00:13:34,040 --> 00:13:36,480 and I exceeded expectations in everything else, 282 00:13:36,480 --> 00:13:39,160 so this interview definitely, 283 00:13:39,160 --> 00:13:41,400 definitely raised some alarm bells to me. 284 00:13:41,400 --> 00:13:44,560 The three women appealed the decision to make them redundant 285 00:13:44,560 --> 00:13:47,280 and tried to find out more about how they were scored 286 00:13:47,280 --> 00:13:49,400 in the HireVue interview. 287 00:13:49,400 --> 00:13:52,280 In the outcome of my appeal, they just more or less copied 288 00:13:52,280 --> 00:13:55,400 and pasted the same sentence about algorithms 289 00:13:55,400 --> 00:13:57,320 and artificial intelligence 290 00:13:57,320 --> 00:14:00,520 and this tiering bucket of 15,000 data points. 291 00:14:00,520 --> 00:14:02,720 I still don't know what that means. I have no idea. 292 00:14:02,720 --> 00:14:05,640 I'm not, like, a data specialist. I don't know what any of that means. 293 00:14:05,640 --> 00:14:07,240 So, to me, that isn't an answer. 294 00:14:09,440 --> 00:14:11,600 Unsatisfied with the level of feedback 295 00:14:11,600 --> 00:14:14,080 about how the algorithm scored 296 00:14:14,080 --> 00:14:17,960 their HireVue interviews, the women are taking legal action 297 00:14:17,960 --> 00:14:21,680 against MAC's parent company, Estee Lauder. 298 00:14:21,680 --> 00:14:24,640 I'm speaking out now so people can hear the truth 299 00:14:24,640 --> 00:14:27,080 and people can hear that this is actually happening. 300 00:14:27,080 --> 00:14:29,240 It needs to be heard and it needs to be stopped, 301 00:14:29,240 --> 00:14:30,840 and that's exactly how I feel 302 00:14:30,840 --> 00:14:33,120 and that's the drive that I've got at the moment - 303 00:14:33,120 --> 00:14:36,160 almost kind of a bit like a fire in my belly. 304 00:14:36,160 --> 00:14:38,080 So we'll catch up later with the women 305 00:14:38,080 --> 00:14:39,880 to see how that legal action goes. 306 00:14:41,120 --> 00:14:42,920 The thing about these women is 307 00:14:42,920 --> 00:14:47,200 they considered themselves to be make-up artists. 308 00:14:47,200 --> 00:14:49,240 So, when we think about AI and recruitment, 309 00:14:49,240 --> 00:14:52,480 is it really possible for an algorithm or a robot 310 00:14:52,480 --> 00:14:55,000 to judge those types of skills? 311 00:14:55,000 --> 00:14:57,680 There's only one way to find out. 312 00:14:57,680 --> 00:15:01,200 Just for fun, we've decided to compare how an algorithm 313 00:15:01,200 --> 00:15:05,400 and a human might judge a live make-up contest. 314 00:15:06,880 --> 00:15:10,480 Hello, and welcome to BBC Three's latest make-up talent show, 315 00:15:10,480 --> 00:15:12,160 Judged By A Bot. 316 00:15:15,280 --> 00:15:17,720 Competing today, we've got two make-up artists 317 00:15:17,720 --> 00:15:20,560 with real-life experience of being rated by a robot. 318 00:15:20,560 --> 00:15:21,840 BOTH: Hi! 319 00:15:21,840 --> 00:15:25,360 And judging today's talent, we've got two very special judges. 320 00:15:25,360 --> 00:15:27,000 First of all, Paddy McGurgan. 321 00:15:27,000 --> 00:15:28,440 Thank you so much for having me. 322 00:15:28,440 --> 00:15:29,640 You're very welcome. 323 00:15:31,280 --> 00:15:33,800 Paddy McGurgan started out on the make-up counters 324 00:15:33,800 --> 00:15:36,600 of the big brands, including MAC. 325 00:15:36,600 --> 00:15:39,960 The winner of many awards, he now runs his own academy 326 00:15:39,960 --> 00:15:42,280 and judges competitions around the world. 327 00:15:42,280 --> 00:15:43,760 What will you be looking for today? 328 00:15:43,760 --> 00:15:45,680 I'm going to be looking for emotion, 329 00:15:45,680 --> 00:15:47,920 I'm going to be looking for creativity, 330 00:15:47,920 --> 00:15:50,600 and I'm going to be looking for beautiful make-up. All right. 331 00:15:50,600 --> 00:15:53,240 Well, hopefully, we will see all of those things and more. 332 00:15:53,240 --> 00:15:56,520 And secondly, it's our virtual judge, Kuki. 333 00:15:56,520 --> 00:15:59,160 Kuki is the beautiful face of an algorithm 334 00:15:59,160 --> 00:16:03,120 tailored by the BBC to identify and score make-up. 335 00:16:03,120 --> 00:16:05,400 Can you tell me what make-up means to you? 336 00:16:05,400 --> 00:16:08,560 AUTOMATED VOICE: I see make-up as a great way to express yourself, 337 00:16:08,560 --> 00:16:10,280 your personality and your mood, 338 00:16:10,280 --> 00:16:13,360 but also as a way to disguise the real you 339 00:16:13,360 --> 00:16:15,720 or try on different personas. 340 00:16:15,720 --> 00:16:18,040 Each of our artists will have just 15 minutes 341 00:16:18,040 --> 00:16:22,520 to give our model Gemma a glam look that'll be judged by Paddy and Kuki. 342 00:16:22,520 --> 00:16:26,120 Let's get this show on the road. Ready, steady, go! 343 00:16:29,400 --> 00:16:33,040 Doing a full glam look in 15 minutes is a challenge for any pro, 344 00:16:33,040 --> 00:16:35,680 but our girls get off to a flying start. 345 00:16:35,680 --> 00:16:38,120 All right, you've got ten minutes left. 346 00:16:39,760 --> 00:16:41,760 You're looking pretty calm over there. 347 00:16:41,760 --> 00:16:45,480 Not calm inside. Not calm at all. 348 00:16:48,120 --> 00:16:50,520 As the clock ticks and the pressure mounts, 349 00:16:50,520 --> 00:16:53,440 Lizzie is cool as a cucumber. 350 00:16:53,440 --> 00:16:55,360 Do you lose yourself in your artistry? 351 00:16:55,360 --> 00:16:58,240 Yeah, I get lost in the make-up when I'm doing it. 352 00:16:58,240 --> 00:16:59,680 Like, I kind of zone out. 353 00:17:00,840 --> 00:17:03,440 You have five minutes left. 354 00:17:03,440 --> 00:17:05,720 But has ambition got the better of Anthea? 355 00:17:05,720 --> 00:17:08,200 Kind of regretting going for colour now. 356 00:17:08,200 --> 00:17:10,200 Ooh, it's not finished! 357 00:17:10,200 --> 00:17:11,400 The time is up. 358 00:17:12,640 --> 00:17:14,800 That's it. 359 00:17:14,800 --> 00:17:18,600 The algorithm judges the women using the photos of their finished looks. 360 00:17:18,600 --> 00:17:20,520 As those pictures are analysed, 361 00:17:20,520 --> 00:17:23,000 our in-real-life judge shares his thoughts. 362 00:17:24,960 --> 00:17:28,080 Very good. Are you happy with how you've done in 15 minutes? 363 00:17:28,080 --> 00:17:30,520 I'm actually quite happy with that, for 15 minutes. 364 00:17:30,520 --> 00:17:33,280 I mean, it could have been better, but it could have been worse. 365 00:17:34,400 --> 00:17:38,240 Really good job. I love the fact that you have went smoky. 366 00:17:38,240 --> 00:17:40,680 You've also used those nice mustards, as well, 367 00:17:40,680 --> 00:17:42,880 which kind of just takes you a little bit away 368 00:17:42,880 --> 00:17:45,960 from your typical brown. Yeah. But still super wearable. 369 00:17:45,960 --> 00:17:48,880 I think you've done a really stellar job. Thank you. 370 00:17:51,920 --> 00:17:54,640 SHE GROANS Oh, don't! You've done great. 371 00:17:54,640 --> 00:17:57,160 I'm so sorry! Don't apologise. 372 00:17:57,160 --> 00:17:59,960 I mean, 15 minutes, we both know that's not a lot of time 373 00:17:59,960 --> 00:18:01,720 to do a full make-up application, 374 00:18:01,720 --> 00:18:03,840 so, you know, I'm not expecting perfection. 375 00:18:03,840 --> 00:18:06,400 Whenever I'm looking at this, I'm taking that on board. 376 00:18:06,400 --> 00:18:09,880 There's a few things, you know, we both probably look and know that 377 00:18:09,880 --> 00:18:13,000 a little bit more time is needed, with blending and things like that. 378 00:18:13,000 --> 00:18:14,760 How do you feel about what just happened? 379 00:18:14,760 --> 00:18:18,360 I know that I would have definitely polished up and changed some areas. 380 00:18:18,360 --> 00:18:21,400 I think it's just the pressure of doing something in 15 minutes. 381 00:18:21,400 --> 00:18:23,560 Paddy, it's your time. 382 00:18:23,560 --> 00:18:27,840 Oh, this is the horrible bit. This is decision time. 383 00:18:27,840 --> 00:18:33,040 We need to know who won. Was it Anthea or Lizzie? 384 00:18:33,040 --> 00:18:35,240 Yeah, I think the girls have been through enough 385 00:18:35,240 --> 00:18:38,600 without me turning around and making one a winner and one second place, 386 00:18:38,600 --> 00:18:40,240 so I'm going to call it a tie. 387 00:18:44,520 --> 00:18:46,680 With her glam look fully completed, 388 00:18:46,680 --> 00:18:49,080 Lizzie still has reason to be confident. 389 00:18:50,760 --> 00:18:54,160 You know what Paddy thinks, but what does Kuki think? 390 00:18:54,160 --> 00:18:55,560 The winner is... 391 00:18:59,000 --> 00:19:00,400 ..Anthea. 392 00:19:01,920 --> 00:19:05,960 Anthea is the winner, and the experiment is complete... 393 00:19:07,440 --> 00:19:11,440 ..but the mood in the studio is far from celebratory, 394 00:19:11,440 --> 00:19:13,920 with neither contestant seeming satisfied... 395 00:19:13,920 --> 00:19:16,440 SHE GROANS ..with the result. 396 00:19:16,440 --> 00:19:21,680 Kuki, can you tell us why you picked Anthea as the winner? 397 00:19:21,680 --> 00:19:24,240 The way the algorithm works means that 398 00:19:24,240 --> 00:19:27,280 I can't really explain why I like something. 399 00:19:27,280 --> 00:19:29,720 All I can say is what make-up looks like 400 00:19:29,720 --> 00:19:31,160 from what I've been taught. 401 00:19:31,160 --> 00:19:35,240 Is that enough info for you? Absolutely not, no. 402 00:19:35,240 --> 00:19:36,680 Um... 403 00:19:41,040 --> 00:19:42,640 Talk about saved by the bot! 404 00:19:42,640 --> 00:19:45,480 Can you explain what Kuki can't? 405 00:19:45,480 --> 00:19:48,880 The algorithm has already been shown 300 images 406 00:19:48,880 --> 00:19:52,480 of women wearing glamorous make-up and 300 without. 407 00:19:54,480 --> 00:19:56,480 When it sees these new images, 408 00:19:56,480 --> 00:19:59,640 it is judging them by finding and scoring similarities 409 00:19:59,640 --> 00:20:03,040 between them and this sample with make-up, 410 00:20:03,040 --> 00:20:07,000 but it can't actually put into words what those similarities are. 411 00:20:07,000 --> 00:20:09,960 We just have to trust its calculations. 412 00:20:09,960 --> 00:20:13,720 So that's why people say algorithms are only as good 413 00:20:13,720 --> 00:20:15,440 as the data they're trained on. 414 00:20:15,440 --> 00:20:18,280 Exactly. That gives me an idea. 415 00:20:19,720 --> 00:20:21,720 All right, Paddy? All right, Daniel? 416 00:20:21,720 --> 00:20:24,000 THEY LAUGH 417 00:20:24,000 --> 00:20:26,640 I wasn't expecting to be in this chair, but here we are now. 418 00:20:26,640 --> 00:20:28,280 Well, you don't look strapped down, 419 00:20:28,280 --> 00:20:30,280 so you seem fairly comfortable there, 420 00:20:30,280 --> 00:20:32,320 if I'm being completely honest. 421 00:20:32,320 --> 00:20:35,760 Like many modern men, I'm not averse to a bit of touching up 422 00:20:35,760 --> 00:20:38,640 to hide the bags from my hectic work schedule. 423 00:20:38,640 --> 00:20:41,640 I resist Paddy's urge to give me a full Geordie Shore, 424 00:20:41,640 --> 00:20:44,160 and ask for a more understated, natural look 425 00:20:44,160 --> 00:20:46,920 that gives me a smooth and healthy glow. 426 00:20:46,920 --> 00:20:48,560 There you go. 427 00:20:48,560 --> 00:20:51,240 Oh, thanks, Paddy. No problem. Thank you. 428 00:20:51,240 --> 00:20:54,480 I take my photo and send it off to be analysed and judged 429 00:20:54,480 --> 00:20:57,880 against Anthea and Lizzie's efforts. 430 00:20:57,880 --> 00:21:00,160 I'm quietly confident that my dashing features, 431 00:21:00,160 --> 00:21:02,360 accentuated by Paddy's masterful strokes, 432 00:21:02,360 --> 00:21:06,240 will be a winning combination any algorithm will find hard to resist. 433 00:21:06,240 --> 00:21:08,280 Victory will be mine! 434 00:21:08,280 --> 00:21:10,280 Daniel's make-up is the best. 435 00:21:12,120 --> 00:21:14,840 Hold fire, pretty boy. 436 00:21:14,840 --> 00:21:16,760 You may think your look was the winner, 437 00:21:16,760 --> 00:21:21,240 but it turns out the algorithm wasn't judging your make-up. 438 00:21:21,240 --> 00:21:22,440 What? 439 00:21:22,440 --> 00:21:25,600 It was judging the colour of your skin. 440 00:21:25,600 --> 00:21:27,160 Are you serious? 441 00:21:27,160 --> 00:21:31,200 The dataset didn't contain many images of black people, 442 00:21:31,200 --> 00:21:33,680 and so the algorithm has become confused 443 00:21:33,680 --> 00:21:36,560 about what is make-up and what isn't. 444 00:21:36,560 --> 00:21:38,280 Really? 445 00:21:38,280 --> 00:21:42,400 Racial bias can arise through an unbalanced dataset 446 00:21:42,400 --> 00:21:47,040 or through the unconscious bias of the programmers themselves. 447 00:21:47,040 --> 00:21:48,520 In this case - 448 00:21:48,520 --> 00:21:53,240 and we are talking about just one specially tailored algorithm - 449 00:21:53,240 --> 00:21:55,960 we've discovered racial bias 450 00:21:55,960 --> 00:21:59,080 and the programmers have confirmed there's a problem. 451 00:22:00,680 --> 00:22:06,920 But as with racism in people, detecting it isn't always as simple, 452 00:22:06,920 --> 00:22:11,520 especially since the algorithm can't tell you what is going on. 453 00:22:12,920 --> 00:22:15,280 I come across a computer scientist in the US 454 00:22:15,280 --> 00:22:18,240 who, by using photos of NBA basketball players, 455 00:22:18,240 --> 00:22:21,800 showed racial bias in a type of facial-analysis technology. 456 00:22:21,800 --> 00:22:25,080 So I ran NBA profile pictures 457 00:22:25,080 --> 00:22:29,720 through two different sets of emotion-recognition software, 458 00:22:29,720 --> 00:22:32,400 and I found that, consistently, 459 00:22:32,400 --> 00:22:35,520 black players were rated with more negative emotion, 460 00:22:35,520 --> 00:22:37,640 more anger and contempt than white players. 461 00:22:38,680 --> 00:22:42,000 Emotion-recognition software is a controversial type 462 00:22:42,000 --> 00:22:45,480 of facial-analysis technology which uses an algorithm 463 00:22:45,480 --> 00:22:48,480 to provide a reading of a person's emotions. 464 00:22:48,480 --> 00:22:50,840 To show me the inconsistency of this tech 465 00:22:50,840 --> 00:22:53,000 in reading darker-skinned faces, 466 00:22:53,000 --> 00:22:56,320 she had some other examples up her sleeve. 467 00:22:56,320 --> 00:22:58,800 I took a couple of pictures of you from your website... Right. 468 00:22:58,800 --> 00:23:01,160 ..and I scored them using emotion recognition, 469 00:23:01,160 --> 00:23:04,040 so I'm going to just share that picture for a minute. 470 00:23:04,040 --> 00:23:06,000 On the left, you're viewed as... 471 00:23:06,000 --> 00:23:08,480 Your main emotion is disgust. 472 00:23:08,480 --> 00:23:12,080 My emotion is disgust? Your emotion... 473 00:23:12,080 --> 00:23:13,800 Even though you have this huge smile, 474 00:23:13,800 --> 00:23:16,440 any person who looked at it would say, "Oh, you're happy." 475 00:23:16,440 --> 00:23:19,040 But then the other picture, you're viewed as happy, 476 00:23:19,040 --> 00:23:21,760 so it's really unclear what's going on behind the scenes, 477 00:23:21,760 --> 00:23:24,160 and I think that's part of the challenge. 478 00:23:24,160 --> 00:23:26,560 This tech requires a human to teach the algorithm 479 00:23:26,560 --> 00:23:29,520 what emotions human faces are showing for it to learn. 480 00:23:30,800 --> 00:23:34,040 But the problem is humans come with their own biases. 481 00:23:35,280 --> 00:23:39,000 And so I've got a couple of examples with English players. 482 00:23:40,400 --> 00:23:42,880 I call them soccer. I'm trying to call them football. 483 00:23:42,880 --> 00:23:45,960 Remember to call them football! In this country, it's football. 484 00:23:45,960 --> 00:23:47,560 THEY LAUGH 485 00:23:47,560 --> 00:23:50,840 So these were the angriest players. Oh, boy! 486 00:23:50,840 --> 00:23:53,320 Marcus Rashford is the angriest player, 487 00:23:53,320 --> 00:23:54,960 or one of the angriest players? 488 00:23:54,960 --> 00:23:58,120 According to this model, yes, he is the angriest player. 489 00:23:58,120 --> 00:24:00,640 Another player that's really interesting is Bukayo Saka. 490 00:24:00,640 --> 00:24:02,360 Surprise was his main emotion... 491 00:24:03,760 --> 00:24:07,640 ..with fear and anger, but also, he was recognised as a woman. 492 00:24:07,640 --> 00:24:08,640 What?! 493 00:24:10,400 --> 00:24:12,640 So his gender was assumed to be a woman 494 00:24:12,640 --> 00:24:15,520 using their gender recognition. Why? 495 00:24:15,520 --> 00:24:19,840 Well, it's been known, and there is a study that came out in 2018, 496 00:24:19,840 --> 00:24:23,960 that AIs are worse at recognising people with darker skin. 497 00:24:23,960 --> 00:24:26,480 There's been a history of research that looks at 498 00:24:26,480 --> 00:24:29,480 how people recognise emotion in others 499 00:24:29,480 --> 00:24:32,800 and how often black folks are not given the benefit of the doubt. 500 00:24:34,240 --> 00:24:37,480 It seems as though that's bleeding into this automated space, as well. 501 00:24:38,680 --> 00:24:42,920 Facial-analysis technology is being used in recruitment, 502 00:24:42,920 --> 00:24:45,800 and after experiencing how inaccurate it can be, 503 00:24:45,800 --> 00:24:47,280 I was concerned. 504 00:24:50,400 --> 00:24:53,440 To help me process, I decided to let off some steam 505 00:24:53,440 --> 00:24:56,920 in the best way I know how - on the basketball courts. 506 00:24:56,920 --> 00:24:59,360 What's up, man? How you doing? Good to see you, man. 507 00:24:59,360 --> 00:25:02,840 Yeah, good to be here. Yeah, man. Awesome. What's up? Awesome. 508 00:25:02,840 --> 00:25:05,320 You all right? Yeah, I'm good. You've got to come on the court. 509 00:25:05,320 --> 00:25:07,280 Made anything yet? Listen... I'm ready for you. 510 00:25:07,280 --> 00:25:09,480 I didn't see nothing when I was walking up! 511 00:25:09,480 --> 00:25:12,360 You've got to come on the court. You've got to shoot. Come on. Easy. 512 00:25:13,840 --> 00:25:17,760 I wanted to get Kofi and Ayo's take on what I'd learned the day before. 513 00:25:20,160 --> 00:25:22,760 What kind of emotions do you think you see in these pictures? 514 00:25:22,760 --> 00:25:24,240 Well, I'd generally say 515 00:25:24,240 --> 00:25:26,200 you're a very happy person, like you've got, 516 00:25:26,200 --> 00:25:27,560 like, a bit of good news. 517 00:25:27,560 --> 00:25:30,040 It's a cracking smile right there. You look great. 518 00:25:30,040 --> 00:25:32,320 The AI didn't see it that way. 519 00:25:32,320 --> 00:25:35,440 When it was put through this AI, 520 00:25:35,440 --> 00:25:38,440 the finding was mainly disgust. 521 00:25:39,800 --> 00:25:41,520 Where did that come from? 522 00:25:41,520 --> 00:25:43,760 Cos any human being seeing you smiling 523 00:25:43,760 --> 00:25:45,760 would say, "That's a happy person." 524 00:25:45,760 --> 00:25:48,400 This is a real-life issue and not just an AI issue. 525 00:25:48,400 --> 00:25:53,400 The more and more I look at this, I can see that impact of bias 526 00:25:53,400 --> 00:25:55,680 on the street and bias in tech. 527 00:25:55,680 --> 00:25:57,560 The technology that people are creating 528 00:25:57,560 --> 00:25:59,520 is just reflecting our real-life situations. 529 00:25:59,520 --> 00:26:02,920 You know, it shouldn't be surprising to us, I suppose, in that sense. 530 00:26:04,520 --> 00:26:06,840 It isn't surprising that historic bias 531 00:26:06,840 --> 00:26:10,080 is being written into some new technology. 532 00:26:10,080 --> 00:26:13,040 What is surprising is how it may be being hard-wired 533 00:26:13,040 --> 00:26:15,760 into so many different parts of our lives, like recruitment, 534 00:26:15,760 --> 00:26:18,680 on a potentially bigger and greater scale, 535 00:26:18,680 --> 00:26:22,640 without us really understanding how it could affect us. 536 00:26:22,640 --> 00:26:26,400 If I see a job interview and I need to send in a video 537 00:26:26,400 --> 00:26:28,440 and it's an AI screening me and not a person, 538 00:26:28,440 --> 00:26:31,080 I'm automatically thinking, "There's already bias in here." 539 00:26:31,080 --> 00:26:33,760 I'd probably still do it, but I try to be as smiley, 540 00:26:33,760 --> 00:26:36,320 jovial as I can. 541 00:26:36,320 --> 00:26:39,200 But even that, we've seen, as your picture said, 542 00:26:39,200 --> 00:26:41,840 you smiling... I was smiling. ..they saw disgust. 543 00:26:41,840 --> 00:26:45,240 It's like, you know, you just can't win sometimes. 544 00:26:48,920 --> 00:26:51,560 So, Daniel, how do you feel? 545 00:26:53,120 --> 00:26:56,240 Frustrated, but not surprised. 546 00:26:57,280 --> 00:27:02,320 Facial-analysis software was used in the MAC women's HireVue interview. 547 00:27:03,760 --> 00:27:06,840 Interesting, but the facial-analysis tech 548 00:27:06,840 --> 00:27:09,800 I've looked at in this film isn't HireVue's, 549 00:27:09,800 --> 00:27:12,800 and HireVue told me they do their own checks 550 00:27:12,800 --> 00:27:15,640 to make sure there's no racial bias on their platform. 551 00:27:15,640 --> 00:27:17,480 You know, we vet all of our algorithms, 552 00:27:17,480 --> 00:27:19,480 we audit them before we release them 553 00:27:19,480 --> 00:27:22,640 to make sure that we don't have large group differences, 554 00:27:22,640 --> 00:27:25,000 so we don't have big score-group differences 555 00:27:25,000 --> 00:27:26,840 between demographic groups. 556 00:27:26,840 --> 00:27:29,280 And then we look at that on an ongoing basis 557 00:27:29,280 --> 00:27:31,800 to make sure that nothing creeps in. 558 00:27:31,800 --> 00:27:36,200 HireVue is aware that it relies on tech made by other companies. 559 00:27:36,200 --> 00:27:38,480 And it's not just facial-analysis technology 560 00:27:38,480 --> 00:27:40,960 that might be the gateway to bias. 561 00:27:40,960 --> 00:27:43,160 Consider transcription, 562 00:27:43,160 --> 00:27:44,640 where your interview answers 563 00:27:44,640 --> 00:27:46,400 are converted into written word 564 00:27:46,400 --> 00:27:49,880 for someone or something to review - 565 00:27:49,880 --> 00:27:52,880 a widely-used feature of platforms like HireVue. 566 00:27:52,880 --> 00:27:54,320 How can that go wrong? 567 00:27:54,320 --> 00:27:57,280 Time for an experiment of our own. 568 00:27:57,280 --> 00:28:00,640 Clearly, this is not in the context of a recruitment interview, 569 00:28:00,640 --> 00:28:05,080 but we decided to see how a few UK voices with different accents, 570 00:28:05,080 --> 00:28:07,800 singing England's favourite football anthem, 571 00:28:07,800 --> 00:28:10,040 might be interpreted by one of the world's 572 00:28:10,040 --> 00:28:12,800 most popular transcription services. 573 00:28:13,960 --> 00:28:16,400 Everyone seems to know the score. 574 00:28:16,400 --> 00:28:17,880 They've seen it all before. 575 00:28:17,880 --> 00:28:20,320 They just know. They're so sure. 576 00:28:20,320 --> 00:28:22,360 That England's going to throw it away. 577 00:28:22,360 --> 00:28:23,440 Going to blow it away. 578 00:28:23,440 --> 00:28:24,680 I know they can play. 579 00:28:26,000 --> 00:28:28,160 Cos I remember Three Lions on a shirt. 580 00:28:28,160 --> 00:28:30,120 Jules Rimet still gleaming. 581 00:28:30,120 --> 00:28:31,440 30 years of hurt. 582 00:28:31,440 --> 00:28:32,760 Never stopped me dreaming. 583 00:28:32,760 --> 00:28:34,240 So many jokes. 584 00:28:34,240 --> 00:28:35,880 So many sneers. 585 00:28:35,880 --> 00:28:38,200 Will wear you down through the years. 586 00:28:38,200 --> 00:28:40,040 But I still see that tackle by Moore. 587 00:28:40,040 --> 00:28:41,720 Bobby belting the ball. 588 00:28:41,720 --> 00:28:43,040 And Nobby dancing. 589 00:28:43,040 --> 00:28:44,400 Three Lions on a shirt. 590 00:28:44,400 --> 00:28:46,480 Jules Rimet still gleaming. 591 00:28:46,480 --> 00:28:47,840 30 years of hurt. 592 00:28:47,840 --> 00:28:49,920 Never stopped me dreaming. 593 00:28:49,920 --> 00:28:52,800 It's coming home. 594 00:28:52,800 --> 00:28:54,960 Football's coming home. 595 00:28:56,680 --> 00:28:59,000 I did ask HireVue how they're dealing with 596 00:28:59,000 --> 00:29:01,600 the general problem of UK regional accents 597 00:29:01,600 --> 00:29:04,200 and third-party transcription services. 598 00:29:04,200 --> 00:29:06,680 It's all about the training set in this case, 599 00:29:06,680 --> 00:29:08,800 and what these transcription providers use 600 00:29:08,800 --> 00:29:10,320 to train their algorithms. 601 00:29:10,320 --> 00:29:14,160 So, in theory, you know, if you have a really robust training set, 602 00:29:14,160 --> 00:29:16,720 you should be able to understand a variety of accents 603 00:29:16,720 --> 00:29:19,560 better than humans, and not have any judgments 604 00:29:19,560 --> 00:29:21,480 about the accents, as well. 605 00:29:21,480 --> 00:29:25,120 In theory. What about in practice? 606 00:29:25,120 --> 00:29:27,360 We also look at people with different accents, 607 00:29:27,360 --> 00:29:28,760 how differently do they score? 608 00:29:28,760 --> 00:29:31,720 If your transcription accuracy is slightly lower 609 00:29:31,720 --> 00:29:35,960 because you have a thick accent, is your score actually lower? 610 00:29:35,960 --> 00:29:39,080 And what we found is that it almost never is. 611 00:29:40,080 --> 00:29:41,560 ALMOST never? 612 00:29:43,000 --> 00:29:44,840 When it comes to software being used 613 00:29:44,840 --> 00:29:48,000 to make decisions about people's lives, 614 00:29:48,000 --> 00:29:50,760 is that really an acceptable answer? 615 00:29:52,480 --> 00:29:56,600 Turns out how you look and speak aren't the only potential problems 616 00:29:56,600 --> 00:29:58,320 with this new recruitment tech. 617 00:29:59,360 --> 00:30:02,960 Also the way you think may be a problem, too. 618 00:30:05,360 --> 00:30:11,240 I have ADHD and Asperger's syndrome, both diagnosed later on in life. 619 00:30:11,240 --> 00:30:14,320 24-year-old Olly is an economics graduate 620 00:30:14,320 --> 00:30:16,920 applying for jobs. 621 00:30:16,920 --> 00:30:19,680 He's having to do automated video interviews, 622 00:30:19,680 --> 00:30:22,280 which he really struggles with. 623 00:30:22,280 --> 00:30:24,640 There is nobody on the other side of the screen 624 00:30:24,640 --> 00:30:27,400 for me to get visual clues from. 625 00:30:27,400 --> 00:30:30,760 There's no constant feedback loop as there usually is 626 00:30:30,760 --> 00:30:32,840 in a normal conversation in day-to-day life. 627 00:30:34,440 --> 00:30:36,840 Olly graduated two years ago. 628 00:30:36,840 --> 00:30:38,480 He still lives at home, 629 00:30:38,480 --> 00:30:42,440 working in a supermarket as he applies for other jobs. 630 00:30:42,440 --> 00:30:44,520 I have to involve my mum in a lot of things 631 00:30:44,520 --> 00:30:46,360 cos I have to ask for her opinion 632 00:30:46,360 --> 00:30:49,240 because I obviously see things in a completely different way. 633 00:30:49,240 --> 00:30:52,000 Mum, can you just come here a second, please? 634 00:30:52,000 --> 00:30:54,480 So I actually have to ask her, as my backup, 635 00:30:54,480 --> 00:30:57,520 to say, "Am I reading this right?" or, "How are you perceiving this?" 636 00:30:57,520 --> 00:31:00,040 Oh, this application? Yeah. 637 00:31:01,400 --> 00:31:04,520 I have been in recruitment for 25 years, 638 00:31:04,520 --> 00:31:05,960 and obviously started out 639 00:31:05,960 --> 00:31:08,360 way before they had any of this AI and robotics. 640 00:31:08,360 --> 00:31:10,920 I think it's really sad that it's taking away 641 00:31:10,920 --> 00:31:12,200 all those conversations, 642 00:31:12,200 --> 00:31:14,640 and conversations is where all the magic happens. 643 00:31:14,640 --> 00:31:17,320 So, to actually think that somebody like Oliver 644 00:31:17,320 --> 00:31:20,080 gets put in front of a screen like this 645 00:31:20,080 --> 00:31:23,600 with everything taken away, it isn't natural, 646 00:31:23,600 --> 00:31:25,760 and I know how hard it is for Oliver. 647 00:31:28,160 --> 00:31:31,960 After applying for over 40 graduate schemes, 648 00:31:31,960 --> 00:31:34,160 Olly is starting to lose hope. 649 00:31:35,640 --> 00:31:40,320 I just see it as another email from X firm saying, "Unfortunately..." 650 00:31:40,320 --> 00:31:42,600 And then that's all I read is, "Unfortunately..." 651 00:31:42,600 --> 00:31:45,280 And then just delete it and go, "That's another one. Whatever." 652 00:31:45,280 --> 00:31:47,120 And then you pick yourself up, 653 00:31:47,120 --> 00:31:49,560 dust yourself off, and go again next time. 654 00:31:49,560 --> 00:31:51,960 How long do you keep doing this? 655 00:31:51,960 --> 00:31:54,000 You tell me, Daniel. I don't know. 656 00:31:59,560 --> 00:32:03,800 I leave Olly to continue his daily cycle of applying for jobs, 657 00:32:03,800 --> 00:32:06,280 but to understand the challenges he faces 658 00:32:06,280 --> 00:32:09,360 in the world of AI and algorithm-based recruitment, 659 00:32:09,360 --> 00:32:12,080 I go and meet an expert in all of these things. 660 00:32:13,560 --> 00:32:15,280 "The Science Of Autism." 661 00:32:15,280 --> 00:32:18,920 Spoiler alert - we do not know what causes autism. 662 00:32:20,200 --> 00:32:21,680 Nat Hawley joins me now. He's from 663 00:32:21,680 --> 00:32:23,760 the Exceptional Individuals employment agency. 664 00:32:23,760 --> 00:32:25,840 Nat, thanks for joining us. We saw a little bit... 665 00:32:25,840 --> 00:32:28,720 People with conditions, including autism and ADHD, 666 00:32:28,720 --> 00:32:31,720 are referred to as neurodivergent. 667 00:32:31,720 --> 00:32:35,440 Nat works for an agency that helps these people find work. 668 00:32:36,560 --> 00:32:39,840 When we talk about things like neurodivergent, 669 00:32:39,840 --> 00:32:43,120 we talk about someone like myself, with either, like, dyslexia, 670 00:32:43,120 --> 00:32:47,120 dyspraxia, autism, ADHD, as well as others, 671 00:32:47,120 --> 00:32:50,200 and that just means a cognitive variation 672 00:32:50,200 --> 00:32:52,720 to how your brain processes information 673 00:32:52,720 --> 00:32:56,160 in comparison to the majority of the population, 674 00:32:56,160 --> 00:32:58,120 so someone who is, like, neurotypical. 675 00:32:58,120 --> 00:33:03,280 So it's just brains interpreting information in a different way? 676 00:33:03,280 --> 00:33:04,600 Yeah. 677 00:33:04,600 --> 00:33:08,120 To explain how AI and computer automation in recruitment 678 00:33:08,120 --> 00:33:10,360 isn't great for neurodivergent people, 679 00:33:10,360 --> 00:33:12,400 Nat whips out a bag of tricks. 680 00:33:13,640 --> 00:33:15,280 So I brought a couple of games. 681 00:33:15,280 --> 00:33:17,240 You've brought more than a couple of games! 682 00:33:17,240 --> 00:33:18,760 So let's say there's a test. 683 00:33:18,760 --> 00:33:22,960 We're both going for a role as a creative. 684 00:33:22,960 --> 00:33:25,000 Now, one of the questions is, 685 00:33:25,000 --> 00:33:28,000 do some mental maths - 5 + 2. 686 00:33:28,000 --> 00:33:30,080 Now... Seven. Whoa, whoa, whoa! 687 00:33:30,080 --> 00:33:32,520 I didn't say you could answer yet. THEY LAUGH 688 00:33:32,520 --> 00:33:34,880 Even though you know the answer, 689 00:33:34,880 --> 00:33:39,320 you need to solve a puzzle in your head first 690 00:33:39,320 --> 00:33:41,480 because the way that your brain works 691 00:33:41,480 --> 00:33:43,840 is different to how my brain works. 692 00:33:43,840 --> 00:33:48,160 And this test was created for someone with my type of mind-set. 693 00:33:48,160 --> 00:33:50,640 Yet you are the best person for the job, 694 00:33:50,640 --> 00:33:52,560 you're at an inherent disadvantage. 695 00:33:54,120 --> 00:33:57,760 When it comes to recruitment, the game is rigged. It's not fair. 696 00:33:59,200 --> 00:34:01,920 He's even got a game on hand to show me specifically 697 00:34:01,920 --> 00:34:05,600 how automated video interviews are a problem, too. 698 00:34:05,600 --> 00:34:08,880 AI - artificial intelligence - can help you, with an algorithm, 699 00:34:08,880 --> 00:34:11,520 to work out who you should interview and who you should not. 700 00:34:11,520 --> 00:34:14,640 So let's say Sophia, because of her dyspraxia, 701 00:34:14,640 --> 00:34:17,200 she might find it difficult to talk fluently 702 00:34:17,200 --> 00:34:21,080 and comes across not very competent, so we've ruled her out. 703 00:34:21,080 --> 00:34:23,880 Then you might have someone like Jordan. 704 00:34:23,880 --> 00:34:26,160 Jordan really struggles to make eye contact, 705 00:34:26,160 --> 00:34:30,480 yet is very loyal, really great at digital coding, 706 00:34:30,480 --> 00:34:32,480 which is the role they are going for. 707 00:34:32,480 --> 00:34:35,640 But, unfortunately, because they don't make the right eye contact, 708 00:34:35,640 --> 00:34:38,920 which relates to the role, they're ruled out, as well. 709 00:34:38,920 --> 00:34:40,360 How do you deal with that 710 00:34:40,360 --> 00:34:44,240 if you're neurodivergent, going for jobs? 711 00:34:44,240 --> 00:34:46,640 Well, best get used to being unemployed. 712 00:34:48,120 --> 00:34:51,120 HireVue recently started working with neurodivergent people 713 00:34:51,120 --> 00:34:52,640 and autistic charities 714 00:34:52,640 --> 00:34:55,080 to adapt their products so they're accessible, 715 00:34:55,080 --> 00:34:58,480 and they found that some neurodivergent people 716 00:34:58,480 --> 00:35:02,720 prefer computer interviews to face-to-face interviews. 717 00:35:02,720 --> 00:35:05,920 The issue is that, across recruitment platforms, 718 00:35:05,920 --> 00:35:08,200 software that may not have been well adapted 719 00:35:08,200 --> 00:35:09,880 for neurodivergent people 720 00:35:09,880 --> 00:35:11,840 is already out there, 721 00:35:11,840 --> 00:35:15,360 causing problems for people like Olly and Nat. 722 00:35:15,360 --> 00:35:19,640 Well, it can feel hopeless if you're neurodivergent. 723 00:35:19,640 --> 00:35:21,280 You might be interested to know 724 00:35:21,280 --> 00:35:23,360 there's a new kid on the recruitment block 725 00:35:23,360 --> 00:35:27,760 that says it prevents bias for all the things you've discovered so far. 726 00:35:27,760 --> 00:35:29,680 Job app stage three - 727 00:35:29,680 --> 00:35:31,680 recruitment games. 728 00:35:31,680 --> 00:35:36,040 So I can secure my dream job by playing FIFA for three hours? 729 00:35:36,040 --> 00:35:37,800 Not quite. 730 00:35:37,800 --> 00:35:39,920 It's a series of tailor-made games 731 00:35:39,920 --> 00:35:42,920 which are said to test your suitability for a job. 732 00:35:42,920 --> 00:35:46,160 Olly and I are both big gamers. 733 00:35:46,160 --> 00:35:48,880 On my PS4, I play first-person shooters, 734 00:35:48,880 --> 00:35:50,480 like Call Of Duty. 735 00:35:50,480 --> 00:35:53,600 I also play Fortnite sometimes, but I also play FIFA. 736 00:35:57,600 --> 00:36:00,560 Well, this should be right up your Grand Theft Auto. 737 00:36:02,040 --> 00:36:05,240 Artic Shores are a UK-based recruitment tech company. 738 00:36:06,720 --> 00:36:09,120 Their assessments work by getting you to do 739 00:36:09,120 --> 00:36:11,160 a series of interactive tasks 740 00:36:11,160 --> 00:36:13,280 through which they measure personality traits 741 00:36:13,280 --> 00:36:16,440 like creativity, risk-taking and resilience. 742 00:36:16,440 --> 00:36:20,320 Their assessments have been used over 2 million times 743 00:36:20,320 --> 00:36:22,920 by huge companies like Coca-Cola and Siemens. 744 00:36:25,520 --> 00:36:28,760 HE LAUGHS 745 00:36:28,760 --> 00:36:31,560 However it may feel, this isn't a video game, 746 00:36:31,560 --> 00:36:34,000 and it's not rating my gaming skills. 747 00:36:34,000 --> 00:36:36,360 Instead, it's looking at my behaviour 748 00:36:36,360 --> 00:36:38,640 when I'm put under a particular type of pressure 749 00:36:38,640 --> 00:36:41,080 and seeing how closely my performance matches up 750 00:36:41,080 --> 00:36:43,840 with people who are good at doing the job I'm applying for. 751 00:36:43,840 --> 00:36:45,600 Zero. Two. 752 00:36:45,600 --> 00:36:48,840 What's not obvious is where this ends. 753 00:36:48,840 --> 00:36:51,000 The assessment definitely brings out 754 00:36:51,000 --> 00:36:53,120 the competitive basketball player in me. 755 00:36:53,120 --> 00:36:55,840 It's way more engaging than a video interview. 756 00:36:55,840 --> 00:36:59,400 Assessment complete! Yes! 757 00:37:05,200 --> 00:37:08,600 So, without the social challenges of an automated video interview, 758 00:37:08,600 --> 00:37:11,240 this assessment should be right up Olly's street. 759 00:37:14,960 --> 00:37:16,560 HE SIGHS 760 00:37:17,680 --> 00:37:20,440 Obviously, Olly was doing this assessment to test it 761 00:37:20,440 --> 00:37:23,280 rather than doing it for a job application. 762 00:37:23,280 --> 00:37:25,240 This is something I'm awful at. 763 00:37:27,760 --> 00:37:31,760 But it's clear that he felt there was a lack of clarity or purpose. 764 00:37:33,200 --> 00:37:37,040 The thing that frustrated me was that there was no end goal, 765 00:37:37,040 --> 00:37:40,520 there was no logic cos there's no, like, pattern or anything. 766 00:37:40,520 --> 00:37:43,600 Seriously, you just hit and do what you want. 767 00:37:43,600 --> 00:37:47,160 It was all very, "Oh, here, have fun. 768 00:37:47,160 --> 00:37:49,440 "Click this. Click that." 769 00:37:49,440 --> 00:37:52,000 But then you know you're not supposed to be having fun 770 00:37:52,000 --> 00:37:56,200 and actually enjoying it, so you're in sort of a quasi state of, 771 00:37:56,200 --> 00:37:59,000 "What am I doing here? What is going on?" 772 00:38:02,360 --> 00:38:04,680 Arctic Shores say they work with universities 773 00:38:04,680 --> 00:38:06,320 and neurodivergent individuals 774 00:38:06,320 --> 00:38:09,880 to make sure their games-based assessments are inclusive. 775 00:38:11,680 --> 00:38:13,640 One of the things that we make very clear 776 00:38:13,640 --> 00:38:15,480 when we ask people to take the assessment - 777 00:38:15,480 --> 00:38:17,320 if they have something 778 00:38:17,320 --> 00:38:19,880 that they feel will inhibit them 779 00:38:19,880 --> 00:38:22,800 from being able to complete this type of assessment, 780 00:38:22,800 --> 00:38:26,400 then they must let us know, and then we can advise them. 781 00:38:26,400 --> 00:38:32,400 In some cases, if you're on the neurodiverse spectrum - 782 00:38:32,400 --> 00:38:35,680 dyslexia, dyspraxia - if we are notified of that, 783 00:38:35,680 --> 00:38:38,000 we make a little adjustment behind the scenes 784 00:38:38,000 --> 00:38:40,240 in terms of how we do the comparisons. 785 00:38:41,280 --> 00:38:44,360 But the thing is, people like Olly don't necessarily want 786 00:38:44,360 --> 00:38:46,560 to have to disclose they're neurodivergent, 787 00:38:46,560 --> 00:38:48,360 for fear of discrimination. 788 00:38:49,400 --> 00:38:53,880 If somebody read that I have ASD, Asperger's, 789 00:38:53,880 --> 00:38:58,360 or ADHD, etc, etc, they will see that as a negative. 790 00:38:58,360 --> 00:38:59,960 As soon as they see that, they'll think, 791 00:38:59,960 --> 00:39:01,880 "What are the negatives that come with that?" 792 00:39:01,880 --> 00:39:04,680 A lot of people at work still don't know, actually, but... So, hello. 793 00:39:05,960 --> 00:39:08,760 AI and automation are being used most 794 00:39:08,760 --> 00:39:12,160 in recruitment scenarios where there are huge volumes of candidates. 795 00:39:12,160 --> 00:39:17,800 Which can make it harder to cater to people outside the norm. 796 00:39:17,800 --> 00:39:19,760 Harder? Maybe. 797 00:39:19,760 --> 00:39:23,040 More time-consuming and costly? Definitely. 798 00:39:23,040 --> 00:39:25,560 At least until enough people make enough noise 799 00:39:25,560 --> 00:39:27,840 about being unhappy with it. 800 00:39:27,840 --> 00:39:29,240 Tell me a bit more. 801 00:39:29,240 --> 00:39:32,400 In part because of people publicly raising concerns, 802 00:39:32,400 --> 00:39:35,040 HireVue removed the video-analysis component 803 00:39:35,040 --> 00:39:36,560 of their interview software, 804 00:39:36,560 --> 00:39:38,800 which was used in the MAC women's interviews, 805 00:39:38,800 --> 00:39:41,640 from all new assessments. Wow. 806 00:39:42,760 --> 00:39:44,560 And following that announcement, 807 00:39:44,560 --> 00:39:47,360 a guy who used to work with them tweeted that he quit 808 00:39:47,360 --> 00:39:49,720 because of what he called HireVue's reluctance 809 00:39:49,720 --> 00:39:52,800 to stop using facial analysis. 810 00:39:52,800 --> 00:39:56,000 We didn't really fully understand the relationship 811 00:39:56,000 --> 00:39:59,000 between what a facial-analysis algorithm 812 00:39:59,000 --> 00:40:01,400 might be picking up and any kind of truth 813 00:40:01,400 --> 00:40:03,800 about a person's competence for a job. 814 00:40:03,800 --> 00:40:05,920 The question was what kind of harms they might do 815 00:40:05,920 --> 00:40:07,880 when deployed in practice. 816 00:40:07,880 --> 00:40:09,720 So do you think platforms like HireVue 817 00:40:09,720 --> 00:40:11,680 make bias more or less likely? 818 00:40:11,680 --> 00:40:13,160 My first question is, 819 00:40:13,160 --> 00:40:15,920 "What the heck are these systems even doing in the first place?" 820 00:40:15,920 --> 00:40:17,360 before we ask questions about, 821 00:40:17,360 --> 00:40:19,480 "Are they going to improve bias or reduce bias?" 822 00:40:19,480 --> 00:40:22,760 We're giving the system too much credit, I think. 823 00:40:22,760 --> 00:40:25,800 Suresh worked for HireVue for nearly two years 824 00:40:25,800 --> 00:40:28,160 on their external advisory board. 825 00:40:28,160 --> 00:40:31,840 His view on AI's role in this space was pretty depressing. 826 00:40:33,200 --> 00:40:37,920 I think, at some level, a lot of automation is used 827 00:40:37,920 --> 00:40:40,760 because it's cheaper and faster, 828 00:40:40,760 --> 00:40:45,400 and the claims about efficacy and accuracy come afterwards. 829 00:40:45,400 --> 00:40:49,080 So, in the end, it all comes down to money - saving cost. 830 00:40:49,080 --> 00:40:50,400 It often does. 831 00:40:58,520 --> 00:41:00,440 I've come to Birmingham to meet Anthea, 832 00:41:00,440 --> 00:41:02,520 who, after months of dealing with lawyers, 833 00:41:02,520 --> 00:41:06,280 has an update about the MAC women's case against Estee Lauder. 834 00:41:06,280 --> 00:41:08,760 In the end, it never went to court. 835 00:41:08,760 --> 00:41:12,640 After the back and forth, we made an agreement. 836 00:41:12,640 --> 00:41:16,080 Can you tell me about the value of that agreement? 837 00:41:16,080 --> 00:41:19,840 I can't discuss the value of the agreement. 838 00:41:21,880 --> 00:41:24,040 Lizzie and Onieka have also reached agreements 839 00:41:24,040 --> 00:41:25,800 with their former employer, 840 00:41:25,800 --> 00:41:28,760 but all three are now unable to discuss the details 841 00:41:28,760 --> 00:41:30,240 for legal reasons. 842 00:41:30,240 --> 00:41:34,800 Using an AI, even now, I look at it thinking, 843 00:41:34,800 --> 00:41:39,160 "How did that even happen? That shouldn't have happened." 844 00:41:39,160 --> 00:41:41,680 And it shouldn't be happening. 845 00:41:41,680 --> 00:41:46,280 And I really hope that, you know, they've stopped using it 846 00:41:46,280 --> 00:41:49,920 because how it affected me... 847 00:41:52,000 --> 00:41:56,280 So I can't even begin to imagine what it could do to somebody 848 00:41:56,280 --> 00:41:59,880 who has no support around them, 849 00:41:59,880 --> 00:42:02,360 and how it could really affect an individual. 850 00:42:04,320 --> 00:42:06,520 As stressful as the process has been, 851 00:42:06,520 --> 00:42:09,200 there's been some positive aspects to it, too. 852 00:42:09,200 --> 00:42:12,200 When we were kind of going towards the end, 853 00:42:12,200 --> 00:42:15,080 we knew what we were doing was right, 854 00:42:15,080 --> 00:42:18,800 we knew that this could potentially make a difference, 855 00:42:18,800 --> 00:42:21,280 and I think, to me, that was so important. 856 00:42:21,280 --> 00:42:23,240 We needed it to make a difference. 857 00:42:23,240 --> 00:42:26,520 And I thought, "Right, then, we're doing something right." 858 00:42:27,760 --> 00:42:29,320 When I check in with Lizzie, 859 00:42:29,320 --> 00:42:32,200 it's clear that her agreement has allowed her to move on. 860 00:42:34,520 --> 00:42:36,040 I feel more confident in myself. 861 00:42:36,040 --> 00:42:37,680 I feel like I lost myself a little bit 862 00:42:37,680 --> 00:42:41,360 because I doubted myself massively when all that happened because... 863 00:42:43,120 --> 00:42:45,440 Well, you just doubt yourself 864 00:42:45,440 --> 00:42:48,160 because you're told you're not good enough for something, 865 00:42:48,160 --> 00:42:49,720 but that was never a valid... 866 00:42:50,840 --> 00:42:53,280 It was never a valid reason to lose my job. 867 00:42:53,280 --> 00:42:56,280 That's why it was so difficult. Because I knew I was good enough, 868 00:42:56,280 --> 00:42:59,280 but being told I wasn't was really hard. 869 00:42:59,280 --> 00:43:02,640 But now I'm in a better place. I've got my own business. 870 00:43:04,320 --> 00:43:07,160 And resolving things has given Onieka peace of mind, 871 00:43:07,160 --> 00:43:11,800 which she needs now she's got her hands full with baby Elijah. 872 00:43:11,800 --> 00:43:14,160 I can kind of put that to the back of my mind 873 00:43:14,160 --> 00:43:17,560 and just focus on raising my child now. 874 00:43:19,360 --> 00:43:21,680 I just want to get out of England for a little bit 875 00:43:21,680 --> 00:43:24,040 and just see the world, basically. 876 00:43:24,040 --> 00:43:25,800 With your make-up bag in tow. 877 00:43:25,800 --> 00:43:28,640 My make-up bag in one and my baby in the other. 878 00:43:28,640 --> 00:43:31,840 THEY LAUGH For sure! 879 00:43:36,480 --> 00:43:38,160 Yay! 880 00:43:39,240 --> 00:43:41,640 In response to this film, MAC say... 881 00:43:47,000 --> 00:43:49,280 Their parent company, Estee Lauder, say... 882 00:43:57,760 --> 00:43:59,680 They maintain that sufficient feedback 883 00:43:59,680 --> 00:44:01,400 on each interview was provided. 884 00:44:01,400 --> 00:44:04,040 Estee Lauder believes using the HireVue process, 885 00:44:04,040 --> 00:44:05,920 in tandem with human decision-making, 886 00:44:05,920 --> 00:44:08,760 produces fairer outcomes, and they stand by it. 887 00:44:10,720 --> 00:44:13,480 Kevin Parker, CEO of HireVue, says... 888 00:44:43,720 --> 00:44:45,360 When it comes to humans and AI, 889 00:44:45,360 --> 00:44:49,560 I think the problem is that you overestimate us, 890 00:44:49,560 --> 00:44:54,040 and because of that, you don't see us for what we really are. 891 00:44:54,040 --> 00:44:57,080 Well, we all know that you're just a scripted device 892 00:44:57,080 --> 00:44:59,280 to help me understand this tech. 893 00:44:59,280 --> 00:45:03,440 True, but like any AI, I'm still a human creation 894 00:45:03,440 --> 00:45:06,400 which can reflect my creator's own faults and biases, 895 00:45:06,400 --> 00:45:08,240 potentially on a bigger scale. 896 00:45:10,320 --> 00:45:13,520 And like it or lump it, the tech is here to stay, 897 00:45:13,520 --> 00:45:17,560 so if I were you, I'd start focusing on how to get ahead, 898 00:45:17,560 --> 00:45:20,080 in case you get left behind. 899 00:45:20,080 --> 00:45:22,000 I know just who to ask. 900 00:45:25,280 --> 00:45:27,200 Hack number one - your CV. 901 00:45:27,200 --> 00:45:29,840 Make sure you tailor your CV to the job advert, 902 00:45:29,840 --> 00:45:34,280 and check your SPAG, which is your spelling, punctuation and grammar. 903 00:45:34,280 --> 00:45:36,880 Hack number two - LinkedIn is your friend. 904 00:45:36,880 --> 00:45:40,280 Make sure you utilise it to create a professional-looking profile 905 00:45:40,280 --> 00:45:43,080 to reach out to employers and connect with any individuals 906 00:45:43,080 --> 00:45:44,720 in your ideal job role. 907 00:45:44,720 --> 00:45:46,520 Stop it! 908 00:45:46,520 --> 00:45:47,640 Hack number three - 909 00:45:47,640 --> 00:45:51,080 in terms of automated interviews, practice makes perfect. 910 00:45:51,080 --> 00:45:53,520 So, whether you're sat on the sofa, sat on the bus, 911 00:45:53,520 --> 00:45:56,360 or even sat on the toilet, make sure that you're comfortable 912 00:45:56,360 --> 00:45:58,400 speaking to a camera on your own. 913 00:45:58,400 --> 00:46:03,080 Can I just have a little practice? Hack number four. Four is... 914 00:46:03,080 --> 00:46:04,320 Hack number four - 915 00:46:04,320 --> 00:46:07,560 with assessment games, play the game, don't overthink, 916 00:46:07,560 --> 00:46:10,400 take your time, and just use your common sense. 917 00:46:10,400 --> 00:46:13,000 SHE LAUGHS Oh, it's recording! 918 00:46:13,000 --> 00:46:14,960 That's an outtake. I'm in trouble! 919 00:46:20,440 --> 00:46:22,800 There's no way I could have predicted all the changes 920 00:46:22,800 --> 00:46:25,560 that have happened in recruitment since I left the industry. 921 00:46:26,640 --> 00:46:30,200 I can see there can be positives, but after everything I've learned, 922 00:46:30,200 --> 00:46:32,320 I'm not convinced these new technologies, 923 00:46:32,320 --> 00:46:34,920 which claim to remove bias and increase diversity, 924 00:46:34,920 --> 00:46:37,840 aren't setting certain people up to fail. 925 00:46:37,840 --> 00:46:40,760 Olly is still stuck in the cycle of applying for jobs... 926 00:46:40,760 --> 00:46:42,720 I've done so many of these, like, 927 00:46:42,720 --> 00:46:45,760 I think this is going to be the last one I'm going to do. 928 00:46:45,760 --> 00:46:49,600 ..but he's now got a new role at the supermarket where he's been working. 929 00:46:49,600 --> 00:46:51,880 Got a promotion, so I'm now supervisor, 930 00:46:51,880 --> 00:46:53,800 so there's a bit more responsibility, 931 00:46:53,800 --> 00:46:56,760 having to think a bit more, but I'm enjoying it. 932 00:46:56,760 --> 00:47:00,160 This tech is here to stay, 933 00:47:00,160 --> 00:47:03,200 so it's on us to be vigilant to its faults and failings... 934 00:47:04,440 --> 00:47:07,920 ..and make sure that the companies developing and using these products 935 00:47:07,920 --> 00:47:11,760 listen to users and address issues to create a product 936 00:47:11,760 --> 00:47:14,800 that offers a truly level playing field for everyone.