Deepfaking It

Speaker 1: So sometime last year, the coach of the Victory Vipers, a cheerleading squad in Buck County, Pennsylvania, gets texted a bunch of pictures from an anonymous number. According to the criminal complaint filed this month, the images painted a compromising portrait of three of the cheerleading squad's underage members. In the pictures, the three girls were consuming drugs, they were drinking, they were naked, and these images had been sent to the head coach of the high school cheer team. If the intent of this message, a targeted shaming, wasn't plain enough, shortly after a different phone number starts sending similarly incriminating images, this time to the girls themselves. Each of these girls opens up their phone to a picture of themselves that could damage their reputation, their high school career, their future prospects, accompanied with a text message reading, you should kill yourself. And for as messed up a situation as that is to find yourself in, these girls knew something that would set this whole drama on the path that it is on now, not towards their Internet shaming and expulsion from the cheer squad, but towards criminal charges against a woman named Rafaela Spone. The girls knew that these pictures are fake. You can make a deepfake right now. It's pretty easy. You can do a really good one on a computer. You can do a pretty good one on your phone. You can make a deepfake picture where you swap someone's face. You can make a deepfake video of a person who never existed. You can even make deepfakes of a person's voice. And once you start thinking in deepfake, in the editing tools and deep learning and generative neural networks that power all of it, you realize you can kind of construct lies that never would have been possible before. You can look someone in the eye and pretend to be someone you're not. You can pretend someone did something they didn't. Or, according to those criminal charges, you can cyberbully your daughter's cheerleading rivals, which is what Rafael Espone allegedly chose to do with it. She was arrested and charged with six counts of harassment, and her first court date is set for the day this comes out. If you've listened to this show before, you know that elaborate hacks and cybercrimes typically at some point come back to social engineering, the human interaction part of hacking, the part where you trick someone, the part where you lie. And Rafael Espone might be the first mom to allegedly cyberbully her daughter's rivals with it, but she is one in a long history of people using artificially intelligence created synthetic media to do some pretty wild stuff on the Internet. Her story is provocative and a little bit scandalous, but sophisticated actors armed with deepfakes have done some pretty extraordinary stuff with them. So we wanted to know, what will cybersecurity look like in a world where deepfakes are mature and accessible? What's left to verify an identity when we can fake all of the evidence of an identity? And did you notice anything funny about this introduction? Narrated by me, Jordan. Just the real normal human Jordan. Trust me. This is Deepfaking In, here on Hacked. Hi. I'm deep fake Jordan. Okay.

Speaker 2: Am I supposed to respond to it?

Speaker 1: Nice to meet you.

Speaker 2: Oh, hey, bud. How you doing, little guy?

Speaker 1: Would you like to host hacked with me instead? Oh,

Speaker 2: host it with my best neural net bud? Of course.

Speaker 1: Please. Please. Please, Scott. I wanna host hacked.

Speaker 2: Of course. Anything for you, deep fake Jordan.

Speaker 1: Isn't that just the most unsettling thing you've ever heard? So that was ten minutes of training data in a free trial of a piece of software we're gonna talk about a little bit later. But I found it pretty wild how easy I was able to create and even, like, okay version of my own voice that I could type in. And it kinda got me thinking that there's, there's sort of like a trust hierarchy. And I think at the bottom you have text and then you have images and then you have audio visual like texts. I, if I get an email, I don't trust it at all. It can tell me I won a million bucks and I scoff and I throw it away. And a photo, I also don't trust even a little bit because I know about Photoshop. But everything above that, audio visual, I trust in just a totally different way because in my head faking that is still very time consuming and expensive. Intellectually, I know about deepfakes but I don't feel it yet. That term deepfake was coined in 2017. So, for most of your history in cybersecurity this specific technology wasn't really on the table, was it?

Speaker 2: No. Definitely not. This is definitely, you know, more of a contemporary tool, you know, for social engineering. Like, the I think if you were to look back, you know, through the eighties and nineties, you would get people doctoring VHS videotapes for evidence and stuff like that. And, there was people that specialized in, like, you know, inspecting tape to see if tape, you know, literally, like, audio tape and video tape had been, like, modified or or messed with in some way. And now, in the digital age, it's it's, you know, a totally different beast.

Speaker 1: Yeah. I feel like I remember seeing, like, CSI type shows in the mid aughts, and it was really going in and looking for, like, seams in audio.

Speaker 3: Like Totally.

Speaker 1: Artifacts of linear editing, and that seems so antiquated now.

Speaker 2: Well, like, you know, we are essentially professional audio editors at this point. And I could tell you if you pop open Adobe edition, there's not really a lot you can't do.

Speaker 1: Yeah. Adobe comes up in this story a little bit later.

Speaker 2: Oh, I I'm I'm down to talk about a bit of Adobe Boco.

Speaker 1: So that term first used in 2017 by a Reddit user of the same name. They do a post and they were using a desktop open source face swapping technology to create videos that they called deep fakes. And I'm curious if you, with even a basic knowledge of the history of where new technology tends to kind of bloom online, can you guess what kind of video that was? Porn. It was porn.

Speaker 2: Of course, it's porn.

Speaker 1: Yeah. And that's an ethical subject, I think, outside the bounds of our discussion.

Speaker 2: I think we've touched on this in other episodes where it's like if you were to literally just sit back and make a tree of all technical innovations, You know, at the top of that tree, you're gonna find pornography, you know, war, and maybe one or two other things that, like, have driven most technical innovations.

Speaker 1: It's a porn tree.

Speaker 2: It's like porn porn, war, and existentialism. Everything links back to.

Speaker 1: And since that Reddit user coined that term, we've all, like I think we've all just watched about a million videos of, like, x actor's face swapped out for y actor or person X saying this thing they never said before. And it's actually an impersonator with deep fakes layered on top. There's that Tom cruise one that's been blowing up lately.

Speaker 2: And I

Speaker 1: think it's worth going into how deep fakes work before we talk about how they fit into hacking and cybercrime. Just get the jargon out of the way right up front.

Speaker 2: Sure.

Speaker 1: So the two terms that you hear coming up a lot are G a N and V a E V a E stands for variational auto encoder. So let's say you're making like a Matt Damon deep fake. What a VA does is it's trained to encode an image down into this really low dimensional representation and then decode those representations back onto images. So what that means in practice is that you have two of these, these deep learning algorithms running. One trained on a huge diversity of different faces, so that it can encode and decode the random person who's gonna be wearing Matt Damon's face. Mhmm. And another one trained on just the celebrity. And then you just wire them together. You encode Matt Damon's face and you decode it onto the face of the random person. I think is the basic, basic mechanism of how this works.

Speaker 2: This that makes sense to me. You know, you've got one one algorithm that you've essentially trained to identify motions and movements, on the random, and then you just match that up with something that you've trained for the for the something that you've trained for the specific subject that you're trying to fake. Mhmm. You know, you you you connect those dots and allow the the computer to take over. That makes total sense.

Speaker 1: And then on the other side of it, I think in terms of how you make them really good is the GAN. And you read about this one a lot. So making a good deepfake and potentially detecting deepfakes involves using these generative adversarial networks, which I can't find a research paper that's high level enough to explain how they interact with the VAEs, but dumb enough to explain it to me. But the basic idea is that with the GAN two different machine learning models are essentially duking it out. One is trained on forgeries and the other one is attempting to create the forgery. And the forgery creates a fake until the detection, machine learning model can't detect it anymore. So the more good fakes you have training one, the better it's gonna be at fake at creating a deepfake that is undetectable by the best detection technology. Does that make sense?

Speaker 2: Yeah. You for somebody who didn't know how to explain it and couldn't find something high enough level, I think they did a pretty damn good job. You've essentially told an algorithm to keep going until another algorithm is satisfied. And and the algorithm that needs to be satisfied is the one that's, like, specifically trained to detect faking. So Bingo. So the other one just keeps going until it makes the other guy happy. And essentially, it just keeps trying things and variations and tuning, you know, millions of tiny little numbers and things to allow for a curve fit to happen in an n dimensional space. And, you know, until it satisfies the other one, it just keeps going and going and going. So, essentially, you know, it's I don't know what you'd call it. Like, it's self testing itself.

Speaker 1: Totally. Yeah. You're letting it, like, try and run this lie Until it believes itself. Until it believes itself. Yeah. And if you swap out facial recognition for voice recognition, you can run the same basic process for audio. So all that jargon aside, deepfakes really just come down to how much data you can heave into this thing. How many pictures of the person's face? How many samples of their voice? I don't want this episode to just be examples of deep fakes, but I remember they were able to feed, just reams of Joe Rogan's, like, voice into one of these things because he's probably the most recorded human being on the planet right now. And they made a pretty good one. Friends, I've got something new to tell all of you. I've decided to sponsor a hockey team made up entirely of chimps.

Speaker 2: Yeah. I guess when you've got, what, three hours a day of him rambling on his podcast, you could probably and, you know, not to mention all of the episodes of Fear Factor, all of the UFC things. Like, you could probably get, like, a, you know, a good couple thousand hours of Joe Rogan just audio without any issues. You could even probably tune it so that you could do, like, young Joe Rogan and then, like, kinda more middle aged Joe Rogan.

Speaker 1: Sure. You got a knob, a dial, you can move around.

Speaker 2: Yeah. Exactly.

Speaker 1: So as you've always explained, most hacks tend to rely on some kind of social in here like, social engineering. And I'm curious just before we dive into it, what application do you think that deep fakes have in social engineering?

Speaker 2: Well, I think, like, you I think, you know, when we started talking about making this episode, you know, based, you know, substantially on the intro story because it's just so funny. Scary, but funny. Is, you know, bypassing checks is like a big part of getting access to things that you shouldn't have access to. So, you know, proving that you're something you're not or somebody that you're not. And it's like, you know, Photoshop does it for photos. You know, we've got deepfakes now doing it in video, which is, like, insane. You know, Adobe Voco and a ton of voice things that do it for audio. And it's like, we're really running out of ways to verify people, except for, like, seeing them in person and testing their DNA or something like that, you know? Like, we're we're running out of simple things, you know? We've gone to two factor authentication by being like, do you have the same phone number? Question mark. It's like, I guess that's kind of an alternative way to verify who you are. All of these checks and balances that we've put in society, kind of depend on identity verification, and a lot of identity verification can be faked at this point, I guess, would be at the end of a long ramble, what I'm trying to say.

Speaker 1: In 2019, the CEO of a UK based energy company gets a phone call from his boss, an executive with the firm's German parent company, asking him to send funds to their Hungarian supplier. So, the CEO is on the phone with his boss. It's really, really urgent if the funds aren't transferred within the hour the project isn't gonna be completed on time. So, they send the money. After the $243,000 transfer went through, the German boss calls back and this time the number is coming from Austria and they wanted The UK CEO to send another payment. And by the time all of that kind of piled up into one really big red flag, it was too late. The first transfer had already gone through. It was gone bouncing first to a bank in Mexico and off kind of into the financial dark. And turns out that the voice on the other end of that call, the German, you know, Lilt, was a deep fake. The insurance firm that covered this told the Wall Street Journal that this was, I think, the first case of AI being used in a hack like this that they'd ever heard of. And I guess I'm curious with that story in mind, knowing that this tech is out there, how would you verify someone's identity remotely? You get that call. How do you check?

Speaker 2: I call them back, I guess, on a Like, I'm just trying to think, because this is all collateral damage to the fact that we've all been pushed so remote. So, it's like, yeah, come on by the branch. It's like, well, I'm not allowed to. And it's like, okay.

Speaker 1: Right. So

Speaker 2: it's, you know, I think the, you know, I think a traditional way to do that would be to call back on a on a line so that you know that you're getting them, like, if you have their actual but then, we're we're going back to something else that you can intercept and hack. But, like, you know, I think that

Speaker 1: Right.

Speaker 2: That's probably step one is call them back on their personal cell number. And if they don't pick up and, you know, say, yeah, that was me, then, you know, maybe you call the police. But that is a that is a tough one, especially if things are done in crisis, like in temporal crisis, where it's like, this money needs to get out now or else we're gonna have adverse reactions and effects. You know? People skip checks and balances when things need to get done in a hurry. So, yeah, I don't know I don't know if there's a easy way to bypass, or, you know, add a check and balance into somebody faking someone's voice. Like, imagine your boss called you today and said, you know, this is what's happening. You need to do this, this, and this. And you're like, okay. Like, would you question it? Would you pick up the phone and dial them back? Would you, you know? Like, how inherently do you distrust that when you hear someone's voice that it's actually them? And I'd say probably pretty substantially.

Speaker 1: I'm not that distrusting, and I host a podcast about techno crimes. Like, it's it's just not on my radar yet. It's interesting also that you went to, a, like, practical solution, which is just I need to call you back to make sure that I'm talking to who I think I am.

Speaker 2: Yeah. I guess you could do it like a like, you know, how would you verify that I am me right now, Jordan? You'd ask me a question that only I would know the answer to is probably the easiest thing. Totally. And it's like, you know, that's a that's a great form of verification, but it's like you have to predefine that or understand it. Like, it's like, you know, you know, when did we first meet Jordan? And it's like, you should probably only know that. Maybe a few other people do, but it's like, you know,

Speaker 1: it'd be pretty

Speaker 2: tough for someone not to. So

Speaker 1: on the software detection side, using a piece of software to detect this software based deception, there are some tools available. But the metaphor came up. It's sort of a virus antivirus analog. So a guy named Howley, associate professor at the University of Southern California, developed a deep fake detection software using visual markers known as soft biometrics, which are like little visual things that are too subtle for an AI to mimic right now. So say, like, the way Trump purses his lips before answering a question. That's their example, and I don't like thinking about Trump pursing his lips or how Elizabeth Warren, like, raises an eyebrow to emphasize a point. And you can train an algorithm to spot these little specific, person specific movements by studying past footage of them. And the end result was a tool that in 2019 was at least 92% accurate, at spotting deepfakes. But even Lee says that it's not gonna be long until that work is completely useless. They said, quote, at some point, it's not gonna be possible to detect AI fakes. So a different approach is gonna be needed to kinda resolve this.

Speaker 2: I think if you just, you know, think about an algorithm that's built to detect fakes, and then we go back five minutes to talking about adversarial networks and literally having an algorithm that is trained to detect fakes as the, you know, as the check and balance to your faking algorithm. You know, I think, you know, really what you're doing is by creating a better algorithm to detect fakes, you're creating a better algorithm to train the algorithm to make fakes. So it's like, you know, we're in a world now where, you know, the solution could also be part of the problem, I guess. Mhmm. So it's it's a strange strange world.

Speaker 1: And it's kinda where we get into the question of where this deception is pointed. Like, are you lying to one person, the social engineering case study, or are you lying to a bunch of people as in propaganda? So let's take the antivirus metaphor, and let's say that Facebook gets there's a huge public outcry. There's a bunch of congressional hearings about deepfakes and Facebook decides they need to become really, really good at detecting these things. They invest a bunch of money in it and maybe in the future, they roll out some kind of a watermark thing that says, hey, we have been able to confirm that this is a deepfake. I think that, a, companies are gonna go as long as humanly possible without ever wanting to claim that responsibility because of how hard it would be to do reliably.

Speaker 2: Mhmm. I

Speaker 1: think there's, you know, if they're saying this is a deepfake and a deepfake slips through, people are gonna hold them culpable. But that propaganda question is sort of different than using a deepfake to, like, lie to one person. To use it for social engineering as part of a larger hack. You know, it's really great, in an informed public fighting misinformation kind of thing, but it doesn't stop me from using the same tech to impersonate a company's CEO and run a con on someone. You know?

Speaker 2: Totally. Truthfully, I think you'd probably have, you know, more success as an individual hacker running a con on somebody using it for a one off thing rather than trying to convince everybody that, you know, whatever you feel the need to socially manipulate the world for. Like, I agree with your tendency to think that Facebook's not gonna wanna be culpable for being the gatekeeper of what is a deepfake and not, You know, for both sides of that coin too, because it's also gonna be the issue of false positives. It's gonna flag things as being deepfakes that aren't. And that's gonna be as problematic. Like, I don't know if you've tried to advertise on Facebook lately, but you know, everything gets rejected essentially, and you have to appeal everything because they're so worried about having an ad with some form of, you know, controversial content in it. Sure. So it's, yeah. I'm I'm yeah. I I've, you know, Facebook's got themselves in a unique pickle there as being the content platform that all of this terrible stuff lives on. Hi, Pedro. This is and I need your immediate assistance to finalize an urgent business deal.

Speaker 1: That was audio from a corporate phishing attempt similar to the one from the story earlier, and that one didn't work because the mark thought I think rightfully that it sounded a little bit suspicious. Mhmm. But this question of deepfakes for social engineering kind of came up at scale in The US surrounding fraudulent unemployment insurance claims, as you mentioned earlier.

Speaker 2: Mhmm.

Speaker 1: In a report from the Labor Department's Office of Inspector General, they found that from March through October 2020, some 3 and a half billion in fraudulent job benefits, which is only two thirds of all of the phony claims were paid out to individuals with Social Security numbers filed in multiple states. A 100,000,000 went to more than 13,000 and then ineligible people who are currently in prison. And what this is, it it's not a person who doesn't qualify for unemployment applying for it. This is people pretending to be other people applying on their behalf and then taking the money. It's identity theft. And apparently applying for unemployment insurance in the name of like a dead person, a person who's in prison, or just someone in a different state is one of the most commonly used tools in an identity thieves toolbox, which makes a lot of sense. Right? Like you get mailed a check that the victim was never expecting to receive, and they don't find out something's wrong maybe ever.

Speaker 2: Mhmm.

Speaker 1: So they've started using this tool developed by a private company called ID. Me.

Speaker 4: ID. Me is a federally certified identity provider specializing in helping people verify their identity online. These days, there are lots of criminals out there stealing other people's identities and committing fraud. Our job is to keep the bad guys out and

Speaker 1: Krebson Security had a big breakdown of id.me, and it started out as an e commerce tool that is now being used to verify identities for unemployment insurance claims. Essentially, they just ask for a lot more information to verify your identity. An image of a driver's license, utility bills, details about a mobile phone service. And when an application doesn't have one or more of the above or something kind of triggers a red flag, id. Me typically requires a recorded live video chat with the person applying for benefits, which is where masks deep fake or otherwise start to come in. People who've been caught wearing Halloween masks to make them look like the person whose identity they're trying to steal. When you look up this story on like cybersecurity forums, the question that emerges in comments is always how long until that mask is a deep fake mask. According to ID me, a really major driver of these phony jobless claims comes from social engineering, where people have given away personal data in response to like a sweepstakes scam or applying for what they thought was a legitimate job. And when I look at these photos of this person who got caught wearing a rubber mask on this, like, identity verification call, This is just speculation, but I have to think that at this point, at least some of those billions of dollars in successful fraudulent claims were using deep fake technology. I have to think this has already been done.

Speaker 2: I would say guaranteed. You know, we've we've created a game, essentially. You know, like, there's a lot of things in hacking that are essentially games. You know, you're trying to outsmart, outwit, bypass. You're trying to, you know, trying to get somewhere you're not supposed to be. You're trying to bypass a check and balance that is not supposed to be bypassable. You know, you're trying to be more clever than the person who set up the checks and balances. And, I feel like this is that kind of game. You know, there's definitely a distinct reward at the end, right? Like, if you can fraudulently submit, you know, 3 and a half billion dollars in, you know, unemployment insurance claims, you know, that's a substantial amount of money. You know, if you can bypass these checks and balances and figure out a way to to to get approved and get on these lists, you know, the the payments are there, and the money's there, and, you know, they probably don't have enough people to enforce it. So, really, what they've created is a race of the algorithms. You know, can their checks and balances algorithms catch the algorithms that are gonna be faking and trying to bypass them? And, you know, we have a, you know, a bit of a war, an arms race, but it's more of an algorithms race. Mhmm. So

Speaker 1: It's a really big it's a really big carrot. Like For sure. It's just a huge incentive for people to get good at this on both sides.

Speaker 2: 3 and a half billion dollars. Like, that's, you know, tangible to me anyway.

Speaker 1: And that's one grift. Like, that's a fraction of one thing people can do with this. Yeah. To go back a pretty long ways, there are records of, like, ancient Romans permanently deleting a person's identity and history by chiseling their name and, like, their little portrait off of a stone record keeping block. People have been editing photos for as long as photos have existed. I think it was Stalin who, like, famously used image editing to scrub people out of history, which is to say that, like, people have always manipulated media. But these specific tools have only existed since 2017. Mhmm. And they're existing in a media ecosystem that works very, very differently.

Speaker 2: And

Speaker 1: I think that means they kind of have the potential to be, like, an order of magnitude more powerful. When we first started talking about this as our subject, you brought up some of the tech that's sort of coming around the bend about what's next. And I wanna talk about that now. Right after the break. Starting some new isn't just hard. It can be downright terrifying. You put a lot of work into a thing. You're not entirely sure it's gonna work out. You're taking a huge leap of faith. I've started a few things. Now I know I was right for believing in, you know, the idea, the product, despite all of those fears and hesitations. But boy, does it sure help when you have a partner like Shopify on your side. Shopify is the commerce platform behind millions of businesses around the world and 10% of all e commerce in The US. From household names like, well, hacked podcasts merch, to brands just getting started, you can get started with your own design studio with hundreds of ready to use templates. Shopify helps you build a beautiful online store that matches your brand style. Did I mention that that iconic purple shop pay button that's used by millions of businesses around the world? I don't know why I wouldn't. I should. It's why Shopify has the best converting checkout on the planet. It also helps boost conversions, meaning less carts, sort of getting abandoned in the parking lot, and more sales for you. It's time to turn those what ifs into sign up for your $1 per month trial at shopify.com/hacked. Go to shopify.com/hacked. One more time, that's shopify.com/hacked.

Speaker 5: When you finally find your thing, you want the whole world to know about that thing. So you use a thing called Canva to make it an even bigger and better thing. Whether you want to create flyers for that thing, make presentations for that thing, or design merch for that thing, You can do anything. So people can see your thing, feel your thing, love your thing. The next thing you know, it's a thing. Canva, the thing that makes anything a thing.

Speaker 6: We're entering an era in which our enemies can make it look like anyone is saying anything at any point in time, even if they would never say those things. So, for instance, they could have me say things like, I don't know, Killmonger was right.

Speaker 2: So I remember being in our office Mhmm. In, like, November 2016. And, you know, we're obviously, you know, an advertising agency and a creative company, and we use a lot of Adobe products. And Adobe Max, one of their big trade shows was on, and they were demoing and showcasing this, you know, beta product that they were gonna be releasing, and they were so excited about.

Speaker 7: Let's hear from Zayu about Photoshop voiceovers.

Speaker 8: Introducing Project Vocal. Project Vocal allows you to edit speech in text. So, let's bring it up.

Speaker 2: And, they could just do this automatically. And, they were so, you know, ready to release it. Like, it was in draft form. It was in beta. And, they were like, yeah. This is coming. It's gonna be great. And, then, the world was like, hold up. This is gonna break everything. Like, if you can get recorded audio of anyone's voice, and it wasn't even a lot of it, you can just become them. And that's gonna be very challenging for tons of things. So, please, for the love of God, do not release this. And it's I

Speaker 1: remember that.

Speaker 2: Still hasn't been released.

Speaker 1: I remember that same Adobe Creative Cloud event, and there was a section where they had Jordan Peele on stage, and he was just kinda, like, politely nodding along as they rolled that product out. And the guy takes a little snippet of Jordan Peele's voice that he just said it like five minutes earlier.

Speaker 8: So I'll just load this audio piece into, Vocal. So wait a second. And, we have this.

Speaker 1: And you can see the waveform of what he says and the the text of it is right underneath.

Speaker 8: So as you can see, we have the audio waveform above it, and we have the text under it.

Speaker 1: And Jordan Peele's kinda nodding and say, oh, that's cool. And then the guy just changes the text.

Speaker 8: We can actually type something that's not here. So Hi.

Speaker 1: Of what Jordan Peele said, and it buffers, and we see the waveform above changing, and then he hits play. And, I kissed Jordan and my dogs. And this is Jordan Peele responding.

Speaker 6: You you a witch. Yeah. You you a demon.

Speaker 1: They explained it as Photoshop for voice overs. They said quote, we've already revolutionized photo. It's time to do the same for voice. And Peele, I would say really accurately and quickly summarizes every single comment and think piece that came out after this. Your take, my take. He said, quote

Speaker 8: This if this technology gets into the wrong hands

Speaker 1: And then they kinda cut them off and they clamor to clarify that, oh, they're gonna be able to detect when a voice has been faked.

Speaker 8: Don't worry. Don't worry. We actually have researched how to, like, prevent forgery. We have, like, think about, like, a watermarking detection.

Speaker 1: You don't gotta worry about it. They're gonna watermark it. It's gonna be fine. And kinda like you said that the tenor of that presentation was, oh, this is a product announcement. Like, we're we're building this and you're gonna be able to use it. But that was in 2016, and five years later, that product is still sitting on the shelf.

Speaker 2: They were pumped about it. They were ready to launch that thing, and they thought the world needed it. And the world, like, I'm not gonna lie, it would be very

Speaker 1: useful. 100%.

Speaker 2: It's just opening Pandora's box. Like, we've never had the ability at least in 2016, there was no ability to really deep fake voice that easily. And this was just like a all in one prepackaged tool that, like, came for $19.99 a month or whatever Creative Cloud cost. And you're just like, yeah. Here. You wanna fake someone's voice? You wanna call somebody and pretend to be Jordan? Do we wanna record a podcast, but actually just type the whole thing in and just, like, have it

Speaker 1: So Adobe Voco, this unreleased thing that they have since said was a research demo, which I respectfully kinda call bullshit on.

Speaker 2: I was there, man. It was not a research demo.

Speaker 1: No. No. It's not a research demo if it's the, like, fifth product in a sequence of products, the rest of which you are rolling out as things people can use. Like

Speaker 2: Yeah.

Speaker 1: And you know what? Good for them. Like, good for them for opening this thing a little bit, letting everyone peek inside, immediately getting so much feedback, being like, you are about to unleash a demon. And then just being like, we're just gonna close this and we're gonna put it back up on the shelf. That's a good thing. I like when big companies do that.

Speaker 2: When you think about a world based on liabilities and based on lawsuits

Speaker 1: Mhmm.

Speaker 2: And how many like, like, Photoshop has already ruined so many people's lives. Imagine what, you know, doing the same thing to voice would do. Like, it's just you're just keeping that going. You know? I just can't I don't know. Know. I'm glad they shelved it even though it just essentially told the world that it was really easy and totally possible to do, so there's alternatives to it now, but still.

Speaker 1: Yeah. So on that note, Adobe Voco works a lot like it seems like it's it's better than, but it works a lot like the software product called The Script that I used to create the deep fake voice at the top of the show. I just fed it ten minutes of data, half of what they used to create, the deep fakes at the Voco presentation, and, like, a third of what they actually recommend. But it still kicked out a voice puppet that I could use to type in my own voice.

Speaker 2: Think, Jordan. Like, how many podcast episodes do we have? How many hours of footage is just freely available of your voice online?

Speaker 1: I'm a thoroughly recorded human being.

Speaker 2: Like, if if we wanted to just train a voice puppet of you and just, you know, be rid of you, That would be

Speaker 1: possible. You could you could do this. You could host this show with deep fake Jordan like it wants.

Speaker 2: Like deep fake Jordan wants.

Speaker 1: Research from Imperial College in London and Samsung's AI Research Center in UK were able to create, a deep fake using a single image, a technology that was recently commercialized with an app that lets you sort of animate old photos of relatives. Mhmm. And the morning that we recorded this, someone sent me a tech demo that Nvidia just rolled out of a video call platform that eliminates the need to send high bandwidth video data at all. All it does is it grabs a single frame from the start of the call and then motion point cloud data in real time, and it creates a real time deep fake using that one frame and the motion data. So your bandwidth is just that motion data in a single frame.

Speaker 2: You gotta be kidding.

Speaker 1: Which is no. It's very, very doable.

Speaker 2: It's brilliant.

Speaker 1: Which it's it's absolutely brilliant. But it also means that I see no way that we can't almost right now create very very usable deep fake puppets of real people without their participation. Not just celebrities who have tons of recordings and images of themselves out there in the world, but just anyone you can find a photo and a sample of their voice of. You can turn on your mic, turn on your webcam, and you can pilot like a deep fake avatar of another human being.

Speaker 2: Well, you you this is also just gonna open up the next Pandora's box of being like, yeah, I don't even have to get ready for a video call anymore. I just literally load in the photo of how I wanna look for this call and, like, train it to grab the point data on my face and then I'm just done. Like, I don't even need to like, shower and put on a clothes to have this call because this is gonna be all deep faked anyway.

Speaker 1: Sure. You'll know when someone's fancy because their deep fake, like, video presence is, like, a nice studio portrait that they took of themselves. Exactly. Like really good and, like, well lit.

Speaker 2: Yeah. Perfect lighting, perfect everything.

Speaker 1: And it's it's tough because I don't I don't think the answer is ever to, like, get rid of this tech because the as we can see even right there, there's such cool applications for it. Like, letting people with no Internet connectivity have video calls is a great use of this technology. But, holy crap, those are some implications.

Speaker 2: Seriously. Well, the other thing is too, is that you're essentially transmitting, like, you know, we talk about recorded audio where we've got, you know, hours of Jordan Blumen speaking, you know, saved into WAV files all over the Internet. At some point, if I had to start recording my voice calls, I would literally just get facial point data for everybody that I talk to you on on video chats, and then I could literally just manufacture them if I needed to. Like, you know, once you start transmitting all that information around, not that it's not, you know, capable of getting it from post processing video footage and stuff, but still. Sure. Like, man.

Speaker 1: You brought this up earlier, the kind of moment we're living in. But at a time when the most socially responsible thing you can do is avoid seeing and talking to another person face to face, the ability to just transform into another person digitally is almost as good as doing it IRL. It's, like, just this side of, like, Mystique and X Men.

Speaker 2: Totally. Like, imagine I could just be you on Zoom. Like, I just had to turn on the, like, Jordan filter. I sounded like you. I looked like you. I was literally I have a photo of your apartment. You know? Any way that's looking for for environmental clues would miss them. I could pretty much do anything. I could call your parents. I could call your, you know, your partner. I could call, you know, your friends. I could just do what I wanna do. And who's gonna be able to tell that it's not you? You know?

Speaker 1: The only way that voice, deepfakes work is if the deepfake creator can do a real time transcription of the person's voice, it needs to be able to render text out of the audio. Right. And if you can do that, you can then feed that same text into the you can decode that into someone else's voice, which means all of the individual parts of I talk into a microphone, it transcribes it, and then it replays that in someone else's voice already exist. It just hasn't been all wired together in that way yet. But the tech is

Speaker 2: there. Going through the the different things that we use to verify ourselves and, you know, essentially checking the box next to that they're easily able to be faked now. You know? Back in the day, it was like, you know, photos. Photos couldn't be doctored. Oh, they figured out how to do that. And then, it's like, okay, that was even pre digital. And then, you went post digital or into the digital era, and it's like, okay, then we got Photoshop, and it's really easy to doctor photos. It's like, okay. Okay. You know? But we can't fake video. It's like, well, actually, we can fake video. And it's like, okay. Okay. But it's still not that good. And you're like, yeah. Yeah. It's not that good. Okay. We got deepfakes. Now it's impossible to tell when it's, like, faked. It's like, okay. Next. It's like the box, we just keep going down the list, and it's like, you know, we got fingerprints. Yeah. You can't fake a fingerprint. Yeah. You can't fake a fingerprint. K. We scan a fingerprint in. We turn it into, you know, byte code. Okay. Byte code is just essentially text or some way of verifying what the fingerprint is. Okay. We can fake that. Okay. That's fakable. What's next? And it's like you just keep going. And it's like, k. There's really nothing left. You know, what is what is left besides, like, sitting down across the table from you and being like, Jordan, on the day we met, we were talking about this. What did you say about that?

Speaker 1: Mhmm.

Speaker 2: It's like, that is the correct response. You know? As long as nobody heard that, we now have verification. And it's like, you know, there's there's very little left.

Speaker 1: Is our ability to do wilder and wilder stuff. I'm always fascinated by the things that we choose to do with it. Mhmm.

Speaker 2: And I

Speaker 1: think that that really nicely brings us back to sort of our opening story. And I think it brings up a different similar story from 1991. And that is the story of a woman named Wanda Holloway. And Wanda Holloway was a Texas mother who approached her former brother-in-law looking for help to hire a hitman. Her target was the mother and daughter who Holloway believed were standing in the way of her daughter being elected cheerleader. And when Holloway decided that she could not afford two whole murders, she just sort of settled for knocking off the mother. Reasoning that that death would traumatize the girl so much that she would not be able to compete, in cheerleading to the best of her capacity. Which is to say that people's reason for doing really wild shit never really changes. It was a competitive cheer mom then, and it is a competitive cheer mom now. But the technology that empowers people definitely changes. As we said in this episode, like, I don't think that we should try and do away with deep fakes or say that they're not allowed in some way. I don't think that's how tech works. I don't think that's how tools work. And seeing all the cool stuff that can be done with it, I I have no desire to see that stifled. But I do think it's worth looking at those behaviors. What have people done before that they're gonna do with this tech in new, more powerful ways? And how can we respond to that preemptively? Catfishing, identity fraud, you know, people have always bullied each other and stolen from each other. We should be thinking about how this tech is gonna empower that, so we can respond to it now before it gets too bad. Agreed. Hi. I'm Deepfake Jordan. Thanks for listening, everyone. If you would like to support the show, you can find us on Patreon at patreon.com/hsckedpodcast. Huge shout out to our new patrons, Blake. Balje and Kyle, your support means the world to me, Deepfake Jordan. Thanks for listening.

Speaker 9: If you've got an insurance question, you could talk to the butcher at your local grocery store. He'd probably talk about trimming the fat, but it'd be about your brisket, not your insurance policies. Or you could talk to your local GEICO agent. They offer personalized assistance in finding the choicest cuts of coverage for all your insurance needs, which means more money for filet mignon. Or if you're a vegetarian, tofu lei mignon. To find a GEICO agent near you, visit geico.com/local.

Speaker 7: This episode is brought to you by Nespresso. Life moves quickly, and taking care of yourself shouldn't feel like another chore. With the new Nespresso Vertuo Up machine, morning routines become rituals. Whether organizing, getting the household moving, or preparing for the day, your coffee shouldn't ask for more. With Vertuo Up, just press brew and your morning begins. Rich aroma, old flavor, zero effort. Press to explore. Every coffee, a new world. New Vertuo Up. Shop now at nespresso.com.

Speaker 3: The right window treatments change everything. Your sleep, your privacy, the way every room looks and feels. At blinds.com, we've spent thirty years making it surprisingly simple to get exactly what your home needs. We've covered over 25,000,000 windows and have 50,005 star reviews to prove we deliver. Whether you DIY it or want a pro to handle everything from measure to install, we have you covered. Real design professionals, free samples, zero pressure. Right now, get up to 45% off-site wide plus get a free professional measure at blinds.com. Rules and restrictions apply.