North Korean IT Scam + TikTok Zero Day + Consumer AI Gets Weird

Speaker 1: The name Riley Park is listed in the incorporation documents of a bunch of companies. 1,436 companies registered under that name. We've got a lot to talk about this episode, but I wanna start here. Riley Park is not a serial entrepreneur with over 1,400 businesses to her name. No one, like, no one hustles that hard.

Speaker 2: Well, you know, when I'm on my quote, unquote grind set, I, hustle pretty hard. But 1,400, that's that's a lot.

Speaker 1: It's pretty hard. That's a that that is the level of grind set that I I don't know anyone that rises to. That is because Riley Park is one of several fictional persona created and used by Registered Agents Inc, a company that allows their customers to register businesses in The US anonymously to help obscure their true ownership. Riley Park is one such fake identity that they do this with. There are a lot of reasons a person might register their business under a fake persona, but we are concerned with one group of actors using this service. Not because they're the only people registering LLCs using an anonymizing service for potentially dodgy reasons, but because of how odd the end scheme is. It is not legal in The US to do business with North Korea. Has been pretty much everywhere since 2003 when they bailed on a nuclear non proliferation treaty and a bunch of sanctions rained down. It's almost certainly illegal wherever you live, and it's definitely illegal in The United States where Registered Agents Inc operates to buy stuff from, sell stuff to, and hire people in North Korea. We've spoken at length on this show about the role that this has played on the birth of the North Korean ransomware industry. Ransomware, it's a great way to make a buck behind a laptop. And if you're already under international sanctions, what do you have to lose? But you don't need an LLC to do ransomware. You do need them to do large scale freelance IT.

Speaker 2: Are you waiting for a shock from me? Because I'm very shocked if you're about to tell me that North Korea is a large scale IT service provider.

Speaker 1: It turns out that for years, North Korean agents have been using shell companies to orchestrate a large scale IT scam in The US. They set up these companies with the help of companies like Registered Agents Inc using fake identities like Riley Park to obscure their operations. These companies then hired Americans dubbed virtual assistants by the FBI to set up laptops with remote access software, allowing North Korean workers to remote access in and perform work that appears to be originating from The States. The scheme then funnels money back to North Korea. FEI agent Jay Greenberg pointed out that based on the extensive, and mass scale of this operation, suggesting that many companies hiring freelance IT workers, quote, more than likely employed someone linked to this scheme.

Speaker 2: Are Are you telling me that they actually did real IT work?

Speaker 1: Yeah. The scam wasn't that they pretended to do. It was real IT work done via remote access set up by Americans being hired by these companies registered through Registered Agents Inc.

Speaker 2: Sure. So you were high you were, like, getting an IT service provider that was actually just a shell company

Speaker 1: Yes.

Speaker 2: With essentially terminals that were digital connective pipelines to North Korea, and then North Koreans weren't hacking you and stealing your information. They were actually just No. Being your IT department.

Speaker 1: Just doing IT work. Just doing IT work. Correct.

Speaker 2: Okay. Right? Weird.

Speaker 1: And that somehow is just the tip of the iceberg this episode. We have so much to talk about.

Speaker 2: Well, I'm looking forward to it.

Speaker 1: We've got a TikTok zero day. We've got Microsoft recall snapshotting your desktop every five seconds. Some folks in The UK set up a home brew cell tower to do cell phone spam.

Speaker 2: Love that.

Speaker 1: I I love that for them. We have so many stories, so little time. Let's get into it.

Speaker 2: Welcome back, Jordan.

Speaker 1: Welcome back, Scott. How are we doing? We're

Speaker 2: okay. We're okay.

Speaker 1: We're thriving.

Speaker 2: We're stretching the back out. We're stretching the back out. We're we're okay. We're okay. Summer might be here.

Speaker 1: Predict some doing some desk yoga. I feel you.

Speaker 2: Doing some desk I've got this had, like, this muscle problem about my too much information, but I have this, like, pain in my back. I work on it, like, daily.

Speaker 1: Do we wanna do a

Speaker 2: little, like, store.hackpodcast.com? Check it out if you need some merch. Added a crew neck and a few other little pieces. Gonna maybe look to add some more stuff.

Speaker 1: Yeah. Buy a hat. Why don't you? Yeah. Let's keep it focused. Go buy buy a hat if if you like weird tech tales and helping out a podcast. Store.hackedpodcast.com.

Speaker 2: And, if you wanna really show some support for the show, you can always be one of our patrons. We love them all. Hackpodcast.com redirects to our Patreon.

Speaker 1: Means a lot to us. Great way to support the show. I I Yeah. I think that's pretty pretty much it in terms of things that we we're here to provide content to the people. That's that's everything we need from y'all.

Speaker 2: That's true.

Speaker 1: But but, boy, do we have a a lot of a lot of stuff to get to this episode? It's been a pretty wacky week in the world of week couple weeks in the world of, like, tech and privacy and security and AI and all the bullshit we like to talk about. Yeah. I'm trying to think of where we should dive in.

Speaker 2: Well, I was just gonna say I'm just waiting for my North Korean IT worker to finish up on my computer here before we can really get into it. But

Speaker 1: So why don't why don't we why don't we finish finish that thought? A few different places have covered this story, but the lion's share of, really, a lot of the best reporting has been a investigative deep dive by wire journalists William Turton and Dhruv Mehrotra. Really impeccable work. And the gist here is that, as we said in the introduction, the FBI has uncovered this very sophisticated scheme orchestrated by North Korean agents to use these US registered companies to funnel money back into the country. A lot of the articulation of this has been rooted in the fact that the country does have a WMD program. So there's this sort of, like, using IT scams to fund WMDs. But it is sort of, you know, as we've seen time and time again, another revenue source for for the country. And it really works on exploiting this loophole in how US corporate registration processes works. And at this heart of it is this American company, Registered Agents Inc. I'm still

Speaker 2: I'm still in shock that they went from massive crypto heists to legitimate IT service out provider. Like, that to me is just a weird story arc.

Speaker 1: Yeah. Right? Like, to go from it was North Korea has a bit of a track record with, like, they do ransomware. They do all sorts of cybercrime and hacking. They have other types of illicit products that they manufacture and outsource, allegedly. And over here, they're basically running Fiverr. Like, it's just a very quaint, strange little operation, but at a at a genuinely massive scale. Registered Agents Inc, this American company, is known for its very extreme privacy practices. They fill this role in the ecosystem, allowing people to register these businesses register business in The States, you know, very, very anonymously. They're based in Post Falls, Idaho, founded by a guy named Dan Keen, and they use this series of made up identities, Riley Park, David Roberts, Morgan Noble, that have, I think there's 36,000 businesses listing officers with names identified as fictitious by this wired investigation and former employees. It's a it's a pretty big operation and at the heart of this very strange story.

Speaker 2: When you started reading it out, I started to get, like, tax evasion vibes.

Speaker 1: Sure. You think it's going that way? Yeah.

Speaker 2: I think it's going that way because it's like, oh, yeah. Okay. It's like, yeah. We got a numbered company with some local director in some in some tax haven. And and all of a sudden, you're like, no. No. And then they literally get people on Fiverr to set up, like, HP laptops that they bought at Best Buy that are literally just, like, terminals where they remote into those and then remote into client computers and do the work. And it looks like they're working from, like, a warehouse bay in Mississauga or Massachusetts, but they're actually in North Korea. I was like, I was not expecting that twist. That one got me.

Speaker 1: Yeah. It it kinda comes out of nowhere. It doesn't seem like it's barreling towards a story about North Korean IT fraud. And even saying IT fraud is misleading because it suggests not what they were doing. They were providing legitimate IT services, circumventing, economic sanctions. Wyoming has played a big role in this. The the FBI kinda comes out this report. People start covering it. The Wyoming secretary of state Chuck Gray took action in May by revoking the business licenses of three implicated companies registered through Registered Agents Inc, CultureBox LLC, NextNets LLC, and Blackish Tech LLC. Gray's office supported the FBI's findings, citing the inappropriateness of North Korean operations, inside of that state. We're just starting to get a sense of the scope of this, but it has become quite clear that this has been working for a while. And there's this whole big body of people in The States who have been hired as these, you know, digital assistants, whose job is just to open up a laptop while connected to an American IP address, open some remote access software and let people cook. And it's I'm very fascinated by you know? I would love to speak to someone who was on the other side on that side of this scam.

Speaker 2: I just can't help but seeing the irony in in the the communist, you know, nation of North Korea executing a large scale identity fraud to execute large scale capitalist ventures in the IT services space. Just this this there's some irony in there that I I don't know how to articulate, but I I enjoy it a bit.

Speaker 1: I think you just did. I think, I mean, what's more laissez faire free market do what ye will than ransomware?

Speaker 2: Yeah. Well, by the end, the fact, like, I can I can get the, like, meddling North Korea that's, like, identity thefting and ransom wearing and stealing crypto like, you know, that that to me, that seems like the evil empire, you know, that that suits this world?

Speaker 1: Different though.

Speaker 2: Yeah. This one's like, hey, let me order you some new printer ink and make sure that you're, like, network switch is working right. It's like it just to me, I don't know. It it's gone from evil empire to, like, hey. We're your online IT buddy.

Speaker 1: Yeah. I'm not trying to, like, humanize a a clearly, not cool government administration at all. But it is it does feel a little bit like a a weird half measure at recovery from an untrustworthy actor. It's just like, wow. We have all of this tech capacity, and, like, people have learned how to do all this remarkable stuff. You know?

Speaker 2: How do we monetize it?

Speaker 1: How do how do we monetize it in a different way? It's like, I don't I deploying ransomware isn't the highest tech thing in the world, but you need to know some stuff. Mhmm. And at a certain point to go, like, we we have a lot of internal capacity. How do we monetize this? What we're gonna do is and then all of this follows. It's it's a strange unexpected story. And,

Speaker 2: destruction with it because, obviously, North Korea is building weapons of mass destruction. And if they're making revenue from selling IT services, obviously, that revenue is going to making weapons of mass destruction. The causal linkage of today's media.

Speaker 1: Yes. Again, not defending. Oh, man. You you always know you're walking along a a third rail when you have to say that twice in an episode. Again, not defending anything to do with North Korea, but an equally accurate statement could be, IT scammed to fund roads. Like, it it

Speaker 2: Our food.

Speaker 1: They're extrapolating food. Like, you could say it about anything.

Speaker 2: Exactly.

Speaker 1: Yeah. Not that I am or maybe I am. Anyway, let's talk about TikTok.

Speaker 2: Yeah. Let's go from controversial North Korean IT service provider to controversial Chinese social media network.

Speaker 1: Yeah. Why did I pick that one out of

Speaker 3: the list of stories? I have a

Speaker 1: fun home brew cell tower story. And I'm like, now I need to pivot to somewhere safer. Let's talk about TikTok in 2024.

Speaker 2: Okay. Let's talk about TikTok. Let's go.

Speaker 1: Yeah. There there's a there is a no click zero day exploit that was going around in TikTok DMs that compromised the accounts of CNN, Paris Hilton, and Sony, a very, very random assortment of targets. The story was, I believe, broken by c e by CNN, which makes a heap of sense. Malware was transmitted via TikTok DMs. Users only needed to open the message for the act to initiate. No further action required. Don't have to click on a link. Don't have to download a file. Looks like TikTok was on it relatively quick. Compromised accounts hadn't been shown to do any unusual posting. But, it's just a sort of another interesting step in the story of TikTok breaches. 700,000 accounts in Turkey were breached in 2023 due to an insecure SMS protocol for two factor authentication. There's this other big breach in 2022, with this vulnerability discovered by Microsoft researchers That involved at least a a a link that someone had to click on, so this is somewhat novel. And it is coming at a time when TikTok security practices have been this really big point of concern for lawmakers, especially in The States and especially regarding potential surveillance by the Chinese government through ByteDance. Promise these two stories in a row wasn't intentional. This has led to a law in April covering, requiring ByteDance to divest from TikTok or face a ban in The US. So an interesting breach at an interesting time in the history of an interesting company.

Speaker 2: It's I feel like if you're if you are these hackers, it's like the trophy is to get something that doesn't require clicking responses, any action from the user. Totally. That to me just seems like the the holy grail of the hack. It's like, okay. I just have to send something to somebody. And if they if they open up their inbox, bang, it's over for them. So it's like, I guess, not I'm not here to say kudos to people doing malicious things, but I think you've achieved probably what you were hoping to achieve.

Speaker 1: I like the idea that someone tuning into this for the first time is, like, I think these guys root for the bad guys. It's like, I promise I I we try and strike a very nuanced ethical lens on all these stories. It's not

Speaker 2: They're they're North Korean apologists.

Speaker 1: They're North Korean apologists. It's very weird. Their last episode was a hotline episode. I liked that, but I'm not so sure about this one. Yeah. It it's a fascinating story. I'm I'm without TikTok in my life at time of recording. So

Speaker 2: Have you ever been a TikToker?

Speaker 1: I mean, I've never been a TikToker. I've had TikTok on my Oh. I've watched TikToker, but I have never, danced my heart out.

Speaker 2: I have I have held fast, and I have never

Speaker 1: Have you really?

Speaker 2: I've never had an account. I think I've created an account to hold a username, but I've never installed it. I've I've been shared TikToks. And at the end of the TikTok, it's always like, install TikTok, and I'm like, no. So I've never I've avoided it in its entirety as I know that it's unhealthy for most people's mental health, and I don't even wanna play in that pool.

Speaker 1: Yeah. I would say I heard someone say this once, which is this is an aside, but that, like, algorithmic vertical video specifically is the closest thing to, like, a digital narcotic we've ever invented. And Mhmm. I know my personality well enough that if you put that on a device in my pocket, it sure is a really good digital narcotic. Like, it you can just funnel time into it. It's not it's not good for me.

Speaker 2: I I watch people in my life lose like, we only have so many hours to exist as a human. You can choose what to what to do with those hours. And I watch people just willingly contribute a large chunk of their life to absolutely no value. Like, I am guilty of, like, being a researchaholic. Like, I love learning things, and I consider knowledge to be, like, you know, of the utmost importance. So, like, I can spend too much time reading, educating, you know, spend time in my digital devices doing things like that. And then I just watch people watching fifteen second loop videos where they're getting nothing from it for, like, three straight hours. And I'm like, how what how do I help you? It's what it makes me feel. Like, I'm like, how do I help you?

Speaker 1: Yeah. I've said it before in this show, and I'll say it again. I have a deep and abiding disrespect for my own time, and even I can't put I I can't feel good putting time into those things. I would much rather read a bunch of weird obscure tech nonsense and then talk about it on the Internet with you. That is a far better use of my time, for that impulse. Whatever that that impulse is, it's like find productive ways to funnel them. There you go.

Speaker 2: Jordan likes to read things and spend time

Speaker 1: with you. Exactly.

Speaker 3: That's that's what we're here to do.

Speaker 1: You are my TikTok alternative. And so is apparently x Twitter, whatever we're calling that. And that maybe transitions us nicely to, another story that I'm I'm quite excited to talk about. It's a short one. Friend of the show, Bruno, shared it with us over on over on Twitter. And it's a story about a fake cell cell tower in The US and a smishing campaign. I just like the word smishing, and that's a big part of why I

Speaker 2: include it. Yeah.

Speaker 1: Two individuals were arrested by British police in connection with a Smishing campaign involving a home brew mobile mast that allowed them to send fraudulent texts. The the text side of this isn't that interesting. What they were sending out were pretty boilerplate. We're pretending to be your bank two factor authentication type stuff. But what's interesting about it is this this fake tower that they employed. Huayong Zhu, 32, from in The UK was charged with possession of articles for use and fraud and is awaiting his court appearance. Another suspect was arrested, not identified. They sent out thousands of these text messages through this illegal mask, posing as, like I said, banks, other orgs, trying to kinda trick people and get sensitive information. The thing that they used, so we've talked about IMSI before, international mobile subscriber identity. There's a thing called an IMSI catcher. It's used for grabbing those, and there's it's looking like what they did is a piece of text similar to that but sort of flipped. Typically, these are used by, like, law enforcement, but it's looking like what they did is they they used it to broadcast a bunch of SMSes, kinda just skirt around some security measures. So a a novel application of a of a piece of technology for a not that interesting scam, but just interesting when placed in context.

Speaker 2: So were they intercepting, two factor authentication messages, or were they kind of broadcasting out messages to people making them do things? Like, what was it a smishing, or was it a was it like a pack like a man in the middle?

Speaker 1: I think it was a smishing attack. I don't think that they were trying. Yeah. I I don't think they were trying to intercept the two factor authentication. I think they were trying to get people to go log in to a platform linked to from a text message that is not the real version. So

Speaker 2: Right. A classic phishing, you know, vector.

Speaker 1: Classic one. And just I I like the visual of some people setting up a fake cell tower and sitting there trying to do this. This isn't a super interesting scam, and I don't think a great use of anyone's time. But I find it fascinating that they cracked up this they they figured out this way to send these things out using this ISMI catcher that we've talked about before.

Speaker 2: Maybe we should buy a a cell repeater. I know you can get them.

Speaker 1: They're not hard to come out. Crimes.

Speaker 2: Two crimes. North Korean apologists and, hacker celebrators, Scott and Jordan from Hack Podcast, were arrested today for doing crimes.

Speaker 1: And talking about the Internet. Yeah. I mean, genuine, clarification, all this. Okay. Where do we start? Don't do submission crimes. Don't hack TikTok, and don't fund WMDs with IT scams. There you go.

Speaker 4: This message is brought

Speaker 2: to you by Scott Scott and Jordan at Hackbackup.

Speaker 1: And on that note, I think we should kick it over.

Speaker 2: Get back on track here. I feel like we're we're on the third rail today.

Speaker 1: We're on the third rail today. I think we should kick it over very briefly

Speaker 2: To the

Speaker 1: To a little place I like to call the advertising oasis. It's calm. It's chill. We read ads.

Speaker 2: It helps pay the bills.

Speaker 1: We pay the bills. And then we get back to talking about fun tech and security stories. Do you wanna kick it out? I'm getting calmer just as I know I'm about to step off the plane in the advertising oasis. Are you ready, Scott?

Speaker 2: Welcome to the oasis. You can feel the sands beneath your feet as you curl your toes into the sand.

Speaker 1: Like the dunes of Arrakis.

Speaker 2: Dunes Of Arrakis. Oh, man.

Speaker 1: Let's get into it. Starting some new isn't just hard. It can be downright terrifying. You put a lot of work into a thing. You're not entirely sure it's going to work out. You're taking a huge leap of faith. I've started a few things. Now I know I was right for believing in, you know, the idea, the product, despite all of those fears and hesitations. But boy, does it sure help when you have a partner like Shopify on your side. Shopify is the commerce platform behind millions of businesses around the world and 10% of all e commerce in The US. From household names like, well, hacked podcasts merch, to brands just getting started. You can get started with your own design studio with hundreds of ready to use templates. Shopify helps you build a beautiful online store that matches your brand style. Did I mention that that iconic purple shop pay button that's used by millions of businesses around the world? I don't know why I wouldn't. I should. It's why Shopify has the best converting checkout on the planet. It also helps boost conversions, meaning less carts, sort of getting abandoned in the parking lot, and more sales for you. It's time to turn those what ifs into sign up for your $1 per month trial at shopify.com/hacked. Go to shopify.com/hacked. One more time, that's shopify.com/hacked.

Speaker 4: This Father's Day, do more with dad and spend less with low prices guaranteed at the Home Depot. Get him fired up with a new grill and accessories, like the next grill five burner for just $299 so you can spend more time together while he becomes the grill master he was always meant to be. Or build memories with savings on top brand power tools so you can tackle projects side by side. Gift more and do more together this Father's Day with help from The Home Depot. Exclusions apply. See homedepot.com/pricematch for details.

Speaker 3: Study and play. Come together

Speaker 2: on a

Speaker 3: Windows 11 PC. And for a limited time, college students get The best of both worlds. Get the Unreal College deal, everything you need to study and play with select Windows 11 PCs. Eligible students get a year of Microsoft three sixty five premium and a year of Xbox Game Pass Ultimate with a custom color Xbox wireless controller. Learn more at windows.com/studentoffer. While supplies last, ends June 30, terms at aka.ms/collegepc.

Speaker 1: Babbidy babbidy babb babb boom.

Speaker 2: I have seen this story about Microsoft's recall functionality all over my timeline, and I and all over my news feeds, and I haven't read a single one of the posts. I don't know why. Maybe it's because I I went from the generation that, like, saw Microsoft as evil and then saw them as, like, you know, saw their, like, their story arc because they came back in as, like, a hero and, like, Microsoft Azure and, like, all these things. And I don't wanna think poorly about them again. So I have not read I haven't haven't looked at any of these articles.

Speaker 1: I don't think this is another step in well, maybe every step for Microsoft is a step in the ongoing saga of are they evil? But I don't know that that's what this is. This feels to me more like a slight privacy misstep or a privacy communications misstep more than anything. I think this is this is a a really interesting one, and potentially a big security vulnerability. We'll talk about it. So Microsoft introduces this this recall feature for Windows that's gonna be coming up. It's part of this big announcement to do with how AI is gonna be integrated. We saw Windows and Apple do similar announcements over the last couple weeks. And what it's basically doing is it's trying to create this searchable database of the user's activity on their computer over time. It's trying to realize that dream of being able to just plain language ask your computer a question. Hey, what was that site I was looking at with the cheap tickets to the thing? Really open kind of large language model esque question, and it can go back and it has that context. Personal context, it's a big key phrase we're hearing a lot in these announcements. And the Microsoft approach to this is the system called recall that functions by taking a frequent screenshots, storing them in an encrypted folder, and that it can then open up and parse through them and try and, you know, figure out what it is you're talking about. It knows what you were doing because it's taking screenshots is the basic idea. The data is supposed to be processed locally on the user's device and involves components of Azure AI. It is supposed to be be encrypted, but that encryption is only effective when the computer is, it's looking like off or the user is locked out. When the PC is active, encryption does not prevent access to these images, making it kind of ineffective in terms of attacks by, you know, maybe an actor with a little bit of malware sneaking onto your system. And what this has done is opened up this huge floodgate of criticisms of, okay, is this what this next era of AI computing looks like? Is just we're gonna take we're gonna gather way more data about you that we can go through and be convenient and helpful about. But along the way, we're gonna gather a ton of data about you. First step, screenshots every, you know, five seconds or whatever it is.

Speaker 2: I got a couple here here's the thing that really gets me is it's like, you have full system control and screenshots and image processing is the best. It it must be one of the facets or one of the pieces of data they're looking at because that to me just doesn't strike me as the best. Like, they could read all of the content on the website. Like, they have full system access. Anything that's rendering, it it knows what it's it's looking at. Like, why do you need screenshots? Like, I get it, but at the same time, I don't. The next thing, which is totally off base and and fits today's episode theme of us being kinda out of it, is are there links to software like, you know, remember when I'm gonna go on a digression here. Like, Steve Jobs introduced the iPad and was like, the iPad's amazing. We don't want kids to use it. We've done some research, and it's probably not good for kids. Do we when when is that moment for, like, we've created an AI so that you don't have to remember anything, and we haven't looked at the impacts to dementia and Alzheimer's? Because it's like flexing your mental muscles is like an important thing, and it and it keeps your mental strength up. Like like the if if you go deaf and leave it untreated, early onset dementia and Alzheimer's comes for you because you essentially check out of social engagements that you're a part of, which reduces your mental functionality and and increases your probability of dementia and Alzheimer's. If I can just be like, I was looking at something on my phone this morning. What was it? And it's like, oh, you were booking flights to Switzerland. And I'm like, oh, yeah. Right. I still have to book flights to Switzerland.

Speaker 1: Do it for me.

Speaker 2: Are the yeah. Are there gonna be health outcomes that people are not talking about yet for us just replacing mental exercise with asking our robot companions what we had to do?

Speaker 1: Well, this this was a fun story about a privacy PR whoopsie, and now I feel sad. No. I I I completely agree. Like, the amount of human activity that has been mediated by well, you do that on a computer. Mhmm. Oh, that? Oh, you do that on a computer. Oh, that? You do it on a computer. And this big shift towards what if we use the computer for you? I think that phrase and that concept of the computer uses itself for you, it's so compelling. It feels inevitable. But when you think about the sheer amount of just human activity that is done on a computer saying we're gonna do that for you, it's like, oh.

Speaker 2: Exactly.

Speaker 1: So what do I do? Like, what's what would you like me to do? The the other thing I find interesting about this Microsoft recall story specifically was the about face that occurred. It was opt out when they announced it.

Speaker 2: Oh, so it was auto opt in?

Speaker 1: I think that was eight days ago. Yeah. It was. And the, like god bless the security community because the, like, strength of the, like, foghorn blast that they let out when this was announced, literally, like, they scurried back inside and came back out. We're, like, it's now opt in. Mhmm. You now have to choose to wanna do this. I'm sure it will be very thoroughly sold to users. Hey. Do you want to be able to do this? Do you want all this cool watch this animation. I'm sure it will be very sold. There will be a lot of love put into getting people to click that opt in button, but I'm happy when I see that. Mhmm. That that's a that's a positive thing. Doublepulsar.com, has been doing some great coverage on this. They dig a lot more deeper into sort of the technical questions at the heart of this. Microsoft is claiming something called UAC user account control prompts, provide this extra security layer. Their documentation potentially contradicts that. It gets into the weeds in a way that is sort of outside our purview. But if you are interested in reading about it, you should check out their coverage of it. Hey, everybody. Jordan here popping in after we recorded this, but before we launched it. Just today, the day after we recorded, Microsoft announced that instead of recall launching June 18 with all Copilot plus PCs, that's their name for, you know, AI empowered laptops. They're gonna be holding off while they listen to feedback. It's still gonna be available through their beta insider program, but the feature will not be pushed live to everyone buying a laptop because everyone got very upset. They said they have, quote, heard a clear signal that we can make it easier for people to choose to enable recall on their Copilot plus BC and improve privacy and security safeguards. So we're gonna hear what that means, but it does mean that a angry chorus of nerds saying this is alarming, did not go unheard. Where it goes from here, we'll see. Just wanted to fill you in on that very relevant update. Let's get back to the episode. It's a it's a it's a fascinating story and feeds into probably some stuff with Apple and and Adobe that we should talk about in a similar feed, but good start there.

Speaker 2: Well, I I was I I was about to make that hot transition when it came from auto opt in. The

Speaker 1: Make it.

Speaker 2: The Adobe terms of service. So Adobe Creative Cloud. So Jordan and I, both work in advertising, And Adobe Suite Creative Cloud is the most fundamental product Mhmm. For that entire space. Design, document layout, illustration, video editing, some great tools for audio editing that we even use here on the pod. Yep.

Speaker 1: They have a

Speaker 2: don't wanna call it a monopoly because there are competitors in each one of the little pieces like After Effects, which is an animation suite. There are some competitors to that. Photoshop has some competitors and Affinity Serif and a few other things. But it's like it is the industry standard and has been for so long that it's, like, an essential tool. It's like if wrenches were owned by

Speaker 1: There's a

Speaker 2: one company and you had to be a mechanic. Anyway, so they they dropped came out hot dropped their new terms of service, which in preparation for all of their AI data scraping and generative AI pieces essentially say that anything that you make in their software, they get a license to

Speaker 1: Mhmm.

Speaker 2: Perpetually. And that's because they're using everything you're doing to train their AI models, and they don't wanna run into like, we were talking about this, like, a year ago when we were talking about Steam and stuff. They don't wanna run into an issue where it's been trained on a dataset that they don't own. So, naturally, everybody freaked out, understandably, because we're talking about people who make their living in the creative fields. And then all of a sudden, Adobe is saying, like, well, we actually kinda own everything you make now with our software. Even though it's your creative output, we get a license to it, and people are unhappy about that.

Speaker 1: Yeah. There's a lot to unpack here. Like you said, the creative community has a very complex relationship with Adobe. I'm not saying the word monopoly. You're not saying the word monopoly. The word monopoly somehow still gets used a lot in discussions about this company, because they they they just make a lot of the software that a lot of people use. They are in AI. They have this Firefly AM model. It's been trained on Adobe Stock Images, licensed content, public domain content, and they roll out these new terms of service that the languages I mean, they're very, generous to themselves language in the press blast after this blew up was that our language was unclear. Uh-huh. We articulated this poorly. There was vague language, and they're gonna overhaul it. It's they're saying they're going to overhaul it to clarify that they won't be training, AI on customer work. But to me, it's very telling that the first pass through the gate was, oh, while it's unpopular, we would never train our models on your work. But we're carving out a part of this, you know, terms of service that explicitly allows us to. But we would know. Don't worry. We're not gonna do that. Yes. We could do it, but don't worry. We're not it's it's that kind of equivocating where I'm like, okay. Well, what what till when? Like, either take it out explicitly and make a promise that you're never gonna put it back in, or I'm just gonna assume this is a matter of of of when, not if.

Speaker 2: Yeah. And this is similar to the Microsoft thing they've all run back in, last I heard, and they might have released it by now, but they were coming back with some revisions to those two terms of services, the new terms of services. I know that there's been a big push. And even, like, internally in our company, like, we're making note of it and starting to evaluate alternatives in case we have to move away from Adobe because the other thing is is that a lot of the work that's done for paying clients, the paying clients wanna own. Yeah. So it's, like, in our contract that stipulates it as, like, we get the sole license to this, and it's, like, well, actually, Adobe also has a license to it. And it's, like, well, now you're in violation with a client license and a client contract, and now now you have a new problem that you have to deal with. Mhmm. So it's it's it's bad. It's bad. It's bad. Yeah.

Speaker 1: Yeah. I mean, it's they've they've spent such a long time as the king of this particular castle, and AI feels like the moment where they decide, are are we gonna be and to be clear, they haven't always been great kings. I they have they're at the vanguard of the subscription model shift. You can't buy pieces of software. Oh, your piece of software that still works? Well, we added this one tiny feature that's now hidden behind the subscription version. They have been at the bleeding edge of that, and that alone isn't great. But AI seems like it really represents an opportunity for us to, like, let's really get into the the data harvesting game in a major, major way because all of it's being produced on our stuff. It is such a glut of value just waiting to be, like, tapped into. Mhmm. And it it it should, I think, rightfully make creative professionals nervous when they see stuff like this. Like, oh, wow. That they're really just walking the perimeter of this big castle, here. They're really trying to find a way in. I don't love it.

Speaker 2: You know who do does love it?

Speaker 1: Who loves it?

Speaker 2: Tim Cook.

Speaker 1: Tim Cook.

Speaker 2: Actually, that's not true. Well, I don't know actually what's true. There there's a lot of discussion back and forth on this. Apple has long been like a company that I actually kind of trust with my private information. Like, they are really good about your phone security, local device security, not they didn't they fought about putting backdoors in their products. Like, they they seem to take your personal private information very, very seriously, and I've always respected that. Like, I it would have been years ago when we had a conversation about the FBI trying to break into a potential terrorist cell phone, and they were like, there are no backdoors. And it's like, we didn't wanna create one because they would be exploited if one existed, so we're not gonna create it.

Speaker 1: Mhmm.

Speaker 2: I'm sure that has changed since, maybe. But but Apple's always generally been on the edge of the personal information protection in that fight. They're not somebody who's like, yeah. Sure. We'll give you full archiving of all email addresses on our server. No problem. You know, rich rich text searching for every

Speaker 1: Sure.

Speaker 2: Email in our massive online hosted search or or emails platform. So so Sure. When they released the Apple private cloud computing AI piece this week, I don't know. I don't know what to think about it. What do you think about it?

Speaker 1: I think for a long time, there's there's a couple things to unpack there. Apple has and I I'm I'm a fan of this fact, has relied on narratives of privacy in their marketing for such a long time and has invested such a significant amount of money in telling the story that your iPhone is yours and it is your content is stored locally unless otherwise communicated. We are the privacy smartphone. You can trust us. They've spent so much money on that that I I'm glad they have because it's boxed them in and I think a lot of ways. Mhmm. So when we started, you know, realizing, okay, all of these companies are gonna start integrating more AI services into these devices. These AI services overwhelmingly involve calls to cloud, you know, to stuff happening on a server somewhere. Mhmm. How is Apple going to navigate that? I think there's a reason that, you know, Google was making announcements about, hey, how AI was gonna be integrated into Android, like, six months after ChatGPT came out, and here we are almost two years later getting Apple announcements. Mhmm. And the the problem that they had to figure out was that all the years they've spent putting way overpowered chips into certain devices are suddenly starting to make sense because the idea is we can do AI locally on a device. We can do a little bit of LLM stuff right here on your device. You wanna clean up an email? We're gonna do that locally. Isn't that cool? Isn't that Apple? Isn't that private? Don't you love that?

Speaker 2: You say that, but only the 15 pro. Like

Speaker 1: Oh, yeah. I know. Don't get me started.

Speaker 2: They're they're they're like, hey. You know, we we know we have all this processing power in your phone, but it's more than a year old. So you're gonna need to spend $1,700 to get a new one. Sorry.

Speaker 1: You can kick rocks. I didn't say they were, like, trustworthy actors. I just said they'd created a financial incentive not to abuse us in terms of privacy. But at a certain point, you know, if you wanna add some really bad mid journey style emojis to your phone, which they apparently wanted to Mhmm. Or you wanna let someone do something that involves, you know, deeper LLM back and forth, they needed to figure out a way to let the phone reach out to a server somewhere and talk to one of these models. And to do that, they've come up with the system called private cloud compute. It is this way of shifting away from on device processing while still hopefully trying to preserve privacy. It rapidly reaches the threshold of my technical knowledge, but from the stuff that I've read from a Matthew Green over on Twitter did a a lengthy piece on it. And the TLDR there is that it is extreme it is basically the big problem in computer security to create trustworthy computers that you can then connect to and send information back and forth on. And Apple seems to have done a they seem to have done a decent a decent crack at it.

Speaker 2: Because I know, like, I initially saw this story by seeing Elon Musk's reaction to it, which was essentially Apple is partnering with OpenAI, which I think was a big shock for everybody given that they're largely Microsoft backed. So it was an interesting kick at it that they would then be Apple's big partner for AI. Anyway, the the Elon essentially said, Apple's is open AI based. We don't trust open AI, and therefore, we don't trust Apple. If Apple goes down this path, unless they can prove to me that it's secure and not gonna steal our information, then we will literally ban Apple devices from our premises, which is quite the reaction.

Speaker 1: I wonder if he'll be mad if people use Grok.

Speaker 2: Especially I was gonna say especially for the cofounder of Ape and OpenAI, which Elon

Speaker 1: Yeah.

Speaker 2: Is.

Speaker 1: Hey. I respectfully chalk that up to a, a PR play in the context of a series of lawsuits that are going on. And I I don't mean that to be dismissive of criticisms about this for privacy reasons, and there are there are good privacy reasons outlined in that coverage I was talking to you about. First off, this is a massive target Even if they build the system where the data gets sent from the phone to the server and we don't even really have access to the server and you do all of this great stuff, you have still created a massive massive target. And you have also sort of created this, like, I'm hit or miss on slippery slope arguments, but you have created a situation where it's like, oh, we're only gonna outsource this to servers. And now we're gonna outsource this. And now we're gonna it's a you have started a process, and once people acclimate to the idea that their phone is is going off device, you can do more and more and more stuff from a long term security and privacy perspective. That's not a great thing. They were the last company doing everything almost locally, and we're starting to see that shift. This is cool tech. It's it sounds like it's about as good a solution to doing this kind of thing as is, like, currently available, and it doesn't sound like it's been mediated by some, like, and we gotta be able to scan it for ads. Like, they they haven't done anything like that. They have taken a really good faith swing at making that interaction very private and locked down and secure using the best available encryption. It's more though, like, what does it all mean of it all Yeah. That I'm interested in?

Speaker 2: I'm interested in why it's taken Apple so long. Like, Siri was introduced in February 2010.

Speaker 1: Jesus.

Speaker 2: February 2010.

Speaker 1: It sucked for fourteen years.

Speaker 2: It's literally as useless today as it was in February 2010. They've had so much time.

Speaker 1: They could

Speaker 2: have been building the frameworks and integrating it into the apps, giving giving it the ability to be better before, you know, OpenAI came along and a bunch of these new GPTs and LLM models. And and, like, to me, I'm just in awe that a company that's so good at reading the room, read the room in 2010 and was like, yo, automation's

Speaker 1: gonna be

Speaker 2: a thing. Let's do it. And then, like, rolled it out and did nothing with it. Like, they never like, I was always waiting for, like, app integrations. So, like, Siri would have hooks into, like, IMDBs. Can talk. Exactly. So I could be like, hey, Siri. What's the review of my phone is lighting up constantly as I say this. Just thinks I'm talking to it. The but you could ask it questions, and it would hook into apps and have prompts that you could execute, like even basic API integrations. And it it none of that stuff really ever came, And they just let it die on like, the only thing I use it for is, like

Speaker 1: Set a timer.

Speaker 2: Trying to call my wife, setting a timer. Because every time I say call my wife and say her name, it'll dial me some business in the city that her name is included in the title of. Like, it's Amazing. Yeah. Yeah. And it's like for a a product that's had fourteen years to mature, it is feels very immature. So I don't I'm that's my shock. That's why I'm shocked at for this. So I I'm looking forward to if OpenAI integration can actually make Siri functional.

Speaker 1: Well, here's the wild part about that is that all of the benefits to Siri don't concern OpenAI. All of the hooks into other apps have nothing to do with OpenAI. All of the, like, the majority of the LLM type stuff where it can, like, understand the basic structure of, like, a plain language sentence so you don't have to talk to it in this void, like that weird Siri voice we all do. Like, all of that advancement that we're seeing in this new version of Siri, that's not OpenAI. That's the local stuff that they've cooked up. That's all Apple. My theory is that if two years ago we hadn't gotten this generative AI explosion, we probably would have just gotten, like, a nice, hey, we made Siri better update, like, a year ago probably. It would have just been like, hey, Siri sucked. It's a meme. Here's a big press event, and it's all about Siri. And it's all about all the cool stuff Siri can do on all of your devices. And why did you buy a $3,000 iPad? Because Siri. But that's not what happened. History played out a different way, and we got an AI boom, and they had to pivot and figure out how now Siri isn't the big story. Siri is one story in this larger narrative of Apple intelligence. What do we do with artificial intelligence as this company that has, you know, told a story about privacy for such a long time? Now Siri is just a part of that. And I think that that's really fascinating to me, and it's it invites this larger question of, well, how long is that the story? Do we did we just need one event where you called it Apple Intelligence? And we all went, yes. We knew you were gonna do that. And now it just goes back to being a background feature inside of a calculator app or a photos app or the email app, which all look genuinely really useful. WWDC was cool this year. Or is this a narrative you continue to tell?

Speaker 2: Yeah. Yeah. Like, I it just like, Chat GPT four o, like, when they launched that Mhmm. With the the voice assistant in it, the content that I was seeing in the news and online was madness for something that had been released for forty five minutes

Speaker 1: Yeah.

Speaker 2: Versus Yeah. Siri that's been out for fifteen years.

Speaker 1: Yeah. It has market saturation. I have a like, I I use it I I use it as a better Siri, essentially, now Mhmm. Just because it is. And a lot of other people using, GPT four o as well for all sorts of purposes that I sense we are transitioning over to.

Speaker 2: That is true. We can that is a great transition away from us dunking on Apple, the world's second most successful business or largest business.

Speaker 1: Love love their products. And now you can use ChatGPT four o, locally on the device largely anonymously. And, also, a couple of months ago, a team of researchers released a paper announcing that they've been able to use GPT four for some other things. Do you wanna tell us about it?

Speaker 2: Oh, yeah. So I saw this article and, promptly fired it over to Jordan because there's a unique twist in it.

Speaker 1: I woke up to it.

Speaker 2: Yeah. He literally did. That That was just

Speaker 1: It's this is the social media app that I want. We were talking about it earlier. It's like, why have TikTok when you can have a Slack channel where you send, like, links about crazy text stories to each other? It's preferable.

Speaker 2: The, yeah. So my my deep need for knowledge consumption at 06:30 in the morning leads to Jordan having interesting things to read when he pops out of bed in the morning because we're in different time zones, which works perfectly. Yeah. So I am your in your, your news assistant, and I'm happy to be it. The,

Speaker 1: Reads a

Speaker 3: lot. Yeah.

Speaker 2: So a couple months ago, a team of researchers, put out this paper where they were using GPT four. So now this isn't even GPT four o. This is previous versions of it, you know, arguments aside over which one's better. But they were getting it to hack o days or zero day vulnerabilities, finding them and hacking them. So they were actually they were feeding the LLMs the CV list, so the common vulnerabilities and exposures, and seeing if essentially hordes of these GPTs could exploit them, and they were finding that 87% of the CVEs Mhmm. That the GPT horde was successful at exploiting around 87% of the common CVEs that were out there, which is crazy. So you've essentially got a red team of robots.

Speaker 1: They're, the thing I found fascinating about this, and it ties into this this kind of question people are asking about the the growth potential of LLMs and this approach to generative AI. You know, how how much better can these get? Where is the ceiling on this technology? Does the what does this tech lead to? Is that this this piece of research involves something called HPTSAs, higher hierarchical planning and task specific agents. You spoke to it a little bit, but it's essentially, they have one LLM functioning as a planning agent overseeing the process and a horde of sub agents underneath it that are doing specific tasks. It's a boss subordinate type model just made up of LLMs, which I find fascinating because it feels like it shifts the ceiling of what can one of these things do to be like, what happens when you fill an office full of them and give them a task?

Speaker 2: So this is exactly the same thing that I was intrigued by is is we went from, like, hey. I've got an LLM that I'm using to hack things

Speaker 1: Mhmm.

Speaker 2: To I have an LLM that has 500 subordinate ILMs, and the LLM is now planning and coordinating the actions of the 500 children LLMs, and it's making it over 500% more effective. So it's like we've given we've given the robots their own robot management, and it's it's leading to higher efficiency and more productivity and output. Like, tell me that's not crazy. Like, we're we're we're in it now.

Speaker 3: No. That sentence just that sucked so hard to hear. We're in it now.

Speaker 1: I think about when we rebooted this show a couple years ago, and it was just, like, sometimes people call, cell phone companies and pretend to be someone else. And now I'm like, so we taught the robot to be the boss of other robots. Like, how much has changed

Speaker 2: Yeah.

Speaker 1: In such a short period of time?

Speaker 2: We created a subject matter oracle robot that we then made a resource for all of the other robots, which made it more productive. It's like, okay.

Speaker 1: Totally. This is such an interesting, like, space for it, like, this this this Oday world. As an application for that, they did a benchmarking test, and it was, 15 real world web focused vulnerabilities and this structure, hierarchical planning and task specific agents, was 550 more efficient than a single LLM at finding and exploiting those vulnerabilities. It's it's all right in the name. It's this we have these these LLMs, but when you add a task specific element to it and a hierarchical structure, it's when you say this one gets to tell this one what to do and they're both trying to do this. It seems to be having some pretty fascinating impacts on what these things can do, not just as generative tools, but as, like, agents. Yeah. It's can we send you out into the world to do something? And this structure, this hierarchical structure seems to be at at that shifts the ceiling up a little bit higher.

Speaker 2: I just wanna hit the, like, hard data on this. So it's like

Speaker 1: Mhmm.

Speaker 2: You get a solo LLM, you train it to, like, hack things, You give it to CVEs, and you sic it on these, like, 15 vulnerabilities. So one well trained robot.

Speaker 1: Yeah.

Speaker 2: It's getting 20%, three and fifteen. You get the whole structure and model and and child agents and and somebody coordinating it and managing it and determining what tasks and processes that the child robot should be doing. You give the robot's management eight and fifteen.

Speaker 1: Yeah.

Speaker 2: It goes from 20% to 53%. Like, just let that hang for a second.

Speaker 1: Woah.

Speaker 2: You create an LLM that's so good at something, and it gets to 20. You create an organization of robots and give them structure and management just like humans. Like, you know, a single single person doing something. Like, all of the organizational behavior and analysis stuff from the business world is about to hit the AI world. I'm calling it right now.

Speaker 1: Sure.

Speaker 2: Like, to get to get that to to get a what is that? So if one's 20% and one's 53%, that's what? Like, a 150% improvement in productivity by essentially giving these things an organizational structure to work in.

Speaker 1: This, this reminds me of this is again way above my head technically, but I I read for I read at length about the, like, nonlinear gains to AI quality output from human reinforcement. Like, you you need to just feed endless data into the training models, but a little kiss of human reinforcement. A person just being like, that doesn't make any sense to me, a human, that had massive gains. And that was something that over the last, you know, five to ten years, that is what has led us up to this point is realizing the, like, disproportionate gains of human reinforcement. And this feels like it potentially has a similar thing where it'd be like, oh, one model's good, but if you put two together, it's not 200% as good, it's 300% as good. Like, the second you start nesting these things, you start having disproportionate gains, And that feels like, okay. That's probably what everyone's gonna start mucking around with next is what happens when we organize these things. How do we organize these things? Give them departments. Give them department. Like, we're literally getting into, like, that kind of, like, HR for AI phase of the whole thing. The models themselves will continue to be getting to be getting better, but it feels like how we organize them will be the next big push.

Speaker 2: If if if university researchers that are looking into this aren't already collaborating with the organizational behavior departments and the business schools Wild. I'd be I'd be shocked.

Speaker 1: Yeah. I mean, I know what you mean. It's like, what what give us some ideas. Like, how are people organized? What are what are the ways? What what are the benefits of a flat hierarchical model? What are the benefits of an incredibly rigid one? One manager five hundred's subordinates? Yeah.

Speaker 2: Yeah. Yeah.

Speaker 1: What's different in the context of an AI where there aren't gonna be communication breakdowns? These things can talk to each other perfectly or close to perfectly. It's gonna be interesting.

Speaker 2: Well, but also think about, like like, imagine everybody was task specific. Like, you watch those TikToks, which I don't watch, but other people do, of, like, you know, the person who makes Japanese teapots.

Speaker 3: Sure.

Speaker 2: And it's like, that's the one thing they do, and they do it every day. And they are the they are the master of the Japanese teapot creator.

Speaker 1: Sure.

Speaker 2: It's like that person that person is solely, like, has a solo task. They are a subject matter expert and and specific set of skills, and they have years and decades of experience doing that one thing. It's very hard in a business to get down to that level of of of isolation. Like, every single person has one specific thing that they do because human resources cost so much. To get to that level of specialization is is virtually impossible. But robots are free. Not free, but, like, you know, cost per resource, very easy to replicate. So you could start to create organizational structures that are way bigger than we could ever have in a real world, in, like, a human world situation because you could allow that level of specialization and that level of specialization management and coordination, communication agents between them. So I think that the I think that this is gonna be an interesting field to follow. The organizational structuring of AI teams.

Speaker 1: And will they do IT projects that funnel money towards WMDs? It's it's

Speaker 2: it's gonna

Speaker 1: be it's gonna be great.

Speaker 3: We're gonna be here

Speaker 1: to talk about all of it. I I think that might be another one in the books. What do you what do you think?

Speaker 2: I think so. Yeah. I feel like that's a good place to end.

Speaker 1: It's a good place to place to end. Thanks again for listening as always. Thanks to everyone who shared stories with us to talk about. Looking at you, Bruno. Dirty eyes emoji. And we're gonna be back at it again soon. Thanks for listening. As always, catch you in the next one.

Speaker 2: Take care, everybody.