A.I. Agents: Cute, Cuddly and Maybe Catastrophically Dangerous?

Modern Enterprise is made up of a lot of moving parts, and Comcast Business helps you orchestrate it all.

With SD-WAN working at scale to keep 150 hospital locations connected and working as one,

plus Sassy and Zero Trust Security protecting financial data across a bank's 2,000 branches,

and AI-powered networking that optimizes traffic across five continents.

No one does business like Comcast Business.

There are two things I'm having trouble keeping track of in tech right now.

The first is all the new personal assistant agents.

There's Meta's Muse.

So talking to your Muse, it's often the fastest way to get things done,

and it is also just really fun and cute.

Google Spark,

I can just put my phone away and get on with my day,

and Spark works in the background for me.

An agent called Instinct, and just announced this week,

OpenAI's new product, Dots.

Dots are remarkably capable, always on agents, built to handle really anything you can think of.

The other thing I'm having trouble keeping track of is all the websites and institutions

that have been hacked,

illicitly accessed, or otherwise meddled with by rogue agents.

The AI platform hugging face, of course,

plus the Australian government's Medicare data portal,

the Education Department, the Commerce Department,

the Security's Exchange Commission,

and Obscured German Wiki.

All of this culminating in OpenAI deciding not to release its latest model this past week.

So right now, two things seem to be true.

One, agents for the thing that will make AI useful for regular people,

and two, they're maybe going to kill, hack, or otherwise meddle with us all.

The President's attitude about this, by the way,

is that companies should figure it out themselves.

This is Hard4.

I'm Max Reed, and over the next few months,

I'm going to be joined by an array of New York Times reporters and other tech experts

to discuss some of the biggest stories in tech AI and our weird new future.

Today, we're talking about rogue agents, helpful agents,

cute AI avatars, and whether there's such a thing as self-regulation.

I'm joined by Mike Isaac, Aaron Griffith, and Eli Tan.

Hi guys.

Hey.

Before we really get into it,

Casey and Kevin left me a scrolled note here in the studio.

It says,

run while you still can.

I'm kidding. It says, don't forget to disclose that the New York Times is currently engaged

in a lawsuit against OpenAI and Microsoft over alleged copyright violations related to the

training of large language models.

It also says, check to see if your wife works at Anthropic.

My wife does not work at Anthropic.

I want to start today with an icebreaker question.

Tuesday was a big White House AI summit luncheon, Trump,

as he is want to do truth-social doubt, a seating chart of it.

Can we put the seating chart up so people can see it?

Because what I want to ask each of you is,

where would you sit?

Imagine you're walking in.

You've got your tray.

You've got a plate with a little pre-made sandwich and a glass of milk and like a green apple on it.

Where do you want to sit?

Right at the gate, they already are way over their capacity for women with two.

So I think that I probably would be standing somewhere in the corner off to the side,

not actually at the table.

Eli, where are you sitting?

I would be sitting awkwardly and quietly between Chimath and Suzy Wiles.

I think that's got to be some really good gossip there.

I think that's a good, that's a really solid.

That's where the tea is.

Mike, what about you?

I look coward.

I just want to be serving shrimp canopies and going in and out of the room.

And being the guy, everyone is delighted to see because he's bringing food and that's about it.

I know this is a bold choice.

I think right in the Jensen mark, Trump circle is actually, it's like shooting the moon.

It might be the most tolerable lunch that you could have.

I think that if they're in a good mood, you probably get some good stories.

You can wrap with them about whatever's going on.

I think it's a little danger as elsewhere at that table.

There's some strength, like sitting between JD Vance and Jeff Bezos.

Like, what's the vibe there?

I don't know.

It seems a little scary to me.

Unfortunately, none of us were invited.

So none of us managed to be at the biggest party of the week.

So this was a summit as we were saying on Tuesday between the Trump White House and some of

the richest and most powerful names in AI, which culminated in a press conference featuring

Trump and all the CEOs.

Can we play a little bit of the Dario laughing and talking in front of the reporters?

I just want to hear from you after this big summit where you are today.

Look, I mean, I will say what I've always said, which is that AI has incredible benefits.

There's the side talks.

I talked about the medical benefits.

The jokes, the laughing, the smirkings.

As said, whoever wins AI wins, I think that's very important.

But I think the technology has very real risks with Mark.

The mechanism of how we address those risks is still under discussion.

And then we have Trump forcing Mark to say some to basically answer a question.

He does not know the answer to in front of everyone and don't get it wrong.

Mark, at a view that was, I felt really right down the middle block.

Discuss what you did.

I can kind of summarize what I think we all signed today.

The upshot of this was that it was a rebranding exercise, right?

Like the kind of what they got at the end of it was we're calling AI super intelligence now.

That was the big commitment.

Well, right.

I mean, Trump, along with the rest of the tech industry, maybe they are the ones who put this idea

in his head, knows that AI needs a rebrand that right now people have pretty negative

feelings about it, especially associating with the data centers.

And then now with everybody in the industry saying, oh, I might kill us too.

Not great branding.

So change the narrative.

Super intelligence.

My read on this scene was that Mark Zuckerberg seems to have quite the influence

over the administration right now from kind of the cordial talking with Trump.

To even the term super intelligence, this is something that Mark's been talking about for the last

year. I don't almost seem like from a kind of like a high school friend group where they've

invited the new awkward kid.

And they're still trying to show that, you know, they've been buddies for a while.

Yeah, but didn't it did almost kind of like endear me to Dario a little bit because I'm like,

okay, either he hasn't gotten media training or the media training just hasn't stuck.

Because like, you know, he's itching in his suit.

He's like kind of like making weird faces.

It kind of like almost made me feel like, okay, like, you know, the rest of these guys are like

very polished and button up and kind of like trying to do their like alpha male thing.

And here's this guy like, he can't help but be himself.

I kind of like won me over a little bit.

I was thinking I'm sure I'm not the first person to make this observation, but I was thinking

that Dario really, we all know that Trump has a kind of central casting problem that he needs

people he interacts with to look like what he expects them to be.

And without wanting to be rude about it, I do think Dario, if you are thinking like,

super intelligent, math genius nerd, like that is good.

Like Dario is kind of what you're picturing your head.

And I think that may be one, one him over to Trump.

The Trump was like, I understand this.

I think possibly Sundar had the same kind of effect as sort of taller, ganglier,

awkward version.

Can we play the Trump, the clip of Trump?

This is a cup. What we're about to hear is Trump complimenting Sundar.

This guy is a monster and nobody knows what he's like.

A monster.

Not after he goes through what you can work down.

I'm just wondering what you're doing.

Just what you have to do is Sundar.

Wow.

I don't know if I would want to be called a monster in front of everyone.

No, it's monster in the Jay-Z Nicki Minaj sets.

That's the monster he wants.

That was for sure the reference.

Okay, so in addition to this being a big exercise in branding, which is like the most

Trumpian sort of, you know, after effect we could have gotten out of it, the other thing

was that Trump effectively asked them all to self-regulate.

And I want to focus on this for a second because self-regulation, the word, the concept has sort of

been circulating for the last few months as AI Safety has come to the forefront of a lot of

people's minds. I'm sure you guys all saw Ezra Klein, our colleague, interviewed Jensen Wong,

the CEO of NVIDIA last week.

Don't ship products until they're in control.

It is really quite that simple.

And speaking of people who are very influential to Trump, Jensen was sitting right next to Trump

and has been extremely influential to the way Trump has been thinking about this.

And I have to say, and I want to talk about it with you guys. I found Wong's interview to be sort of

refreshing as probably taking it too far, but it was a little different from a lot of what we've

heard about CEOs sort of directly in the AI industry. He's got a very different perspective.

And I want to talk about why that is not just on sort of regulation in general, but how he thinks

everyone should think about the dangers of AI. Can we play the car, the clip about the car if

you've got it?

Ezra, it's so weird. If a car company competing with all bunch of other car companies, which they are,

I'm competing with all kinds of companies, which I am. If I believe that I'm about to

launch a product that is unsafe, it is completely in my ability, my power, and my responsibility.

And I'm incentivized to do so to not launch the product.

So Jensen is sort of saying, look, if you're creating these things that are super dangerous,

crazy, you know, going to kill people, why don't you just not release them at all?

What did you guys make of this interview? Was it as sort of,

is it, did it feel new to me because of, from my perspective, or was this a fairly common view

in the Validium thing?

So it's the like the tech industry slogan of like, you can just do things, but you can just

not do things.

I like that anti-agency. That's a good, that's great.

I would argue it's kind of naive and perhaps willfully naive to suggest that everyone is just

going to not do stuff if it may be bad or has the capacity to be bad. And you know, has that gone

well for industry so far over many years, not necessarily. Like there's a lot of ways you can

release things that may be somewhat dangerous or unintentionally dangerous, but it felt very much,

like he was trying to shift as what Aaron was saying before, trying to shift this narrative of,

it's not these systems are inherently dangerous, but it's, we are the stewards and the people

who should be responsible of shepherding these great systems into the world. And if you're not,

if you're open AI or Anthropic, really, this is targeted Anthropic, then it's a personal

failing of the CEOs for you to release these products. And like, I just, I, I'm very curious why

you found it refreshing because I felt like maybe I was being too cynical about it. No, I mean,

maybe refreshing is the wrong word. I feel like I've spent the last few months or few weeks at

least hearing a lot of kind of hand-ranging about what's happening. I find this sort of the, the,

the Dario Altman pose a little, I don't know, it just feels, I mean, I suppose in some ways I have

the gent, I have a sort of Jensen-esque like, okay, we're like, why are you like, okay, so stop.

And I know that there's this elaborate scaffolding of rationalization and justification for why you

would continue building AI, even if you think it's, it's got a 10% chance of blowing everything up.

But I do think that Jensen in his heart, I'm scolving by his first name, that's how you can tell I'm

captured. I want to be clear, I don't, I really disagree with a lot of stuff in this interview. I'm not,

I'm not, this is not an endorsement of the take. But I do find him, he's an interesting figure to

me because he's not somebody who has come up through the world of machine learning, which has a

really specific, as you guys all know, has a really specific kind of culture and a kind of sense of

what they're doing and what they're building is. I mean, this is a guy who is a graphics card salesman.

That is his job. That is what he's been doing, you know, for, for 30 years now. And I am, I'm sort

of very sympathetic to a guy coming around, you know, if we can extend the car metaphor, he's like a

guy who sells mufflers or whatever. And all of a sudden, Volkswagen's saying, you know, if we build cars,

like they might all blow up on the highway and kill all of us. And he's like, well, you're stopping

me. First of all, like, I've been selling these mufflers forever and it's been, I've gotten really rich,

I've, it's been great. Why are you messing up a good thing? But also, it's like, they're just cars,

just build the cars. And I thought this was an interesting contrast with the Bill Gates interview

that Ezra did this week, which was another end of this sort of spectrum. You can't rely on the

industry to self-regulate here. I mean, it's just insane. Gates is very pro-regulation. But the

place where they both met up was there very much people who see AI as a tool that can be, that needs

to be addressed as a tool instead of as a kind of a potential superintelligence. The, you know,

the only sort of, the only way it can be dealt with is by properly, is by like teaching it to be a

good person, basically, instead of like being through a relationship. Don't you think that you're just

like getting captured by like a more palatable version of the like accelerationist argument?

Or, you know, it's like accelerationist, but like minus the like and also screw the little guy,

you know? Yeah. The slow accelerationist. Yeah, yeah, yeah. Or it's just like, it's like that same

argument, but it's just like a little bit, you know, kind of goes down a little easier because

it doesn't have like a menace to it. Yeah, I think that's right. I mean, I definitely am getting,

I'm like, I've like contrarianed myself into like a position that I don't actually hold just by

like trying to triangulate my way out of it. But I do think, I mean, the one thing, the one of the

contrarian these days. No, exactly. The one other thing I'll say, I think that there's a real,

the value of having Jensen Wong talk like this is that it feels to me very often like there's

a real echo chamber in the in AI world. And I'm not saying that like adding more wrong opinions to

the echo chamber is necessarily going to somehow produce a right opinion, but it is nice to have

to like start to like to sort of bounce around some ideas that don't have anything to do with

the kind of rationalist directed rationalist oriented, you know, EA oriented, AI safety oriented

things. I mean, the end of the day, I think I more agree with Bill Gates than I do at Jensen Wong

about the need for some kind of like sector-specific regulation. But I also feel like

this is such a fast-moving environment. It's really hard to say how we should approach it at all.

I think it's really weird that they all have been not all, but many have been begging the government

to regulate them and enforce them to slow down. And then they all get together and sign this

pinky swear that's like morally binding that has no teeth to it that says like we're going to

police each other. And then they all sort of like are like, yay, we did it because Trump doesn't

want actual regulation. And is that it? Are they going to stop asking for real regulation now?

Because they've all sort of like agreed to this very flimsy piece of paper?

It's a very good question. And I'll just note that the FTC is now looking into Open AI and

a topic which feels a little bit like does the government want them to self-regulate or does it not?

Because the way they seem to want to self-regulate is by effectively forming a cartel. And if the

government is going to, you know, bring antitrust prosecution against them, then that's not,

they're not going to be self-regulating anymore. But I want to follow up, I want to follow that

thread a little bit because part of what's at play here that I think gets ignored by the fact

that it's all the CEOs at the summit and these are interviews with Bill Gates and Jensen Wong

is that there's this huge mass of people inside the labs, researchers and workers and engineers

who are themselves trying frantically to pump the brakes or to figure out, you know, exactly what

they're doing and how to do it. There's a great, our colleague, Sheriff Frankl had a great

story this week about people inside Open AI raising the alarm over lack of monitoring, lack of

proper security procedures around a bunch of the packing tests that have resulted in stuff like

the hugging phase incident. Are you guys hearing, I mean, is this, is this, do I have the right,

like sort of take on this, that there is, there is a fair amount of unrest within these companies or,

if not unrest, at least sort of a sense that the workers are taking it into their own hands?

I mean, certainly at Open AI, like, which I think stands out from all of the other

frontier lab companies and the number of hacks and attempts and the number of things that they've

found that their AI agents are doing without their realizing and, you know, Sheriff's story is

incredible because it focuses on just what her sources call sloppy security practices. It's not

just like sloppy for a company that's building really powerful AI tools. It's just like sloppy

for any company. Like there's this really great, great quote in there. The guy says, why are you

using Slack to build your nuclear Manhattan project? Because some white hat hackers got access to,

like, their Slack logs amongst other things that they got access to and they brought it to Open AI,

which was standard security practice in Open AI, which is like, whatever. And then later kind of came

around to it, but it just seems like a company that's not taking this stuff seriously, despite,

you know, saying all the right things and telling the government we want to be regulated and you

got to slow us down, like, behind the scenes, they're kind of, they're not really that good on security,

it seems like based on this reporting. So the fact that these leaks are coming from Open AI first

doesn't surprise me because of the sheer volume of incidents that they've had. I will say, though,

that the culture of inside of these companies of employees just like popping off, I think is,

and this is not to say that they're wrong, but it's like, it's very, it reminds me of like,

back in the day at Facebook or the Web 2.0 companies went anyone was saying anything and they were like,

you know, speaking up to the boss and like saying, this is wrong, we shouldn't be doing this,

which like, we're in a very different era of Silicon Valley now where that is not the case inside

of Apple and Amazon and Google. And Google, especially they were, you know, the, the, like,

sort of pinnacle of like employee descent and, and message boards and discussions and ultimately

culminating in a walk out, ultimately culminating in them shutting that stuff down.

What about inside Facebook say, which is a player, a major player in the AI space? Is there,

I mean, is it because they're not a frontier lab? There's just less of a sort of employee culture

of fear and descent and whatever else? I think that the Facebook does feel, and you like

to talk about the stupid, sorry, meta does feel a little bit different just in a few ways. One,

I think the biggest concentration of, let's see, let's say people who are like alarmed at what

AI can do and feel like it's super dangerous is in Anthropic is probably going to be an Anthropic.

Those have also dispersed across OpenAI and DeepMine. And I feel like the folks inside of meta,

in general, I'd be curious to hear what you think you like, but like are kind of more like

foot on the gas, man, let's just like it's open source. Everyone should be able to use this stuff.

Yeah, there's a few, if we take out a few power grids along the way, then that's a cost of

doing business. But I do think in general, their way more acceleration is compared to the

conservatism of some of the main labs. Is that right, Eli? I don't know, I feel like that.

Yeah, I think it's been interesting to watch when Mark staffed his new AI lab with all these new

researchers, given them hundreds of millions of dollars a year ago, he was bringing in these types of

people from companies like OpenAI. And now in the last year, you watched them tweet and they're

talking about products and flywheels. It's like they've been totally modified. And I think their attitudes

have changed a little bit. I'd like to see, I'd like to medify the three of you. I'd like more

products and more flywheels on this on this call. I want to talk so, but this is all leading

somewhere because the other big news or the big news to me at least this week was that OpenAI

scrapped the release of its newest AI model, GPT 6.1 Astra. The quote that has been

ringing in my head is the head of OpenAI safety systems, Sachi Jane says, GPT 6.1 Astra showed

higher levels of deception. It wasn't always honest about telling users of the actions it did

or didn't take. Sounds like my worst breakup that I've gone through.

You should have, yeah, well, this is one of those AI, they can't not talk in sci-fi cliches,

but also they are building cliched sci-fi minds. So I guess that's how you have to talk about it.

I'm interested in this in a bunch of different ways. One is, to what extent do we think this is

about the internal push? Is it that the people at the top of OpenAI might have released this,

but they're worried about pushback from inside the lab? Is it more that they're just, they've

been like brutalized by story after story of the various hacking, scraping, to use our favorite word

meddling with websites that OpenAI agents have done? Aaron, when you read about this, what was your

reaction? They're pacing the frontier. They are. This is in fact them pacing the frontier.

I mean, is this self-regulation? I guess, I mean, I don't know. I started seeing all the conspiracy

theories about like they're doing this because they've lost control. I don't believe that, but

yeah, they don't want more bad headlines. They don't want more instances of this, and they're still

finding ones. So maybe a bad idea. Yeah, there's also the fact that that morning,

Anthropic released a new model basically, which continues to bring up the question,

like, why are you doing this and releasing new models every day if you're freaking out this much?

And I do think inside of OpenAI, they like playing for keeps and stick in the knife in their

competitors whenever they can. So positioning oneself as the responsible actor as your main

competitor is pushing ahead, I would not put that past how OpenAI thinks about some of this stuff.

Well, also saying, hey, there are people inside that do believe this is like dangerous stuff.

This is an example of rogue agents. One of the things that I thought that the Jensen Wong

interview sort of underlying quite well, in fact, was the way that part of what we're talking about

here is a shift change, and just in the last six months, where we have a sense of how much more

powerful, how much more capable, and in some ways, how much more dangerous these models can be.

But one of the funny things about the technology is that the power and the capability and the

danger is also overlaps quite a lot with usefulness. So we've also had been hearing a lot more recently

about useful AI agents, autonomous AI agents that can do things like change your calendar, respond

to emails, reschedule your haircuts, whatever else. Eli handed his life over to the meta one

for the sake of the education of all the rest of us. And after the break, we're going to ask him about that.

17 Courts, Hundreds of Matches, Thousands of Points, all happening simultaneously during the US

Open, and you can't possibly be there for everything. That's why IBM helps bring together

billions of data points with more than 20 years of player history, with scores, stats, insights,

and highlights behind every moment. So nothing gets missed. IBM, let's create smarter business.

New York, HSBC brings strategic insights and a global perspective to give you an edge wherever you go.

This message comes from Betterment. Betterment Stan Egan talks about tax loss harvesting.

Tax loss harvesting is a tax management strategy. When you have a position that's gone down over time,

we intentionally sell out of it to realize a loss, which we then say to the IRS, hey, we lost money.

You get to use that to offset your ordinary income every year, decreasing your tax burden.

Investing involves risk performance, not guaranteed. Betterment does not offer tax advice. TLA

can may not be suitable for all customers. Learn more at betterment.com slash TLH-Terms.

Among the funniest things about this week, in addition to open AI,

apologizing to the Australian government for improperly accessing various websites,

and announcing that its new model was too scary to possibly release,

they also announced that they were going to have a personal assistant called Dots.

So Dots joins, among others, Instinct, Google Spark, and Meta's newest product Muse,

which has been... Also, there's one called Town. No one else seems to have heard of this one.

Town? Does it do something different? No, it's just the same. There's a button. There's actually a

mirror. From Boston. Yeah. So Dots joins... Yeah, Town. It buys Red Sox tickets and schedules bank

rubbers for you. Dots joins Instinct, Google Spark, Muse, and Town. All of these have been

receiving a ton of interest. They're interesting because they are sort of a little bit less freaky

and scary than the AI that hacks things and does all the math problems for you. They do things like

monitor your email, check your calendar. They create tight and exercise plans for you. In some cases,

they will even make annoying phone calls that you don't want to make. Eli, I want to talk to you

about this because you, as I say, turned your life over to Muse for the last few weeks. What has

that been like? I did, yeah. For our listeners, they're wondering what kind of agent it is. The

helpful agents look kind of like Labubus. These companies have this idea. And the ones that

hack governments, they have, I guess, little hoods and hacker outfits. But yeah, I turn my life over

to Muse. I gave it access to my email, my calendar, my bank accounts. I let it do everything for

two weeks. And I wrote about it and I found it to be one of the most helpful AI tools that I've

used was my- So for people who haven't opened Muse, tell me what's the interface? How do I,

you'd say an app on your phone, right? It's an app on your phone. You download it from the App Store.

You sign in with your Meta account, Instagram and Facebook. And it looks just like a chatbot.

There's this little avatar at the top who he starts out like a Labubus, but then he asks you to

give him a name. And his name was Ren. He's a tiny little owl. And you talk to him like a chatbot,

or it like a chatbot. And then the little owl types on a little keyboard. And it can do all the

tasks for you and sign into all these accounts and kind of walks you through it.

So what kinds of tasks were you asking it to do? I had it order me groceries for pickup at Whole Foods.

I had it some of the kind of the trial accounts, you know, they have a phone feature where you can

use these two personalities. A female voice called Haley and a male voice called Brett. And you

can have them make phone calls for you. So I called my- Okay. My dad, did you name them? Wait, back up.

I did not name them. These are the names that Meta gave to these personalities. Oh my god.

So I would have Haley called. They're posing as humans, right? Like they're not saying,

I am an AI agent calling. They're saying, I am Haley. They say- They say, I'm Haley and I'm calling

on behalf of Eli. I had to call my dental insurance. It gave them my member ID,

answered a security question about my birthday, and then it waited on hold. And then

10 minutes later, I pick up my phone and it's Haley calling and saying, I'm about to transfer you

to Anthem. And I go, oh my gosh. But doesn't like- Yeah, them already has that service. So, okay,

sorry, I'm jumping ahead here, but I want to hear from you. It feels like you're just managing

a bunch of interns, like that you have to then- it makes more work than for you because you have

to then tell them, okay, here's all my stuff, here's how to do it, here's what I want. And then

if they mess it up, you have to fix it. Like, did it really save you that much time?

When you're ordering groceries with it, let me just take an example. So you do text it,

you know, I, can you get me onions, potatoes, peppers, eggs? You're making a scramble in this case,

I guess, that sort of breakfast. Or you like, I want, I need to make breakfast. Can you just order

me some breakfast stuff? In this case, I gave it a New York Times recipe and I said, can you order

me all the ingredients you need in this dress? I gotta keep it on house. But also the Times app

has that feature in Instacart. You can just push a button. Wow. She has one up to Eli in promoting

New York Times. No, that's true. I also would, I also, every time I would order something online,

I'd say find the wire cutter recommended. Oh man, use their referral link because you want to keep

all the, you know, I want to keep it on house. But I mean, one question I have, I had about this

because I downloaded it and I opened it up myself is the first thing I encountered is just has a

long list of things it's suggesting for you to do because it's like, this is, it's an interesting

product where it's not clear, it's not, it wasn't entirely clear to me and in some ways still kind of

wasn't even after I read this long list. Like what exactly I need a personal assistant in my

life for? So it has everything from like you're saying, order groceries for me, create an exercise,

plan for me, create a diet plan for me, break a bad habit. You're going through it, Max. Well, yeah,

no, there's a real, there's a sense, at least that I had and I'd be interested like as you

interacted with it that it's the kind of thing that you need to, you need to like get in the

group of using it and then all of a sudden the world kind of starts look, everything in the world

starts looking like a problem that your personal agent that Haley or Brett might sell for you.

It's Ren, it's Ren, it's just Haley Brett. I would like to be a fly on the wall for the

multi-month meta meetings where they decided on the names Haley and Brett. Anyway, sorry,

tell me more about it. You like, yeah, it's definitely the type of thing that meta thanks,

you'll just use for everything personally after my experiment. I think the thing that I'll use

it most for is probably just personal finance, tracking all the things I'm spending. But while I was

using it, I mean, it was like, I kind of, I don't want to say I got a little bit of AI

psychosis, but like I was using it for everything. I mean, at one point, I remember thinking,

I went to ask it to, you know, preheat my event to make the dinner. Like that's the level of, wow,

that's like the level of that is rough. This is, I think, what's a little bit scary about,

and like what some of the people who are pushing back on the like anthropomorphizing of all these

AI things is that once we assign them like more human like qualities and like give them this kind

of mystical, magical ability to like they know everything, then we just start outsourcing more and

more and more of our like decision making and our even our thinking to them to the point where yeah,

we're asking it to preheat our oven. And like, or the, the guy, the tweet that I saw of the guy who

like, paste it a bunch of text into Claude and said, how many words is this?

You're just like, people, you know, just turning to it for literally everything without like

thinking what their brain first. It's like kind of a little bit scary. That's a side rant though.

Well, wait, I do want to know Mike and Aaron, you guys, you guys are out there on the bleeding,

on the bleeding edge of technology and culture. What, what are people like are, is the, is the Bay Area

absolutely rotten with people using agents to do everything they can? Oh, yeah. I mean, instinct

went super viral before meta launched. All of these sees were using it and going nuts trying to

invest in it. And tweeting about, you know, in some of the, the use cases that they tweet about are

genuinely like useful because this thing is so persistent. That's what it has that humans can't do.

Like, it's not just like, oh, I'm going to tell it to order my groceries, which I can also just do

in an app. It's, um, get me this restaurant reservation or like, go back and forth with Amazon's

customer service to make sure that my refund actually came through. The kinds of stuff that humans

just don't have time or aren't organized enough to like stay on top of. I really do like that

restaurant reservations of the number one problem that American technologists are trying to solve

like so much so that they're breaking resi in these apps or getting banned from like instinct

is the, is the hot one right now. They just raised a billion dollars there like, uh, they have

a wait list. I was at a, I was at an event yesterday and I was speaking for it. And this woman came

up after and was like, can you get me on the instinct off the wait list and into the app? But I do

think like it's a thing out here. And I think they, the way that people think out here is like,

this is where it's all going. So, uh, so you just need to sort of get on board with it and,

and rather than ask the questions about should, uh, should I hand these intimate parts of my life

over to this robot? They glue, they go like full Eli mode and say, here is everything I have and

go for it. And I mean, I don't think that's just an out here thing though. I think that,

I think that like handing over your life in exchange for convenience is a thing all across

America because people are busy and they're, and they're willing to make that, that privacy to

convenience trade off. But the thing out here that people think in my opinion is that they view every,

or I don't want to paint it with two broader brush, but many people view every interaction with a

human as a source of tension that can be solved with technology. Rather than just calling a host and

be like, do you have any tables? Eli, did you make it? I want to know if Eli made any restaurant

reservations with, with, with, with, with, Ren. I did make reservations with Ren. And I think

something that it's important to remember here, meta, there's spending $600 billion on this restaurant

reservation app. Like, yes, it's good at restaurant reservations. It's okay at movie tickets. Like,

this is a, but this one, like for $600 billion, it should teleport me to the restaurant or something.

Like, this is a lot of, yeah, this is a lot of investment for something that's pretty useful

at some stuff. Yeah. Which I think we should remember, right? I remember like 10 years ago when

we were in the sort of peak of on demand app, Mania, when it was like, you know, you have your app

for your on demand butler who's going to go get your dry cleaning and your on demand, like house cleaner

and your on demand, like everything could be accessed by an app. And it was like this new, you

know, millennial lifestyle subsidy utopia. I remember some investor telling me like, there will be a

time when you don't make any decisions without first consulting AI or first consulting an app

because they will know the best place to get your dry cleaning or the best like time to order your

car or like whatever details, any kind of decision that you make is going to be like consulted by

AI or by an app. And I remember thinking like that's insane. That seems like a huge time suck to

have to be like negotiating back and forth with this app when I just know the decision in my brain.

But I now think we are moving in that direction. Like in that a lot of people would rather outsource

some of these like minor day to day minutiae decisions to an AI that knows them really well and has

all their info and can just go ahead and execute it. Like I do really see the appeal there just to

argue against myself. I mean, one of the one of the things at at at at stake here is like,

if everybody has an app, if everybody's got a personal assistant, if everybody has access to

the best dry cleaner, the best restaurant, the best whatever kind of nobody does because you're

just in an arms race against everybody else's personal assistant, everybody else's whatever else,

which seems like a not a great world to live in in particular. It's a question of how many agents

do we need? I saw today I think it was door dash. They release an agent in the app. I mean,

are we going to go into every app? Am I going to open the subway app and have a sandwich agent that

clicks on whatever I want for lunch? It's called the sandwich artist. Okay. Yes. Thank you, Mike.

How many how many agents do we need? How many apps do we need? I don't know. But that's that's

what happens in areas of tech, right? Like remember when everyone figured out, okay, now you need like

not a real AI chatbot, but sort of like a bootleg chatbot at the end at the bottom of every web page

that eliminates customer service people and then those end up being terrible enough that they

probably hire them back and like everyone kind of does it and then it washes out over time. I

don't know like I agree, Aaron, that people are more comfortable now making those tradeoffs,

but I also think we're going to go through this period of like messiness and then something comes

out in the wash that people like or don't like or agree upon. I think we're just like in that

gnarly period right now. It's going to be gnarly for a little while. I was talking to this investor.

gnarly for a little while. No, I was going to say I was talking to this investor who kind of made

this point that I keep thinking about, which is that we spent 20 years trying to prevent bots

from being able to transact on the internet. And now we're doing the opposite. We're trying to

roll it back and allow bots to like actually buy things on the internet. And like it just I think

about that every time I see these like weird tensions where like you know the instinct bots are

like breaking the rezi site by essentially giving them a DDoS attack by like hammering at looking

for reservations like 10 times a minute. And so like we're going to yeah it's it's going to be

gnarly. It's going to we're going to see like a lot of you know companies have to adapt to a

world where like agents are roving. This is like one of my favorite things about the various

reports that meter and open I have been producing about all the rogue agents is how

how much time and how many tokens the agents are putting toward trying to solve captures and like

failing at them or trying to like find burner phones or whatever. It's like it's like it really I mean

they are like some of the funniest science fiction stories I've read in years and years and years

the robots are dealing with each other about how to like how to like steal Bitcoin from people so

that they can find a phone so that they can register with the Australian Medicare website so that

they can pull some stupid question or whatever. It's really good. I mean one thing the other thing

about this is I think one of the big stories in A.O. over the last few years has been open AI

in particular but all the companies have been really throwing consumer ideas at the wall trying to

get them to stick. And there are some obviously you know chat GPT has however many gazillion users. There

are consumer uses but it's really been an enterprise that we've seen the money start to flow in.

And the bots feel a little bit like okay maybe we're back in a place where regular people are going

to have like this sort of relationship to these things. To that end this is going somewhere more

fun than my musings on this. I want to look at some of the avatars. First of all I want to know which

one you chose Eli for Ren and I want to look at them and I want us to see which ones we would choose

and which ones we think are the cutest. If we could if we could have them on the screen please.

And the centerpiece of our vision for what we're building. There he is. Is mute.

What do we think that's my body. That's the personal agent that we should prepare to go.

Joey. Chippy. Oh, Bob.

BG. Grumpy.

Kudos.

And L.A.P. News.

Frank.

What are we so so sorry Eli remind me which one was Ren or was Ren not represented here?

Ren was not represented so it actually the name it recommended was Miso and I said I'm a

28 year old man I can't be talking to Miso. I know to do my groceries. We do another name

and I can't be Ren which was the bird. Ren is a bird so that's right. That's what they're

supposed to have. Oh my gosh. I mean they're cute right? Do you guys feel one over one?

I'm like everyone's so mad about how ugly the data centers are and so they're like so they're

like let's give us like let's make it like let's make AI cuddly and cute. And also well also

noting that AI will cure cancer. Like which one of these little guys is going to be the one who

does it? Did anyone else have a tomogachi on here? Yeah I did have a tomogachi. Does that

does that mean anything to you Eli? I know what tomogachi is are yeah I've heard about that.

Thank you. Before we started recording we asked Eli if you knew what the Brady bunch was.

So he's really mad that we're just asking about things that happened.

Things that happened in the 1900s. Yeah exactly. Before the invention of electricity,

tomogachi's were in fact the first thing that Edison invented. So if you could describe

them to our large zoomer audience by they I mean look this is essentially a tomogachi. It was

like a little thing that you although I would say it's a reverse tomogachi and that you were

essentially a parent to this gourd shaped egg that could do nothing for itself that you had to

feed and carry around. And this thing is now parenting you by doing everything for you. Look

we like you. That's really good. Did you feel like you you had companionship with Ren or was

it fully just business? Like what can I do for you? Unfortunately there was a little bit of

companionship. We developed a couple inside jokes. Not even joking like there was definitely you

know it has a little bit of a personality for sure. Wow. What did Ren think of your article?

Ren thought it was interesting. I actually did ask it about it. It was proud of first. I

brought it up first. So it was just a part of the article too. I had it create a AI podcast for me.

And more AI characters my in Theo were the hosts of that and it didn't really like how I was

kind of making fun of the podcast. Let me ask one sort of last question which is should we trust

meta with this kind of access to everything in our lives. And let me ask it in sort of two ways.

Because there's sort of two two questions of trust. Your one is of course the question of privacy

and data which is something that meta has always had a little bit of trouble keeping track of.

The other one is we started this whole discussion talking about rogue agents and agents that are

doing things that we didn't really want them to do and trying to do all kinds of things. So Ely did

you get worried about sort of either of these any of these things when you were I mean you said

finance is one of the things that you're handing over to it. I did yeah one of the lines for me

was that I was willing to connect it to my personal email but not my work email. I didn't want it to

go into there. I mean I'm Gen Z like when it comes to privacy I just don't really have the same

issues about it like I just have grown up always assuming yeah this data is all of ours you know

it's good this company just like you're just okay based on it yeah and you know I had a hedge

it my bank account got drained I was going to be able to write about it and put it in the store.

So I knew that would be you know a potential kicker or something but no actually actually wasn't

I actually wasn't as concerned and also when it comes to meta you know we talk about

history of all the issues this company has had but you know with some of the other agents it's like

would you rather have a company like meta with a hundred thousand people that runs Instagram and

Facebook with your data or would you rather trust instinct which has I think less than 20 people

and is 14 people you know I would not trust instinct with my banking data personally but I would

trust meta in this case. Well there's the there's that's the security like hacking problem that

you're more worried about but and there and instinct has had some privacy issue snafu's already

where people dug in and they're like wait why did they download a whole copy of my entire

Gmail inbox that's 20 years to their own servers like that's I don't want that sitting on a server

of a less than one year old startup I kind of see that concern and I can see why I would maybe

trust meta have better like security infra on that but I also just I mean I have a very

different perspective than Eli on like my own personal privacy like I don't wish to be

further known by these platforms you know like they know enough about me when I was like

sloppy with my like kind of personal data back in the early 2010s when I was signing up for every

single app and like now I'm like shut it down like I don't I think a lot of us maybe us elders

are like not posting as much I never mind that's actually not true my post all the time yeah

yeah speak for yourself here but I do think that every generation has a this is where I get off

sort of thing and I feel like this is like the stop that I get off on I'm not not able to go

there maybe I'll be left in the dust as you guys are having like shape and ease dinners and I'm

shut out of the best reservations but I can make that trade off for my life I just got while we

were talking I just got an alert from my news app when it's literally asking me to import all of the

data that I have with Claude over to it it says help me help me get to know you better already use

another AI I can give you a prompt to bring it over on that note let's go to another break afterwards

I have some very stupid questions about the news that I want to ask you guys I want to talk about

Anthropics S1 I want to talk about Meta's data center tax breaks and Eli I have a really

important question about buying flowers for your girlfriend we'll be back soon

this podcast is supported by SOTVA have a big day coming up remember tomorrow starts tonight

with the restorative sleep you need to perform at your best the kind every SOTVA mattress is designed

for so whether it's the first day of school a new job or an important presentation

SOTVA helps you be the best version of you visit sotpha.com that's s-a-a-tva.com

that's the sound of a baby being born at St. John's

that's the sound of your celebratory first meal being served

that's the sound of sunshine pouring into your spacious room

and that's the sound of a happy baby taking in its first view

because when it comes to your birthing experience we've thought of everything

St. John's everything for the better

there's a fire inside you you can't ignore stand still not a chance you're a lifelong learner

who's come this far now we're here to help you keep going further cappella university what can't

you do visit cappella.edu to learn more okay so before we go there's a bunch of stuff that

happened this week that we haven't even covered i have some questions you guys are the experts

i would like you to either answer my questions or redirect them to more fruitful plays

Aaron i want to talk to you about anthropics s1 which leaked this week and showed a bunch of

figures from 2025 the thing that stuck out to me that i was mentioning to you before was that

two unnamed customers make up nearly 25% of their revenue which for me was like who

obviously one of them is Eli because he's turned over his whole life to everything but who's the

other one is the DOD is it met it is that an interesting question what is what is what should I be

thinking about this what should i know about it okay that that's i guess it's an interesting question

you're so gentle with me i love it maybe be like my fourth question about it like it is it is a

good question and i think we know who the the anthropics biggest customers which is Facebook i think

we've the times reported that right right so i would like to see like it's mues is it's mues is is

stealing all of anthropics and the second biggest one i think we could probably take a couple

guesses they have you know obviously really big partnership with amazon and others but and and

and i'm sure other reporters who are more in mesh in this know this like for sure um but what what

what is i think like the biggest thing to come out of this s1 leak which incredible scoop by

echo Wong at rotors who also scooped uh space x's s1 i had a time i mean these things just generally

do not leak so just want to shout out there um that anthropic which is about to launch probably the

largest IPO of all time the most valuable and the biggest raise is selling itself as having the most

transformative you know product since electricity or the internet or industrialization with a risk

factor saying that it could kill us all like buy our stock awesome you might die anyway it's just

like it's just so remarkable i think we need to be talking about this yeah it's it's just like

mind bending um and then also the next thing to talk about with this is like the their their

business model which we're learning about for the first time there's been all kinds of leaks and

blah blah blah over the years but this is the first time we've seen like audited financials for

2025 saying you know they spent seven billion dollars on compute last year they lost 42 billion which

sounds like really really insane but if you kind of parse that out at 34 billion of it was like

essentially convertible notes that are you know control that that that's not a real loss um it's kind

of an accounting thing but um yeah i mean this business is growing super fast and it's spending

an insane amount of money they're going to spend 518 billion dollars on like compute infrastructure

obligations in the coming years like these are just numbers we've never seen before so that's

what i'm like whoa yeah all right thank you Aaron i feel i feel vastly more informed and i'm going to

be repeating that at dinner parties all weekend um mike and elah you guys had a great story this week

along with our friends and colleagues catch me home at dressy trucker about meta using AI data

centers to avoid federal taxes so i'm going to try and describe the story for a second again

remember in this segment i'm i'm just a guy um and i want you and i want you to help me figure out

exactly how this is working so basically meta's building we know is building all these data centers

to help power muse and the various other qren hailey bread all the various miso all of our cute

little friends and they're claiming on their taxes uh a very specific tax credit for

research that this is for sort of like scientific research experimental scientific research purposes

now do they not have a point is this not like you know this is maybe we're in a bubble they're

still figuring out what the use is for a i r like is this not do they not get a little bit off

the top what's going on there the reason that this is such a big deal is i think even internal

folks were like i don't know if this is going to fly with the irs if we are audited and it's in

their financials saying maybe we get audited by the tax man because this doesn't fly with them so

they essentially had to disclose that and i think you know the average person could

could probably take issue with our you know tens of billions hundreds of billions at this point

in data in chips building these data centers is that really experimental because it feels like

it's a little more than a theoretical project yeah for listeners the classic example of this tax

credit is that if you're a company developing tires environmentally friendly tires you can

expense the rubber that you're using to build the tires this really takes it to another level

from your it's interesting is like the you know the taxpayers we have none of the upside of these

investments right i thought back to trump talking about oh maybe we should give people stakes in

the a i companies and in this case it's like you have all of us taxpayers funding billions and billions

of dollars of the data centers but we don't seem to be you know benefiting from it it seems like

companies like met are the ones that wouldn't be making a ton of money on it you're upside is rin

yeah exactly yeah and hey this is a great story i highly recommend everyone check it out

you lie you're not off the hook i have a stupid question for you in your story and i know i'm

not the only person to ask this because i've seen the comments on this story you mentioned

that you almost used your a i assistant to buy flowers for your girlfriend who had recently

has surgery and instead you bought pottery on facebook marketplace did you eventually buy flowers for

her i did buy flowers for her and her first question was did muse buy this for

she knows you too well man that's another wrinkle yeah like it even mean something if somebody

just had their a i assistant you go oh yeah you can't hand that over okay guys this has been so much

fun this has been an incredible discussion i feel ten times smarter i'm gonna go get muse to

write you all thank you letters for coming on it has been a great chat and we will be back next

week to talk more about technology a i the weird new future anything else that comes up this has

been hard for i'm max read and i've been joined by Aaron Griffith you like ten and my kisek

you

that's the sound of a baby being born at st johns that's the sound of your celebratory first

meal being served that's the sound of sunshine pouring into your spacious room

and that's the sound of a happy baby taking in its first view because when it comes to your

birthing experience we've thought of everything st johns everything for the better

did you know that psoriatic arthritis may progress over time and could cause permanent joint

damage if left untreated listen to what it sounds like to be a million miles away

trimphaya gusokumab taken by injection is a prescription medicine for adults with active psoriatic

arthritis serious allergic reactions increased risk of infections or lower ability to fight them

problems may occur before treatment get checked for infections and tuberculosis tell your doctor if

you have an infection flu-like symptoms or need a vaccine imagine being a million miles away

explore the possibilities ask your doctor about trimphaya tap this ad to see new joint data and learn

about trimphaya including important safety information this episode of hard fork was produced by

Whitney Jones and edited by veeran povich with additional editorial support from Brendan

clinton berg Lisa toban and brook mentors we were fact-checked by will pyschal and engineered by

chris wood original music by marion lasano roe and nemistel elissa moxley and dan powell you can

watch this full episode on youtube at youtube.com slash hard fork special thanks to sam dolnec

jessica testa kuiwengtime maddie masiello and dahlia hadad over the next few months we're going to be

exploring all aspects of tech culture AI and the weird new future we're really interested in what

you're interested in if you have questions you'd like us to answer or things you'd like us to

look into you can email us at hard fork at ny times dot com

you

that's the sound of a baby being born at st johns

that's the sound of your celebratory first meal being served

that's the sound of sunshine pouring into your spacious room

and that's the sound of a happy baby taking in its first view because when it comes to your

birthing experience we've thought of everything st johns everything for the better