The Bitter Rivalries Behind the Race to Build Superintelligence

Today, understanding the people who are building super intelligence, I want you to think

back to 2017.

In some ways, it probably just feels like yesterday.

Nine years ago, your neighborhood probably looked exactly the same.

There were cars, coffee shops, phone in your pocket.

But in the history of artificial intelligence, 2017 is the distant past.

That GPT doesn't exist andthropic doesn't exist.

The mega project of AI construction that's currently driving American economic growth, it

was on nobody's horizon.

But the following things all happened in 2017.

Number one, open AI, the nonprofit co-founded by Sam Altman and Elon Musk, started work

on GPT-1.

Five years later, this technology would serve as the guts of chat GPT.

Also in 2017, a physicist at open AI, named Dario Amade, wrote a secret 26 page internal

memo entitled Big Blob of Compute, in which he made the argument that AI research should

resemble a massive infrastructure project, where bigger models with more data using more

chips would be the path to super intelligence.

Nine years later, this thesis has proved eerily accurate as the entire AI build out has

been motivated by the idea that AI's capabilities scale with compute spending.

And number three, also in 2017, AlphaGo, an AI model, went undefeated against the world's

top ranked Go player.

The model came from the company Google DeepMind, run by Demis Hesapis.

Three years later, that company would use similar technology to solve the famous protein

folding problem, and achievement that many scientists considered impossible.

For this, Hesapis would win the Nobel Prize.

Nine years later, it is not a stretch to argue that the entire frontier of AI has been

shaped by these three men.

Sam Altman, the CEO of OpenAI, Dario Amade, now the CEO of Anthropic, and Demis Hesapis,

now Chief Scientist and Alphabet.

But who are these men?

Who are Sam, Dario, and Demis?

Well sometimes read or hear coverage of the space that makes it sound like the men building

advanced AI are all the same.

They all want the same things.

They are powered by the same greedy motivation.

But these men are not the same.

And today's episode is about the hidden motivations and rivalries among these three men that have

arguably brought us to the precipice of superintelligence.

What books and ideas inspired them when they were young?

What do they fear?

What do they really want?

My guest is Kevin Rus.

The author of a new book, The AGI Chronicles, the inside story of the race to create an artificial

superintelligence.

This is a story of bitter rivalries and secret documents.

And as technology moving faster than the people building it can understand, let alone control.

I'm Derek Thompson.

This is Planning.

This episode is brought to you by Ethos.

One of the less glamorous parts of building a life is thinking about what happens if you're

not around for it.

But if people depend on your income, life insurance can help protect the life you've built

together.

Ethos makes the process straightforward and entirely online.

Get a quote in seconds, apply in minutes, and potentially get covered the same day with

no medical exam.

Take 10 minutes to get covered today with life insurance through ethos.

Get your free quote at ethos.com slash English.

That's E T H O S dot com slash English application times may vary, rates may vary.

This episode of Plain English is presented by Comcast Business.

Did you know Comcast Business detected 79.3 billion cyber threats last year?

That's why they're introducing security edge preferred.

It's big business, cyber security designed for small business.

It started with Comcast Business Internet for $50 a month for one year on a qualifying

internet package and find out how to get one month of security edge preferred on them.

Plus, ask about a $500 prepaid card.

Limited time offer, restriction supply requires 300 megabits per second internet security

edge security edge preferred and wireless connect call today.

Kevin Rousse, welcome back to the show.

Thank you for having me back.

I was a pleasure.

Kevin, what is super intelligence?

And why do you think we're on the precipice of building it?

Super intelligence, you could think of as an AI system that is smarter than everyone

at everything.

There are sort of these gradations of AI capabilities.

People talk about AGI artificial general intelligence as a computer program that is human

level or better at most tasks or most intellectual tasks.

Super intelligence would be the level beyond that where we have something that is vastly

smarter than us at everything that we could conceivably do.

Your new book is called The AGI Chronicles, the Inside Story of the Race to Create an

Artificial Super Intelligence.

It seems from your reporting that you think this is a race that is on the precipice of

achieving something quite dramatic in the near future.

What have you seen?

Let's just say in the last 12 months that have made you think we were really making progress

toward this goal as you defined it.

Well, let's back up more than 12 months to 10 years ago when the book starts, which is

we had these AI systems that could do things like identifying objects in images or translating

between languages or helping you with auto correct on your email.

But those were narrow systems.

They could only do one thing.

They couldn't be repurposed to do 20 different tasks.

Now we have these general models, Claude, ChatGPT, Gemini that can do a wide range of things

near or above human level.

We have systems that are capable of solving millennium prize problems in math, of discovering

new protein structures, of helping people, billions of people around the world every day

in various ways.

I think that's what I've seen is just the continued march of this technology.

And then the sort of, the non-public thing that I've seen is that the people at the companies

who are building this stuff, I guess this is not a secret anymore, are terrified of it.

They have been for years.

And they're not getting any calmer as we approach what they think could be some kind of intelligence

explosion, which we could talk about.

That's sort of another concept that they think is not far off and I'm inclined to believe

them.

One thing I think is so strange about this entire project is how small this world is.

And when we talk about the frontier labs, we were talking about three companies leading

the race to build super intelligence.

They are open AI and theropic and Google DeepMind.

Those companies are run by or have until very recently been run by three particular and

peculiar dudes, respectively Sam Altman, Dario Amade and Demis Hisabis.

I want to spend the bulk of our time trying to understand these men, who they are, what

motivates them, their strengths, their weaknesses.

And let's just jump right into it.

Sam Altman, Paul Graham, the former head of Y.

Combinator once famously wrote this about Sam Altman, quote, you could parachute him into

an island full of cannibals and come back in five years and he'd be the king.

End quote.

What makes Sam Altman so good at becoming powerful?

He has this superpower, which everyone he's ever worked with, all of his mentors, they

have recognized this in him.

And he had it from quite a young age of figuring out exactly what someone wants and giving

it to them or at least appearing to give it to them.

So back in his days running Y.

Combinator, people would sort of talk about this power of his and they didn't know where

it came from.

But people like Paul Graham recognized it very early.

He was very good at reading the room, at figuring out, you know, this person expects me to do

this or wants me to do this.

So I will do that.

And it led to him becoming one of the most successful young entrepreneurs and investors

in the valley.

The flip side of it is that he would often get into trouble because he would tell one person

one thing that they wanted to hear and then tell a different person something completely

different that they wanted to hear.

And he sort of got this reputation as a sort of slippery people pleaser, which is I think

a criticism that many people who have worked with him over the years have identified and

have made in pretty consistent ways.

He just seems to be in the business of both being quite ambitious, but also of being quite

eager to please.

I mean, one way to say this is that he is a brilliant deal maker in the mold of a JP

Morgan.

And I wonder, after all of your reporting, whether you feel like underneath all of that

deal making, there is a core philosophy.

Like just to put my thumb on one distinction that might help you answer that question, there's

a 2015 blog post that Sam Altman wrote where he called machine intelligence, quote, something

we should be afraid of.

And when he found open AI with Elon Musk, he found it as a nonprofit because he thinks

that this is a project that should not be a typical commercial business.

But now he's running a business that is by some accounts valued at $800,900 billion,

a trillion dollars.

And later colleagues who work with him are very skeptical of degree to which he's serious

about AI safety at all.

So one way to answer this question is to say, does he believe in AI safety?

What I'm really asking here is does he believe in anything outside of just making the deal

that increases his power?

I believe he has a core.

I don't know what it is.

I'm not in his inner circle of people that he confides.

And I've spent time with him.

I've talked to him.

I've had encounters with him as a journalist.

But I've always sort of understood that he was playing the part in those interactions

and wasn't necessarily showing me his true self and for what it's worth.

I think most executives do this.

They have their persona that they put on.

I think the thing that kept coming up in reporting on Sam and the people around him

is that he is a person who believes first and foremost in agency.

But he has this famous phrase, you can just do things.

And this is something that he believed at Y Combinator.

He would tell these young startup founders that he was investing in.

You don't have to wait to change the world.

You don't have to like, nobody has to give you permission.

You can just go do things.

And that is a core part of his belief about himself and about those around him.

Like, you get another way of saying it is like there's no grown-ups.

Like you are the grown-ups.

And you can just do things that change the world without permission.

He's also a fervent believer in scale, the power of scale.

People would tell me these stories about he would walk into a meeting.

And he would just tell the people in the meeting to add a zero or two zeroes

to whatever budget they had for their project going forward.

Or increase the scale of a training run just because he thought that bigger was better.

So I think that love of scale is probably part of his experience running

my combinator and being an investor before open AI.

But certainly open AI's success has confirmed that just having this kind of

delusional self-belief and belief in the power of scaling is actually what you need

to win an AI.

Because this is something that didn't really occur to me until midway through the book.

Like, belief is an active ingredient in AI success.

You have to believe that the scaling laws will continue.

That putting billions or trillions of dollars into building data centers and acquire and compute

will pay off.

Like, you have to, like, the belief is the enabling force that allows you to win.

And if you don't have it, no amount of technical expertise can save you.

Your book has several examples of this delusional self-belief that

Sam Altman demonstrates.

And one example is as open AI is scaling their GPT technology.

And they've just come out with GPT2.

But they realize that they're going to lose the money spigot from Elon Musk.

And so they need to get a billion dollars from someone.

And a billion dollars at that point in the AI super cycle is a lot of money.

And so they decide to put on a little demo for Microsoft CEO,

Soctin Adela with GPT2 to rest a billion dollars from Microsoft.

Can you tell this story and what this story represents about Sam and the company?

Yeah, it's an incredible story.

And it hadn't been reported before.

And I was sort of delighted to find it out because it does really illustrate

who these people are.

So they go up to Redmond for the day from San Francisco to Sam and a few other open AI executives.

And they've got this very high pressure meeting with Soctin Adela and his leadership team at

Microsoft.

And they've got this new model, GPT2, that is still, it's not good by today's standards.

You would never use it for anything that you needed done.

But it can sort of complete a sentence.

It can sort of generate coherent text.

And it can do all of this more flexibly than any of the chat bots that had come before it.

So they go up, they run this demo.

They bring a Microsoft laptop borrowed from one of the open AI executives,

girlfriends, because they're like Microsoft will think more favorably of us if we're using Windows.

So they borrow Windows laptop.

They go up, they run the demo.

And they're showing some sort of canned prompts that they've prepared that they kind of know GPT2

will be able to do.

And then Soctin Adela asks, can we see this thing?

And another Microsoft executive sort of wants to try their own prompt

that wasn't part of the canned demos.

And so they're like, they're sort of looking around each other.

They're like, can we do this?

Can we do this?

Yeah, I guess we have to do this.

So they put in a prompt that's like, Soctin Adela was hired as a CEO of

Microsoft. And the first thing he did was blank.

And then they hit enter and they're sort of like sweating as they wait for this thing to come back.

And then it spits out the completion, which is that on his first day,

Soctin Adela fired all the employees and pivoted the company to Azure, their cloud platform.

And this draws a big laugh.

And it's sort of an amazing moment in time.

Like an AI model has never been able to do this before.

And it was very high stakes.

Like had this sentence completion not worked, they might not have gotten the billion dollars

from Microsoft. But as it turned out, that was exactly what they wanted to see.

And they invested a billion dollars in the rest of history.

It's an amazing story. And one other piece that I think is really important

is that Bill Gates at the time was quite biased against AI chatbots.

Because Microsoft had a terrible experience with a chatbot named Tay or something.

Do you remember Tay? It was like a glorious and...

I just said, no, you're the chatbot nerd here.

You've had all of the deep intimate experiences with chatbots.

I just, I have a thing for very chatbots.

And Tay was one of the more deranged ones we've had.

It was this experimental chatbot that Microsoft built pre-LLM.

And they put it on the on Twitter.

And people just started messing with it.

And turned it into essentially a white supremacist within 12 hours.

It was spewing Nazi propaganda.

And the lesson that Bill Gates and other Microsoft executives learned from that

is don't make chatbots. That's a bad idea.

I think that's a really important ingredient in what makes a story so dramatic.

Because Microsoft is not beginning from a neutral place of maybe chatbots are good,

maybe chatbots are bad. We have no experience.

Like, they're like, we've done chatbots. They're terrible.

Yes.

Who knows? Maybe if we invest $1 billion into this company,

our chatbot might engage with the New York Times reporter

and proposition him for marriage. Thank God, something...

That would never happen to them.

Never happened.

I'm a huge sucker for like understanding

the internal experience of like hinge points in business history.

And whether you're a listener who actually despises AI or loves AI,

like the release of chatbots absolutely changed business, technological and economic history.

It was so interesting to read your reporting on just what an FN nothing burger,

the folks at OpenAI thought chatbots would be.

At one point, they're taking bets on how many people are going to sign up for this chatbot.

Most people think 10,000, the highest bet at the company is Sam Altman saying,

no more than one million.

I mean, we're talking about a technology now that has a billion,

a billion users around the world or more.

Can you flesh out the details of why OpenAI launched chatbots?

Why ironically it started with a rumor about a rival company launching their own chatbot

and just what happened behind the scenes that gave birth to chatchbt itself in November of 2022?

Yeah, it's an incredible story because it truly was an accidental hit.

The people at OpenAI, as you said, many of them didn't think this thing would be very popular.

And I think to explain why it's like we had to sort of step back in the time machine to like the

sort of summer and fall of 2022, at the time, GPT-4 was just sort of finishing being trained.

This was the most capable model. It would not be released until the spring of 2023,

but OpenAI employees already had seen versions of it and were playing around with it.

And they were all racing to get this thing ready for a launch in early 2023.

And then in November 2022, so a couple months before the planned launch of GPT-4,

this OpenAI co-founder, John Schulman, here's this rumor at a party where he hears that

Anthropic, which is a splinter company that's sort of formed out of a group of dissatisfied OpenAI

employees, is going to release Clawd, their chatbot. Now, the rumor is not true.

Like they were, they had no plans to release Clawd as a commercial chatbot. They were sort of

testing it in a private Slack channel with friends and family. But this gets sort of passed back

to Sam Altman and the folks at OpenAI. And they're like, shit. Like we have this plan to release

GPT-4 in a few months, but we think that this chatbot Clawd might be really good and might kind of

upstage us. And we don't want that to happen. So John Schulman proposes this sort of rushed launch.

Of a thing that was originally called chat with GPT 3.5, which was basically to take their old

model that had already been out for like six months for developers and put kind of a free easy

to use chatbot interface on it and just throw up in the doors as what they called a low-key

research preview. Basically, it was not their best model internally. And I think that's why a lot

of people at OpenAI had low expectations. Because they're like, this thing sucks. It's like

six months old. We already have this better model that has finished training that we've been

using ourselves. That's a lot better. Why would we like pump the brakes, throw a new skin on this

old model and like put that out as if it's supposed to be impressive. But they did it anyway. They

decided to make it this low-key research preview. They had low expectations and then it just caught

fire. They had like badly misjudged how impressive the things that they were using internally

would look to the outside world. Really, really quick question. Do you think OpenAI would have

inevitably released a chatbot with GPT 4.0, GPT 5.0 eventually, or was there something about

that dynamic of believing that clawed would become public that forced them to take their GPT

technology and turn it into a chatbot when they had been thinking about an entirely new or

different product or business model? They were fully planning to release GPT 4 as a chatbot.

Actually, they were planning with Microsoft, which had this exclusive partnership with OpenAI,

had plans to build it into their products and make chatbots as well. It was going to happen

in the spring of 2023. But the launch of chat GPT brought that up in the timeline to the fall of 2022.

Good. It's good to know that. Last question about Sam Altman. Tell us why he was fired and how his

return of the company changed the culture of the company. Because we don't have time for you to

explain all of the reasons why he was fired. Why don't you just shade your answer toward the most

salacious of those reasons? You want the dirt? Well, he was fired because the non-profit

board that governed OpenAI had been configured in a way that gave the more sort of safety

conscious members of the board a temporary majority. And these four members of the board were also

getting worried about how honest Sam was. This is again a pattern that has been observed by people

close to Sam for many years. In some ways, it was exactly the reason that Dario Amade and the

anthropocopounders had quit in late 2020. They just sort of saw Sam lying and observed that he

would say things to different people that were contradictory. And so Ilya Satskiver, who was an

OpenAI co-founder, chief former chief scientist also on the board, starts assembling this rap sheet

Sam Altman and Greg Brockman, the president of OpenAI, who was also on the board at the time.

And he and Miram Aradi, who was then OpenAI's chief technology officer,

start secretly compiling this dossier about all of the times that Sam and Greg would say something

to one person and a different thing to another person or would undermine executives or pit them

against each other. Just bad management practices. But displaying that this was a consistent pattern,

they send it to a member of the nonprofit board and they basically plot this coup that they're

expecting will be greeted with a hero's welcome. They're like, oh, if we're seeing this stuff

about Sam, then everyone else must be seeing it too. And we will be greeted as liberators,

essentially. And they carry out this thing on a random Friday in November. And they just are

sort of amazed and pulled over by how much that does not come to pass. The employees rally behind

Sam and Greg, they demand their reinstatement, they threaten to quit and go to Microsoft. And there's

a lot of good, gory detail in the book about how that all went down. But essentially, Sam engineers

his own comeback with the help of open AI's employee base. And five days after he's fired,

he's reinstated, you know, people have heard this story a zillion times. But what's new and what I

was interested in are the sort of dynamics of this employee led resistance. And how and why they

decided to band together to sort of restore the position of this guy that I think a lot of them

had misgivings about, even at the sort of rank and file level. So it really is if what like one of

the most fascinating five day periods in business history, I think. And so I devoted two chapters

five days because I was just like, it's not going to get any juicier than this. Yeah, a lot of

chicken nuggets ordered a lot of boba teas consumed. And you even have stuff about how like the

image of the chicken nugget and boba tea became like the hammer and sickle emblem of like

open AI's. Yeah, no, like the war medals of people who survived this. It's a really,

really colorful and strange story. That closes this sort of mini chapter on on Sam Altman, who

I think it's important to say emerges from this deal making tradition at Y Combinator,

he is a deal maker. And that is both his strength and his weakness. It's his strength in terms of

being able to secure money, secure compute, which they are lapping the field in. That is the

the amount of chips that they have to train and run inference on their AI. But that's also his

weakness in a way. It was almost his downfall because he's so much of a deal maker without a clear

moral or philosophical center that people feel sometimes that it doesn't exist in their form

is trusting for it. And in a way that's very, very different from our second character, Dario Amade.

Dario now the CEO of Anthropic, but up until I mean in the first few years of open AI, he was

there, their head of research. Before we get into his own personal history, I want to talk about

his motivations. You report that one of Dario Amade's favorite books is Richard Rhodes,

Pulitzer Prize winning history, the making of the atomic bomb. And if you've read that book and

I highly encourage people to read that book. Really just read the prologue. The hero of both the

prologue and the entire book is Leo Seilard. Leo Seilard, I don't know how to exactly pronounce

the Hungarian name. It's Seilard, but it's fine. Seilard, okay. It's it's SCIL ARD. It's called

Leo. Who is Leo? And what does it tell us about Amade that he sees himself in Seilard's mode?

So Leo Seilard was a Hungarian physicist and he has this discovery, this breakthrough. He's the

person who came up with the nuclear chain reaction, the thing that paved the way for the splitting of

the atom and eventually to the atomic bomb. And he has this realization, there's this very dramatic

scene where he sort of has this realization when he's waiting to cross the street in Richard

Rhodes book. And then once he has this realization and proves it out, he tries to keep it a secret.

He goes around to all of the sense telegrams, all the scientific labs, all the nuclear,

the sort of ish labs in Europe begging them not to look into this or not to publish their findings

because he is terrified that the Nazis will discover this secret and use it to build bombs and

use it to win the war. And he tries to classify it as a secret with the British Admiralty. He tries

to patent it to sort of keep it out of the public view. And he fails at that. The secret gets out.

And after the secret gets out, he has this moment where he realizes that the only moral thing to do

now that the secret is out is to build it before the Nazis do. So he, there's this sort of amazing

sequence in Richard Rhodes book where he basically petitions the president with the help of his friend

Albert Einstein to start the American nuclear effort that becomes the Manhattan Project. He then

signs up for the Manhattan Project. And basically his theory is if we don't build it before the

Nazis do, the world is going to end or we're all going to be living under Nazi rule. And Dariel

Amadei not only read this book, it's his favorite book. He bought copies for early Anthropic employees,

like on your first day at Anthropic for years, you would get handed your onboarding packet and

a copy of the making of the atomic bomb, which is like 1300 pages, by the way. This is not

this is not like reading, not like reading. And he would, he gave out a Sillard award at Open AI to

like people who would make sacrifices on behalf of safety. So like really idolizes this guy and

has patterned his career after Sillards. And you know, the obvious parallel here is that Dariel

Amadei discovered a secret. He was the essentially discoverer of what became known as the scaling

laws, this method of improving AI performance by just throwing more compute and data and parameters

at a model. And he tries to keep it a secret. He like, you know, really, really tries hard not to

let anyone else know about this discovery. And then once it gets out, he decides that he has to

build a GI or super intelligence first before, you know, not the Nazis, but, you know, China or

Sam Altman or someone else builds it before him. So this is the sort of the the matter narrative

of his career is that he sees himself as essentially a modern day, Leo Sillard working to build

a powerful technology in a safe way so that others don't build it in an unsafe way first.

Yeah, I don't think you have to necessarily accept the entire argument for

anthropics existence to at least appreciate the sort of internal logic of the sentiment.

I'm building something that could be the most dangerous thing in the world, but I have to build it

as fast as possible because it's important for us to have the thing first. And like that internal

logic does not make sense for most products, but it made sense for Einstein and Sillard with the

atomic bomb. It made sense to Oppenheimer to a certain extent with the atomic bomb and everyone

at Los Alamos. And it makes sense to people working at anthropic into a certain extent also open AI.

That's just a part of the weird self-justification that motivates a lot of activity in the Bay

area that I think in many ways seems completely alien and sometimes downright immoral to people

outside the Bay area. But I think that Sillard is an important skeleton key in terms of understanding

at least how these people think of their own project. You can think they're wrong, but it's still

an interesting skeleton key to understand their motivations. Totally. Amade love to write essays.

He's written a bunch of essays and anthropic circulates all the essays, well the journalists.

One essay that I had not read in a while is one you write about in your book is 2017

26-page internal document entitled Big Blob of Compute. You were referencing this in your

previous answer. And he argues essentially that AI researchers overthink things. They try to

sort of finely tune all their models, but at the end of the day progress comes from this simple

recipe of make the models bigger, get more data, get more compute and scale, scale, scale.

Can you explain just how important this paper has been in terms of pointing to the future of

and almost accepting the future of the AI super cycle? Because reminding myself of the paper in

your book, I was like, it's a little bit bay breuth pointing to the left field stands and then

hitting the whole run. Like he kind of just like says, we're going to spend trillions of dollars

on this thing. And then that's basically exactly what's happened. So tell me Big Blob of compute.

What was Amade's thesis in 2017, nine years ago? And how has it come to pass?

Well, I hope you have not read Big Blob of compute yet because I think this is the first time it's

been published anywhere. This is not a document that Amade has published or even really talked much

about publicly. So this is something an internal research memo that I obtained that I'm going to

publish the whole thing on my sub stack this week, I think, because I just think it's an important

historical document. For exactly the reason that you just said was like he points to left field,

he says, this is how it's going to go. And then it essentially goes exactly like he predicted. So

this was a paper that predates the LLM, GPT-1 had not been trained yet. But it is essentially the

scaling hypothesis in miniature. He says, essentially, you're all doing this wrong by you all means

like other AI researchers. He says, you're building these small, narrow, single-purpose model to do

discrete tasks. You're building language classifiers. You're building image recognition systems.

You're building speech recognition systems. And you are you're training them at a much smaller

scale than would be optimal. You're not putting enough compute or enough data into them. And you are

engineering these kind of narrow solutions to your problems. When the real way to create better

AI models is to just make them bigger. Like this is a principle from nature. Animals with bigger

brains, all things else being equal are smarter than animals with small brains. So if you want to

improve the generality and the performance of your systems, just create this mass of unstructured

computational capacity, this big blob of compute. And he talks about these seven sort of variables,

these knobs that you can turn, like how much data you put in and how much you know how much

how much a pressure what goal you give the system. And then you essentially let the intelligence

fall out of the model rather than trying to like code it in by hand. This was a I think this is

the most important document, maybe ever in AI research because it it it it came at a time where

open AI was really struggling. They were trying to do these sort of narrow things. They had a

robotics team that was trying to build a robot hand that could solve a Rubik's cube. They were

trying to teach AI's to play video games. But none of it was generalizing very well. Like they

would have one success in one little area. And then they would like it would it just wouldn't matter.

And big blob of compute really steered them toward language models because language models were

a thing that you could put a big blob of compute behind that would get better the bigger you made

the blob. And that was the key insight that led them to sort of bet to go all in on scaling up

language models. And it's why we have the trillion dollar you know build out that we have today because

they realized and they proved that the bigger your blob of compute was the better your model was.

And there's another paper published in the exact same year that it essentially hooks up with,

right? Attention is all you need because you get both big blob of compute saying we need to secure

as much possible computing power as possible. Combined with attention is all you need a paper that comes

out of Google that essentially says the transformer could be an incredibly useful generalizable model

to do this thing that all these smaller models can't do. And essentially is when those streams collide,

you get this explosion of this modern super cycle of of AI building. Is that fair to say that like these

two twenty seventeen papers sort of collided in hyperspace and created like the modern AI moment?

Absolutely. I mean you it wasn't enough to just have the big blob of compute. You also needed

an architecture that could accept a big blob of compute. You know there were all kinds of language

models before twenty seventeen but they didn't work that well. In part because they just like you couldn't

scale them in the same way that you could scale this new kind of language model that Google had developed

that the transformer, which was basically a way to radically increase the amount of compute and

training that you could do on a large set of text. So you needed both. You needed the the piece

from open AI was the belief in scaling and the big blob of compute and the piece from Google,

which they didn't even recognize at the time because they didn't have the belief in scaling.

They just thought, oh this is a cool thing to a cool way to make our language models better at like

translating and you know writing emails. It's not going to like they didn't think this would be

a foundational technology, which is why they released this paper publicly. So they sort of drop

this thing in open AI's lap, the transformer, Dario sort of sees this and runs with it along with

some other researchers at open AI at the time and outcomes the AI revolution.

This episode is brought to you by JP Morgan Chase. In today's unpredictable world,

geopolitics isn't just a backdrop. It's shaping markets, business strategy and risk. The JP Morgan

Chase Center for Geopolitics brings together former senior policy makers and season practitioners

to translate fast moving global events into clear decision-ready insights from conflict and great

power competition to energy and supply chain disruption. The center helps clients anticipate what

comes next. Read the latest insights at JP Morgan Chase.com slash center for geopolitics.

This episode is brought to you by Lincoln. What if your vehicle could help give you a reset?

Lincoln Rejuvenate is designed to do just that. This multi-sensory feature combines expansive

visuals, digital sense and a seat massage for the driver and is available in the 2027 Navigator SUV

visit linkin.com to learn more. Lincoln Rejuvenate does not operate unless the vehicle is in park when

the vehicle is off or if any door is open. Do not use the feature in a closed garage or in other

enclosed areas. See owner's manual for important operating instructions. This episode is brought

to you by Lincoln. What if your vehicle could help give you a reset? Lincoln Rejuvenate is designed

to do just that. This multi-sensory feature combines expansive visuals, digital sense and a seat

massage for the driver and is available in the 2027 Navigator SUV visit linkin.com to learn more.

Lincoln Rejuvenate does not operate unless the vehicle is in park when the vehicle is off or

is any door is open. Do not use the feature in a closed garage or in other enclosed areas. See

owner's manual for important operating instructions. In 2019, OpenAI releases GPT-2 and at this point

Dario Amadea is still the VP of research at OpenAI. The release of GPT-2 does a few things.

One, it proves Amadea right. They use something like a thousand times more compute than with GPT-1

and they can see that the model is generalizing. It's getting smarter across a bunch of different

domains including those it's not specifically trained on. I guess that is generalizability.

But the release of the model also really scares Dario Amadea. Why?

He is worried that a text completion engine that is good at spitting out plausible

generated text could be used for bad ends. It could be seized by propagandists or foreign

disinformation campaigns. It could be used to pump out misinformation and fake news on an industrial

scale. He knows that GPT-2 is not going to take over the world. It's just a text completion engine.

But he is worried about how bad actors might use it to do unscriptuous things.

And there's a big debate inside OpenAI about what to do with this. It helps to know, I think,

that Dario Amadea is a sort of neurotic by temperament. But he is also part of this community.

AI safety warriors in the Bay Area. At one point in 2016, he's living in a grouphouse

with Paul Cristiano and Chris Ola and Jared Kaplan and all these now leading lights of AI and AI

safety. They are all just sitting around at night, reading posts from less wrong and other AI

safety focused publications and talking about them and debating what the existential threats from

AI might be. And this was considered insane behavior in 2016 because nothing actually worked

yet. But they were sort of peering into the future and imagining when systems did work and could

be improved through scaling what that would mean for safety. Dario Amadea obviously no longer

works for OpenAI. He's the CEO of Anthropic. You talk about all of these different rivalries

that have been created in his final months at the company. It's based that it's a three front war.

It's Dario versus Ilya Satskever. It's Dario versus Greg Brockman. The president. It's Dario versus

Sam Altman. What can you tell us again, shading towards salaciousness here in your brief answer,

to help us understand why Dario Amadea felt like he had to leave OpenAI and start a new company.

So he had been thinking about this for years. I think this was not a rash decision and he had actually

sort of thought very seriously about quitting before. Mostly because he thought that Sam Altman was

dishonest. He had this sort of running list of times where he thought he had caught Sam lying to him

or saying something that wasn't exactly true or just doing things that seemed like bad judgment.

This example that I just love that I was so happy to be able to reveal in the book because it

was very salacious and very funny was that apparently Sam Altman met Kanye West at a party.

Off-handedly mentioned that they were doing this private beta test for GPT-3 and offered to let him in.

Sam likes talking with celebrities and having celebrity friends. You can see why he would want to

show off this new model that he built to this very famous person. And then he brings this back to OpenAI

and they're like, what the fuck man? Kanye is not well. He had just posted a video of himself

like peeing on a Grammy Award. He was not yet talking about Hitler but things were not going

well for him mentally. And here Sam is offering to let him into this product that they think could

be quite dangerous in the wrong hands. And so they sort of put a stop to this. There are like,

you know, dozens of incidents like this. And the way it was explained to me is that none of them on

their own would have been sort of worth quitting over. But together the sort of snowball effect

meant that one day in 2019 he essentially just sort of decided he couldn't do it anymore. He was not

going to be successful at steering OpenAI towards safety from the inside. He wanted to leave and

bring his co-founders with him and start something new. I think one question that I've had about

anthropic is how did they go from being a company that splintered off from OpenAI for the purpose

of protecting this technology from the forces of commercialization. Only to become in just a few years

even more commercially successful than OpenAI. I mean, how can you not even defend just describe

why Dario and the other leaders and anthropic went from essentially thinking

andthropic should never release Claude as a consumer product to releasing Claude as a consumer

product that makes tens of billions of dollars for anthropic a year.

Yeah, it's the central question of anthropic. How did this

small, I mean, when I first visited anthropic in 2023, it was a couple years after it's founding,

but they had not yet released like a good chatbot that people were actually using. And it was like

150 people and they were all cooped up in this office and they had sort of no business strategy.

It was just kind of like a research lab, but chatGBT had come out and they had sort of been

convinced, apart by Dario, that they needed to compete. Like if they wanted to be relevant on safety,

they had to have a model that was as good as any on the market. And I think part of that was

seen internally as kind of like Dario just kind of justifying something that he wanted to do anyway

because he hated Sam Altman so much and hoping AI so much and couldn't bear the thought of

losing to them, that he sort of concocted this rationale of he calls it the race to the top. Basically,

if you are in the lead or close to the lead, you get to sort of lead by example and everyone else has

to follow you when it comes to safety and making your models more aligned. So this becomes his sort of

his bet. But you're right. At the start, they had no intention of making money. I talked to lots of

early anthropic employees who were like, yeah, it was not even a subject that came up. We thought

we were going to build safe AI systems and do safety research on other people's models. They had

this list of crazy ideas for how they could make money, including having cloths start a hedge fund

and give them trading tips, didn't end up working out. But they eventually decide, well, two things

happen. One is, AGI turns out to be more expensive than they had bargained for. So in this founding

document that Dario writes in 2021, it's called Vision for a New Co, he predicts that anthropic would

need about $10 billion to build a full-fledged AGI. And as we know now, as we now know, $10 billion

is a vast under account. You need like hundreds of billions of dollars, possibly trillions of

dollars to build something that will fulfill the original vision of AGI. So they need more money.

And they have raised a bunch of money, but they are going to need more. And so they have to have

something to sell to raise that money. So that's part of it. There's a part of the story that you

absolutely have to include, which is that their top benefactor is a little man named Sam

Beckman-Freed. He is the initial investor who buys, I think, owns 13% of the company's equity at

one point. Yes. So Sam Beckman-Freed. And so once Sam Beckman-Freed is in jail, then not only do

you need a lot of money, you also no longer have access to the person who owns 13% of your equity.

So you need to go further and you can't rely on EA billionaires. And so you put that together and

then you say, okay, big blob of compute. We need hundreds of billions or tens of billions of dollars.

We can't rely on philanthropy. We have to go through commercial ends. That means we have to build

a commercial product. Like I can see step-by-step. It just doesn't necessarily make it like the most

the most moral step-by-step logic. No, and I think they've done a lot of moral, I think there have

been some stances that have been both the right thing for Anthropics mission and the right thing

for their business. And some decisions that have been more the right thing for their business

than their mission. I think these deals that they've done with Gulf states, their willingness to

raise money from dictatorships, I think has been, their own employees have flagged that as sort of

an instance of hypocrisy. And I think they will continue to face these pressures because they are

now, you know, a huge company. And they are the fastest growing company by some measures in the

history of capitalism. And yet it is still not big enough. They still have not reached where they

are going. And so they will continue to have to raise money and make commercial sacrifices.

And this is what I think is maybe the most interesting part of this framework because, and I'm

quoting from your or summarizing your book here rather than coming up with this idea on my own.

But I think it is both very important to point out that Sam Altman and Dario Amade are extremely

different people motivated by not only different philosophies of the world, but even literally different

books like Sam Altman is a Ray Kurzweil and Nick Bostrom guy, super intelligence. And then you've got

Dario Amade reading about the making of the atomic bomb by Richard Rhodes. But despite their

extraordinary differences and mutual self and mutual loathing, they both end up in the same bucket of

accelerate. They both have to release chatbots that make tens of billions of dollars a year.

They both have to raise money from benefactors that aren't necessarily liberal capitalists.

And so no matter where you start out in this race, you end up in the bucket of speed, speed, speed.

I mean, that does seem like an important takeaway from this story.

It's a funny thing, isn't it? It's like whatever your motives are when you start,

you end up racing right alongside everyone else who's racing with very different motives,

but you're kind of checking your sides to make sure that no one's pulling ahead of you.

I think that's a really important point. And I think it's part of what is freaking out the insiders

at the labs. The people like Jacob Coxon, the people who are writing essays and open letters

about how we need to slow down. They see their bosses engaging in this kind of ego-driven

brinksmanship where it's like it is clear that part of the reason they are racing so hard

is because they can't stand the thought of these particular rivals winning.

And I think it makes them very uncomfortable. And it makes me very uncomfortable too.

I want to introduce our last character here, Demis Isabis. He might be the least famous of these

three men, but he is the only man on this list who's won a Nobel Prize for their work in AI.

And there's absolutely a version, I think, of the future, where his contributions might in 20 or

30 years be seen as the most important because of how significant the science contributions of

Deep Mind are. Let's just start in the biggest picture. Who is Demis Isabis? And what is he

want from Super Intelligence? Demis is a fascinating character. There's a great biography of him

by Sebastian Malibi that just came out. Yes, it's one that has way more material on him than I was

able to fit in my book. But basically, the story of Demis Isabis is he is a child chess prodigy.

He is a competitive chess player, you know, very highly ranked in the world for his age group.

Sort of a brilliant and competitive mind. People have described him to me as the most competitive

person in the world. There's this sort of famous anecdote about him, which is that this outfit

known as the Mind Sports Olympiad starts this competition called Pentamind, where it's basically

an all around Olympics of games, essentially. And they are trying to find sort of the best

game player in the world. And Demis Isabis learns about this contest decides that's going to be him.

He masters all five of the games included in the Pentamind and wins the contest five out of the first

six years. So he is the most competitive person in the world. And he's also very interested. He's

a cognitive neuroscientist by training as a PhD in that. And so he's very interested in this

question of what is intelligence and how can you build it into a machine. And this is the thing

that upsets us him. He becomes sort of convinced that it AGI is possible long before this is common

wisdom. People sort of look at him as like a fringe exotic like like who's this weird British man

talking about the coming machine God. And they they set up this little company deep mind and they

start making progress toward their path that they choose is teaching AIs to play games. So they

start with Atari games, then they move on to go and show key and all these other game Starcraft.

But their thesis is basically that you can train an AI to play a game, get really good at the

game and then sort of expand the domain to include things that are less and less like games and

more and more like tasks in the real world. And that that's higher going to build AGI.

And this is a very different approach from open AI and entropic and they are not going the language,

the large language model approach. In fact, you've got lots of quotes in your book of Demis and

people working at deep mind explicitly rejecting the language approach saying language is like a

simulation of intelligence, but it's not intelligence itself. Can you help us understand how

game based approaches to growing AI differ from open AI and entropic emphasis on

transformers and large language models? Yeah, they're essentially competing philosophies and

competing approaches to creating powerful AI systems. So the approach that deep mind uses and

specializes in the approach that helps them build this system alpha go that beats the best human

go players and that's a very famous moment in the history of AI. That was known as deep reinforcement

learning. This is basically trial and error. You know, you give an AI system a task, you know,

make make a go move that is likely to increase your probability of a win and you give it a reward if

it does that and you give it a penalty if it doesn't and then you just run it in simulation,

millions and millions of times, it gets a little better every time and eventually it surpasses

the best human go players and becomes superhuman. Their view is that a perfect, elegant AI system

should be able to learn without any human data whatsoever. You should just be able to stick it into

a novel environment like a go board or a chess board or something and it should be able to kind of

bootstrap its way to superhuman performance because it is learning from its experience. Open AI

and transformers and LLMs are based on essentially the opposite view. They are built out of human data.

Take the entire internet, jam it through a transformer, come up with a system that is extremely good

at predicting the next words in a sentence and use that to increase the overall intelligence of your

model. There's this very famous debate inside deep mind in these years, known as the language is

enough versus reward is enough schools of thought, which is basically the sort of microcosmic

version of deep mind versus open AI and anthropic. Do AI systems develop better intelligence as a

result of experiencing things and learning through trial and error or do they develop better intelligence

because they learn more of language and language is what we use to encode intelligence in humans.

I love that. Like Phyllis, obviously I was just thinking as you were saying that. I remember that

part of the book but I didn't have this particular thought as I was reading it.

The difference between those approaches philosophically is maybe a little bit like this. Tell me

if this metaphor is wrong. Language is enough. It's essentially like saying if someone could memorize

everything that had ever been written in science and great literature and human psychology,

they would be super intelligent. They would be able to, if they could hold it all in their head,

make connections and map new recombinations of ideas that would make them essentially super

intelligent. That's language is enough. And then there's reward is enough is kind of like

what if someone could live a million lives and experience in those million lives,

how some decisions lead to happiness and some decisions lead to unhappiness and some decisions lead

to making a company that makes 10 billion dollars and some decisions lead to making a company that

sucks. You wouldn't. Those million lives learn all of the positive and negative reinforcements that

would lead you to becoming super intelligent. And so to kind of interesting to think of like,

this is like sort of a Borhesian way of sort of summarizing these ideas. Like would you rather,

if you wanted to be super intelligent, memorize every word produced by all of the geniuses in human

history or just live a million times? That's like a funny way of putting it. That is almost entirely

true. And as it turned out, in the end, you needed both.

I'm all right. Are a combination of LLMs and reinforcement learning. So they all sort of converged

on this hybrid solution. But there were years where like open AI and anthropic or like,

open AI and parts of Google were like very interested in scaling up language models. And the people

at DeepMind just said like, that's crazy. You cannot build super intelligence out of Reddit posts.

Like it's just not going to happen. And they were wrong. And they they they have since, you know,

realized the error of their ways that are now building LLMs too. So Sam Altman and Dari Amadeh

hate each other. Altman and Hassabas met at the Vatican in 2015, 2016. As you reported,

they also don't like each other very much. I have a question about why they don't like each other,

but I'm going to put that on ice, read the book if you want to know why they hate each other.

Last question about Hassabas specifically. You know, Alpha Fold, which is the protein folding

technology that won the Nobel, which emerged from this RL theory of AI is a triumph. But Google

doesn't have an AI consumer business that rivals open AI or anthropic. And Hassabas initially

dismissed our language models as you've just said in a way that I think has cost Google significantly

on the commercial front. So it's a mixed legacy. And I wonder if you if you stop the clock today,

how do you see his legacy? I see it as both an incredible scientific legacy. I mean, as you

mentioned, he's the only one out of any of these guys who has a Nobel Prize, who has produced

something that has actually revolutionized a scientific field today. I think his theory of change

was correct in the broadest sense of like he believed in scaling, he believed in reinforcement

learning. He was early on a number of these things, including just the belief in AI itself.

But I think he, the dye may have been cast when deep mines was sold to Google. I don't think they

necessarily had a choice. They needed to build bigger blobs of compute to run their experiments.

They needed deep pockets. They needed to partner with a hyperscaler or someone with, you know,

the willingness to spend millions or billions of dollars on what they were doing. But I think that

ever since he sold deep mine to Google, he has been, in some cases, fighting against the commercial

incentives and the pressure, the short term pressures of being part of a large search-based

internet giant. And, you know, he explicitly tried to get away from Google. There's some

reporting in the book about this thing, Project Mario, which was sort of his attempt to kind of carve

deep-mind out of the larger Google apparatus. But that didn't work. And then he ended up running

a division of Google for her, you know, years with, you know, thousands of employees, which is not

what he wanted to spend his time doing. Most more recently, he's been sort of kicked upstairs to

this, like, you know, this sort of executive chairman position. And Google, you know, deep-mind is

firmly part of Google now in a way that it hasn't, you know, always been. So I think there's a tragic

element too of like, here's a guy who really wanted to use AGI to unlock the mysteries of science

and to discover, you know, new things in physics and math and biology. And who, you know, sort of

by no fault of his own, was stuck running, you know, a large division of a commercial search giant

that made most of its money through ads on search results. So, look, I think it's too early to

account Google out. They have lots of money. They have lots of smart people. And I think as long as

as Demis is there, they will have, you know, his sort of inspiration, his sort of drive. And he's

very inspirational to the people there. But I also think there is this reality that they are just

they are just a more traditional company than either of these other labs. And they have

lots of competing incentives and feuding internal teams and bureaucracy to navigate. And it's

just going to be a lot harder for them to move quickly. So, something up, we've got Sam Alton,

who I think of as kind of the JP Morgan of AI, like the mogul, the deal maker, someone whose

company, Bet on Transformers, Ship Chat, CBT, started this whole infrastructure cycle and has

continued has been very successful at using other people's ideas to build the power, his power

and the power of his company. Big blob of compute was Dario's idea, but who leads the labs in

compute secured by a mile, it's open AI. And so that's there you've got Sam. Dario, I see as you

described as something more like a moral crusader in the Oppenheimer mode, someone who's both an

inventor again, like Oppenheimer and but also someone who's who's safety crusading both inspire some

people and also rub some people the wrong way. You've got a lot of that in the book about how it

opened AI, some people were just so tired of Dario using the moral argument to try to win every single

corporate fight. And there's Hassabas, who's historical analog is harder to come up with,

because in some ways he's kind of like the Nikola Tesla, like a great inventor who was a little bit

in the shadow of his contemporary, but it doesn't work because he's like a Nikola Tesla who worked

at GM or something, because he's, as you said, one blocker for his success is that he's been inside

of a bureaucracy rather than leading his own thing outside of that conglomerate.

Final question to you now that we've sort of talked about their different motivations, different

inspirations, who do you think got the most right? I think scientifically,

Demis and Dario got the most right both by betting early on various forms of scaling.

You know, Sam Altman's not a scientist. I think he got the politics more right than they did.

I think he understood what it would take to make all of the deals, to secure all of the compute.

There's this incredible scene in the book where he's basically scheming to get around Elon Musk,

one of his nemices, to get into the good graces of the newly elected Donald Trump.

I think he is a savvier political operator than either of his rivals and maybe than anyone on earth.

I mean, he might be one of the savvier steel makers and operators we've ever seen.

I think that has been his downfall at times too.

I think they've both gotten pieces of it right, but no one has gotten everything right.

I think Dario's gotten the science more right than anyone. You can go back and look at these

papers he was writing in 2016 or 2017. It's like he's an oracle who sees the future.

But he is too honest. His fatal flaw as someone put it to me, he was close to him,

is that he cannot tell a lie to save his life. You could see him at that gaggle on the White House

lawn the other day, just sort of fidgeting and being visibly agitated and nervous.

While all of the other AICOs are there, just kind of smoothly kissing up to Trump and kissing the

ring. And so I think that is a quality I admire in him, but it is also something that has caused

him an anthropic a lot of trouble. Finally, I think it should be said. We've talked about three

people in this book. There's a lot of characters in the AGI Chronicles and one character that's a

little bit like a forest gump who just appears in every single story. Elon Musk not only

co-founded OpenAI with Sam Altman, he also tried to hire Dario Amade to SpaceX and tried

to acquire deep mind and Demis' work. I guess after he initially went to Google in addition to

funding some of Demis' work. So it is remarkable how like just cropping up through the book,

seen after scene is Elon Musk being like, hey, by the way, I'm holding a string here.

It's an amazing story. I'm really, really glad you wrote it. It's a wonderful, wonderful book.

Thank you so much. Good luck with that. Kevin Russe. Thank you so much.

!

Brussels clean up nicely at Sweet Green. Maple glazed, roasted, and edges perfectly

caramelized. Sweet Green's fall harvest is back on the menu and the season's most overlooked

little green vegetable is dressed to be devoured. You know what to do. Order on the Sweet Green app.