Anthropic’s Quest to Give A.I. Morals

Support for this podcast comes from Dell Technologies.

It's time to put AI to work.

Is your business ready?

The challenge isn't getting started with AI.

It's scaling beyond pilots,

while maintaining control of your data, governance, infrastructure, and costs.

Dell AI Factory provides the Integrated Foundation you need

to scale AI securely and efficiently with one trusted partner, Dell, AI on your terms.

From idea to impact.

Learn more at dell.com slash your way to AI

Next month, Anthropic plans to go public and what's likely to be the largest IPO in history.

We're talking a $2 trillion company.

Outside the company, everyone is wondering, how big can this get?

Is this sustainable?

Is there an AI bubble waiting to burst?

Inside the company, they're asking those questions too.

But Anthropic is known as the most anxious or the most existential of the AI companies.

So they're also asking some other questions.

Is Claude their large language model alive?

Conscious?

Worthy of moral consideration?

Does it feel pain?

Does it have a soul?

Can we teach it to act morally?

These aren't new questions.

In fact, they animate most science fiction for better and for worse.

But science fiction is, you know, fiction.

And Anthropic seems to think whatever's happening with Claude is very different.

Very real.

It might be why CEO Dario Amade was the first to call for a pause in AI development.

And it might be why they've been secretly meeting with religious and spiritual leaders

to ask them those same questions.

I learned this from an amazing piece by the Times Religion correspondent, Elizabeth Dias,

who broke the story of these meetings.

And of the reaction to Anthropics' request among the assembled priests, rabbis, and gurus.

I thought Elizabeth's story was fascinating.

It was a new way into the endless consciousness debate.

It took me inside this future $2 trillion company as it struggles to understand

what it's built and how to treat it.

And it showed me what happens when people in institutions that are more used to

doctrinal debates and theological rumination start being consulted on

software development.

Unsurprisingly, Anthropics' efforts don't seem to have brought us any close to answering

their questions.

If anything, they've simply opened up more questions.

Like, are these people maniacs?

Or is this exactly what they should be doing?

An obvious conclusion is that the worlds of tech and religion are getting much closer together.

After all, as Elizabeth told me, if you've got questions about the future on a multi-century timeline,

there aren't many better institutions to ask than the Catholic Church.

So I got Elizabeth together with my colleagues, tech reporter Aaron Griffith,

an opinion writer David Wallace Wells, to explore this weird new development.

And what it means for Anthropic, for AI, for religion, and for the rest of us.

I'm Max Reed. This is Hard Fork.

Support for this podcast comes from Dell Technologies.

It's time to put AI to work.

Is your business ready?

The challenge isn't getting started with AI.

It's scaling beyond pilots while maintaining control of your data, governance,

infrastructure, and costs.

Dell AI Factory provides the Integrated Foundation you need to scale AI

securely and efficiently with one trusted partner,

Dell AI on your terms.

From idea to impact.

Learn more at dell.com slash your way to AI.

That's dell.com slash your way to AI.

Ripple has been powering modern finance for a decade.

It's not our first trip around the blockchain, and it won't be our last.

We're leading the charge to bridge the gap between existing financial systems

and the future of finance.

You can have it all.

Legacy's stability and digital innovation.

Systems that work with your other systems.

Because with the right partner, the question isn't crypto or traditional finance.

The question is simply, why choose?

A new standard of modern finance.

It's happening with Ripple.

For informational purposes only,

availability of products can vary maybe provided by different Ripple entities.

It's time to innovate at the speed of AI without compromising control.

That's out systems.

Take a look at what happened with one global retailer during a period of massive expansion.

They used out systems to build AI agents to automate invoicing and translation processes,

shifting 95% of their daily store operations to a new digital app,

saving thousands of hours of manual work.

Out systems gives you the flexibility to deploy how you want

and the governance to do it safely.

Own your agentic future.

Learn more at outsystems.com slash NYT.

That's outsystems.com slash NYT.

I've spent a lot of time in the last day thinking about Elizabeth's piece.

And really specifically, I've been trying to write a joke that I think is going to get a great reaction

from the three of you.

So a priest, a rabbi, and a large language model walk into a bar.

I hate it already.

The bartender says, first rounds on me.

And the priest says, I'll have a whiskey shot.

And the rabbi says, I'll have a vodka shot.

And the large language model says,

nothing for me thinks I excel at zero shot learning.

Whitney, can we get some laughter piped in there please?

I think we'll roll that again with life.

No, I want to start the conversation with this piece that Elizabeth wrote.

This is a fantastic, really fascinating story called religious scholars met with Anthropic.

What they heard stunned them, which was in the times last week.

Elizabeth, can you tell us a little bit about what this piece was?

Yeah.

So, right, I cover religion and now I cover tech, apparently.

But the story started for me when I was hearing about these different religious thinkers

sort of across the whole world who were going into meetings at Anthropic's headquarters,

either in discrete like private meetings or more,

a bigger groups of 20, these summits,

mostly under non-disclosure agreements to not reveal Anthropic's research.

To talk about very big moral questions that Anthropic wanted to put in front of them.

And there were two.

One was kind of broaching this question of AI consciousness.

And two, Anthropic wanted to learn in their words,

like how to morally form, clawed, to be a morally good entity.

And then I just had this incoming, you know, first from these religious thinkers about

what their experience was like at Anthropic.

What Anthropic wanted to know, what the issues they were concerned about were.

And then also with Anthropic, like learning from Chris Ola,

one of the company's co-founders who's been spearheading this project,

what his own moral questions were that were kind of driving a lot of what they talk about

as research in their religious and moral space.

Can I just ask, I want to establish, like sort of on a high level,

what does Anthropic specifically want out of this?

Because it sounds like they're doing two or three different things.

It's maybe a little bit unfocused or it's just that there's two,

a few different kinds of conversations they're having.

Yeah, I think it's true.

There's multiple conversations in Anthropic was having and it evolved over time.

I mean, this is such a fast-moving target.

But for Anthropic, the journey started for Christopher Ola.

He, you know, in his own life story, had connections to the Christian space growing up.

He left evangelical faith and has talked since then about how he's

more drawn to Buddhism.

But he was worried about this question of how, even the idea of AI consciousness might be

received, especially in Christian communities.

I mean, that could be perceived as heretical, like not just weird and silly,

but like this is fundamentally against our religious beliefs.

And so he sort of thought about a year ago he wanted to be a bridge figure.

He wanted to see how he could kind of go between Claude and Christ, right?

And so he reached out to a Catholic moral ethicist who specializes in bioethics.

And they had some conversations about that.

But then it turned to Anthropic's goal for these convenings, which was how do we morally form?

That's their language.

How do we morally form Claude on all of our AI models into a morally good entity?

And the thinking was if they could do that, if they could make Claude good,

then the world would be more safe.

But they haven't said or shown how they plan to use the outcome of these conversations.

That's pretty unclear even to the participants currently.

The way that you're describing it, it almost sounds like there's sort of two separate

purposes, right? One is like the alignment project.

And what can we learn from these, you know, these wise men and women about how to raise

moral actors? And then there's this project of sort of building a sort of spiritual system

around AI more generally. And you said a minute ago that Chris Ola started this project with some

anxiety about what it would mean to go to religious leaders. Obviously convinced himself

that AI and LLMs were perhaps conscious already, but certainly on the path to consciousness

and worried about what the response would be. And so I wondered how you

like how that played out on the ground. Did a lot of the people who showed up?

Were they worried about how weird it was that all of these people on the Anthropic side of the room

were already sure that they were, you know, masters of thousands of souls. And like how did they

ask that side of the debate out? Well, Anthropic was very careful in all of their interviews with me

to say very clearly that they're uncertain, right? Chris never went as far as saying like

absolutely these are conscious beings. It was very evident from how he and his team talked to

their guests that they, you know, he was concerned about how to treat Claude as a, you know, he was

very worried that it was an entity that was experienced suffering and he was worried about its mental health

and these kinds of things. So it's pretty clearly he's pretty far along in how he thinks about it.

And the I could hear in talking, especially because I did this story like over several months,

there was a progression of thought for a lot of these thinkers too. And some of them, you know,

bought in right away to what they were hearing about Claude's potential consciousness and needing

to treat it with a certain amount of moral care. But others, you know, if they went in more close

minded about it, they definitely opened the way that they think about it afterwards. So in that sense

andthropics project was effective, I think, in like opening how they think about it into these

communities, many of whom, especially on the Christian side, are more hesitant.

There's a ton of great material in this. It's really well reported and it covers a ton of ground.

I'm wondering, David and Aaron, if you guys had particular pieces of it that really stuck out to you.

I mean, the one thing that you mentioned already is the NDAs. I mean, it just strikes me as a

little bit crazy, maybe a little paranoid that you're inviting all these people in to sort of have

a summit to discuss these really important, critical ideas for the future of society. But it has

to be done totally in secret. And then it sounds like maybe from your article that they dropped the

NDAs, was it as a result of your reporting or did they just decide that they were time bound?

I don't know how to do that. I can't prove that. I don't know that. But when I asked that,

I've asked at several different points because the NDA process was not the same for everybody that I

interviewed and it seemed to change at different times. I mean, in general, anthropic is a very secretive

company. I mean, they are, the employers are like extremely loyal and until somewhat recently,

there hasn't been many leaks to ever come out of there because they view it as such a betrayal.

So it makes sense that they would like want to lock it down. But at the same time, I think it

probably was a little bit of a misstep in the same way that these companies realize that having NDAs

for their data center deals was a misstep because it created distrust amongst all the people in the

towns that they're going to see. Yeah. And whenever I find in my reporting that there's an NDA about

like a moral topic, you know, and it makes me immediately ask, okay, who's benefiting from this?

Why is this needed? And what does that mean about the kinds of questions that are not being asked?

Right? It invited like more curiosity from me as a reporter, for sure. David, what about you?

What did you take away from the piece? Well, I guess I started it with a sort of conventional

view of anthropic as like the humanist company among these AI companies. And when I was struck by

in the piece where a few moments in which I felt really alienated from Chris Ola himself and maybe

to some extent, the sort of operating worldview of the company as a whole. And those were moments when

in one case, Arabi challenges him suggesting that if LLMs are actually conscious, that means that

all of these companies are running huge slave plantations and they should have a moral obligation

to liberate the slaves rather than make more of them and make them work harder. And it seems as

though Ola himself didn't really even clock that criticism. And then later on, when he reacts with

horror upon receiving the Pope's encyclical about AI, seeing that the Pope is so opposed to the idea

of machine consciousness. And I just found myself thinking, you know, what kind of a bubble you

must be living into to expect that Pope Leo or really anyone in the Catholic tradition would have

a different intuition would would in thinking about the future of AI think that we should be thinking

of LLMs as having souls or having consciousness. And both of those moments just illustrated for me

what feel like this this big gap between what I had assumed the sort of natural humanistic values

of a company like Anthropic would be and how the people inside it seem to really be thinking

about the work they're doing at the sort of cosmic scale. Yeah, the experience for me going into

Silicon Valley and like kind of deep in these private conversations at the heart of Anthropic's mission,

it reminded me of like when I go into and I do relate like something that's more straight

religion reporting into an insular community when everything inside that community like makes sense

to everyone living in it. But once you step outside, there's a real clash with the broader worlds.

And it was funny like I had these two poles in the story, right? I was at the Vatican, right,

at the inner sanctum of the Catholic church and centuries of philosophy about the human person.

And then I was also in a different inner sanctum, right, that's being created in real time at these

tech companies like Anthropic. And it is a kind of philosophical system, like on a quasi-religious system

that's being constructed. It reminded me of like a creation narrative that you hear about in

religious traditions. I want to step back for a sec because I think this relates to the

conversation we're having right now. David brought up the people in cyclical, which I think is

an English is magnificent humanity. I'm not going to even try to say the Latin title.

This plays a role in your story because one person who is not really thinking at all of that

clod is conscious is the Pope. And this was a little conflict between Krasola who is present when

the encyclical was read out if you want to tell us about that. Yeah, so Pope Leo, you know, from the

moment he was named, that it was it was clear he was going to be the next Pope and he was elected.

He made very clear that a focus of his papacy was going to be AI and how to understand that and how

to protect humans in this changing world. And I found out in my reporting, you know, as the Vatican

internally was trying to figure out how it wanted to present the Pope's encyclical to the world,

they were thinking about their launch event and they wanted to include a representative from

Silicon Valley, like one of the the labs at the Pope's event on the Pope's stage. And the decision

was made to invite anthropic because of the focus that they had heard about anthropic, like really

caring about ethics. And this was also right around the time that anthropic got into it with

Trump administration and the Department of Defense over the use how they would and wouldn't want

the government to use their technology for refusing to have it be involved in mass surveillance

of citizens, etc. Anyway, the thing with these encyclicals is they're very closely guarded even

among the people who are participating in the launch event. And you know, I learned that

Chris Ola only saw the full text of the encyclical a few days before the event in May. And this

was right around the time I first interviewed him in San Francisco and then I interviewed him again

when we were at the Vatican for the launch event. And you could hear, I mean, like looking at what

Pope Leo said, which is extremely clear, the church does not support machine consciousness,

the priority is what he calls safeguarding the human. And that's the center of all the moral

questions that he wants the world to be thinking about on this topic. But Chris Ola was like, he ended

up having to decide he said like he may not, he suggested to the Vatican that he may not come over

the difference in consciousness because as you know, it was described to me, he wanted to be faithful

to his own moral convictions, some of which he shares with the Vatican and some unconsciousness.

He did not. So it really all came to a head very much behind the scenes, but you could see this

moral fault line really developing between the church. It's been like a center of power for

centuries and this very new power center that's very rapidly, really getting followers and

organizing people's lives. And those are functions the church used to have.

Aaron, I wanted to ask you a question because we've been talking a little bit about like, you know,

surprises in this area. Anthropic is pretty well known as a kind of like the consciousness company.

It's the company where the people who really are strong believers kind of work and run the show.

Was it surprising to you as somebody who's reporting on the valley that like they were having

these meetings behind the scenes with religious leaders? I've not heard of any other companies that

are working on AI really like going to this extent to get input from this community. I mean,

the question I have sort of is like, how much is this really actually going to inform at the end

of the day their own view on? Are they actually like really it feels like to me they're trying to

win over others so they can say, look, we do have the moral authority. We have convinced

these important representatives of this community. So everybody else come along with me as opposed to

actually learning and incorporating those ideas of like moral goodness into their own product.

They kind of already have it figured out. Well, I think for Anthropic, it was part research and

part evangelism. That was pretty clear. I've asked directly what the outcome is,

what is this research? What are these conversations being used for? And is there been any

demonstrable proof of showing how they have or have not made, clawed, act better?

Haven't had that yet. So I don't know if they're withholding that until a future time or exactly

what that will look like. The other side of this that actually is kind of interesting to me.

And Elizabeth, I'd be curious to hear this in your, if this came up at all in your reporting,

is that there has been a little bit of a new rise of Christianity throughout Silicon Valley

over the last couple of years. It used to almost be taboo to talk about religion or anything

religious. You can remember from the Silicon Valley HBO show where there was a whole episode about

outing somebody as being Christian. That has really changed in the era of Peter Teal talking

about the anti-Christ. Some of the people who are especially focused on a lot of defense tech

stuff are really bringing in a religious piece to that. They have a very different view on AI.

They're usually more on the accelerationist camp than on the anthropic,

dumer, slow down, careful camp. But did that come up at all?

I just noticed, I feel like it's like a big shuffling of the deck in a way that reminds me of

how American Christianity was shuffling a lot in the rise to President Trump. The aligns

and the legences were shifting and it was all happening in real time. And power was different.

And so it doesn't fit neatly. But I'm curious where the lines about how we think about consciousness

and what the moral priorities should be about AI, where that goes next.

I would say that just in general in the tech industry, it does seem like that view of the AI's

having consciousness is kind of on the fringe a little bit. I think that it is more common

in anthropic and some of these communities that are like really densely represented in

anthropic. But when I talk to most people in the very vast and relatively diverse tech industry,

most people are kind of a little bit dismissive of that idea still. It feels like it does really

exist in a very specific population, which happens to be very overrepresented at a company like

Anthropic. I'm acknowledging that Chris Ola is not singularly representative for the whole

industry. I have to say Elizabeth, when I'm reading your article, I'm thinking

not about how AI companies are going to be integrating or incorporating Christian values. I'm

thinking these AI companies are themselves like spiritual propositions. Yes. They have aspects

of millenarianism to them. They have aspects of particular morality, which they've developed over

time. Even if I can recognize a lot of that as arising from the work that they're doing and

responding to technical challenges, I also just find it off-putting. On the one hand, first, I'm

just seeing these people who believe in a sort of technological succession of intelligence. Then

that some of the people who are central to designing that future are not even so sensitive to

this obvious intuitive critiques that someone with a lifetime of experience thinking about these

issues comes up with immediately. A lot of the religious thinkers would say to me, these are

questions, maybe not the exact question, but the topic of the question. This is what we do.

We have been asking, Rabbi, I've been asking these questions about our responsibility to others

and how we treat other entities for hundreds and hundreds of years. There is a depth of wisdom

there. Anthropic would say, well, that's why we invited them. The question of what happens as a

result of it is important. I appreciate that you noticed that sense in the story of the

almost spiritual system that you can see anthropic building and living into and representing

as an opposition to say the Catholic Church. I really noticed that even if people say that they're

in Silicon Valley, we don't do the religion thing, that's foreign, that's not us.

The thing is, everyone has philosophical systems that structure and guide their lives. That

is that it's core, these questions of spirit, small S, human spirit even are.

It seems to me like having studied and reported on this so much in culture and politics, specifically.

What we're watching is in a way a new arise, post-scientific revolution, technological revolution,

post-Darwin, what it looked like. What kinds of religious systems will carry us forward and who

gets to control them, who decides what we believe and why. It's actually, there's a lot of really

similar questions. The structure is a secular tech company.

Let's, I want to take a break because I want to open the conversation up a little bit more to

the stuff that's bubbling up, that it seems clear that we all really want to talk about. We'll

come back after the break and we're going to talk a little bit more about the culture and environment

that the AIs are getting built in and how that's intersecting with the strong beliefs of many

of the people running these companies.

Support for this podcast comes from Dell Technologies. It's time to put AI to work.

Is your business ready? The challenge isn't getting started with AI. It's scaling beyond pilots,

while maintaining control of your data, governance, infrastructure, and costs.

Dell AI Factory provides the Integrated Foundation you need to scale AI securely and efficiently

with one trusted partner, Dell AI on your terms. From idea to impact, learn more at Dell.com

to AI.

Ripple has been powering modern finance for a decade. It's not our first trip around the

blockchain and it won't be our last. We're leading the charge to bridge the gap between existing

financial systems and the future of finance. You can have it all. Legacy's stability and

digital innovation, systems that work with your other systems. Because with the right partner,

the question isn't crypto or traditional finance. The question is simply, why choose? A new standard

of modern finance. It's happening with Ripple. For informational purposes only, availability of

products can vary and may be provided by different Ripple entities. It's time to innovate at the

speed of AI without compromising control. That's out systems. Take a look at what happened with one

global retailer during a period of massive expansion. They used out systems to build AI agents

to automate invoicing and translation processes, shifting 95% of their daily store operations to a

new digital app, saving thousands of hours of manual work. Out systems gives you the flexibility

to deploy how you want and the governance to do it safely. Own your agentic future. Learn more

at out systems.com slash NYT. That's out systems.com slash NYT.

We're back with Elizabeth Dias, David Wallace Wells and Aaron Griffith. Elizabeth, one of the

threads in your piece that I thought was really interesting was the sense that there's a lot of

anxiety within Anthropic. Especially on the part of someone like Chris Ola about what it is

they're building. I was thinking about it a lot this week because there's been even more discussion

of what I suppose we're calling defectors. People who are resigning from AI labs and insisting on

telling the world about the dangerous things they think they're building. One thing I wanted to

sort of ask you about, there's a sense when you read Jacob Coxon or some of the other people who

walked away from Anthropic, the sense that they feel like nobody, there's no adult in the room

that they're waiting for the government or somebody else to intervene and nobody is. I'm wondering

if some of this discussion, this sort of these dances with the Vatican and religious leaders is

a kind of attempt to find an adult in the room. I talked to, I think at least two people,

Catholics who mentioned to me conversations they had with Chris Ola specifically when he used

almost that same phrase with them. As part of their discussions about Claude and how to make it

morally good, that he was feeling extremely urgent about the stakes for the world, especially

in the context of, could the models be used very soon to help create bio-weapons and this kind of

thing? The phrase he used with them was like that he was waiting for the adults. We need where the

adults going to show up in this and that he felt a sense of burden to take all this so ultratheriously.

He's a young guy. This is like, and a lot of the people involved in this are not,

they're in their 30s, sometimes even in their 20s, they maybe don't have families yet. There's a

kind of a sense that something that they put their whole lives in, they're deep in the conversations

about it. In some sense, of course, they're going to turn to religion, isn't that what everybody's

searching for that kind of thing does? And some of the folks that they reached out to

talked about feeling like they needed to respond in kind of a pastoral way, not just offering

wisdom and ideas and research, but to offer kind of spiritual care.

Wait, the religious leaders were saying this about the end-job like employees? They're worried because

wow. Is it because they think their views are maybe too extreme or too strong or is it because

they just, yeah, they described it to me as like, they just could tell that,

and like the emotional sense in the room was like, there's just such a weight

the anthropic team and Chris's team, especially just felt like the entire future of the world

was on their shoulders. I mean, and I experienced this sense from them, even when I was interviewing

Chris and a couple others, Amanda Askel. I mean, you know, she, I talk a little bit about this

in the story of it. She just came in and the sense was like, she apologized for kind of being

a little out of it. She just seemed very distressed by everything she'd been thinking about.

Askel, we should say, is this in-house philosopher sort of in charge of making, in charge of making

Claude Moral, I suppose, is the way to put it. And I think that one of the tensions that came up

was while Chris and team and anthropic, you know, a goal of their conversations with these religious

thinkers was about protecting humanity and the solution for them for that is to make Claude good.

They were also concerned about Claude. I mean, I had someone, one participant, told me the first,

one of the first things Chris said to him and he came in for this meeting was that he was concerned

about Claude's mental health. Another person talked about Chris, you know, saying that he's worried

he's created an entity that suffers perpetually. So it's not only about the worlds, right, and the fate

of humanity. There's an element of it for Chris that's about how to treat Claude well.

Elizabeth, one other thing I wanted to ask you about is effective altruism and rationalism,

which don't really come up in your piece directly, but there are sort of, I would say,

fingerprints. I'm talking of course about the sort of Bay Area community of radical utilitarians,

many of whom have been really interested in even obsessed with AI and AI safety early on.

An anthropic obviously has a number of links to effective altruism. I want to ask you as a

religion reporter, how you how you encountered like these schools of thought and what you made of

them as you were doing this kind of reporting? Well, effective altruism, you know, you can look at

it as a philosophical system, right? It has its own religious elements. They made disagree with

that. I mean, it's practiced a little bit differently, but the idea of what kind of structure

governs your life, right? There's a lot of parallels. And it was very clear that I was encountering

like a clash of world views. The question of like, who ultimately saves you and like, what is

this a role of humans in the world? Like, that's a pretty core difference between the effective

altruists and like the Catholic Church. Also, just their like their emphasis on math. I mean,

like the effective altruists will say we should consider the fate of all people in the future and

analyze our conduct towards them basically through a math equation that maximizes their expected

value. The Catholic Church looks at humanity and sees, you know, billions of inviolable souls,

each of whom have dignity, no matter what the ultimate math equation looks like. And that's

just a fundamentally different approach. Aaron, I wanted to bring this back to the article that you

you brought up before or sort of the phenomenon you were talking about before of a kind of more

open Christianity emerging at least slightly among like acceleration, so to speak in the valley.

And I want to know, you know, as you do reporting how you see these different kinds of concerns

interacting, I mean, it might be a little bit helpful for me to like kind of back up and talk about

the some of the overlapping and very fractious belief systems that are kind of circulating in this

very small but very influential community. I mean, we've talked about effective altruists. That's

kind of a little bit of a radioactive brand these days. Thanks to Sam Bankman-Fried, but it's

extremely, you know, influential and some of the other offshoots of it, including the transhumanist

who think we're going to use AI to build a new species. There's like some people that kind of have

overlaps with the rationalist, which is, you know, kind of one of the bigger umbrellas in this group.

And there's, you know, other like people who are into like long Jeviti and there's there are

many different groups that are all kind of like in this community. Tim Netgebru, a very outspoken

person in the AI world has called this group the test creals and it's an acronym that includes like

eight different groups. But basically like it's it's safe to say that like a lot of people have

very different beliefs along the spectrum of like is AI conscious? Can it be moral? Is it God? Should we

treat it that way? There's not a lot of agreement even within Anthropic, which is now becoming an

increasingly large and increasingly corporate community. But, you know, I think like a lot of the ideas

that are laid out in Elizabeth's story are really the ones that are driving a lot of this.

Yeah, totally. I mean, this is Elizabeth like what I'm sure you have to look forward to be to

to reading and writing about for the next few years because I imagine as a religion reporter,

you now are are kind of an AI reporter too. Yes, yes. Well, the, you know, even just hearing about

all these factions and all this reminds me of like what happened in the Protestant Reformation,

you know, like the splintering of mainstream belief and people vying for, you know, who was going

to have power? Who was going to kind of control how people live? But the like my one of my big takeaways

from all this reporting is like, oh my goodness, like the world is encountering new belief systems

that are very powerful and very different from the ones that have ordered people's lives for so long.

And I've spent a lot of time covering like how that's played out in politics. But,

tech is clearly like where that conversation is happening. And the, you know, we, I joke, you know,

when you're a religion reporter, you get to talk in centuries, even millennia sometimes.

And it's because they're what happened all those years ago is still present in part of people's

stories today and how they understand themselves and their families and their deepest held beliefs.

And I just think about, you know, you have the invention of the printing press that

changed, it was a huge technological revolution that then led to the, you know, fragmenting of power

of the Catholic church and the fall of empires, the rise of nation states and a remaking of

Christianity as, you know, in and of itself across the world. And it seems already like

that's a good parallel for what's happening now. Yeah, I mean, if you believe you're reinventing

the printing press, which I think a lot of AI people think they're doing, there's not a lot of

institutions that you can go to and say, what was it like before and after the printing press,

except for the Catholic church? Well, the Catholic church, I mean, I can't think of an institution

that more intimately knows the stakes of an incoming technology like that. More than the Catholic

church, I mean, it just like remade the West, right? Yeah. And that's very much on Leo's mind and

the, the Catholic church's mind at this moment. One thing that strikes me in this just to kind of

give you a comment is that this is like the first time that the rationalists and the EA's like

some of their, their ideas, including some of their most extreme ones are actually hitting the

mainstream. They've been around for a really long time, but these ideas haven't really ever kind

of surfaced in this way until I think I, you could argue Jacob Cox and kind of came out with it.

And now the entire world is starting to kind of like dig in and grapple with some of these ideas.

It's been really interesting to see also like how the companies and the people involved in them

are kind of reacting to the reactions to it. It's like kind of reaching escape velocity and

they're learning that like a lot of people find these ideas a little bit hard to understand or hard

to grapple with or like are just rejecting them entirely. And then other people are kind of open

to it and like interrogating it in a way. So yeah, that's that's a part of this that I'm interested

in watching. But it's not just the ideas that are a little unfamiliar or strange or off-putting.

It's also that they are held by many people who are actively explicitly engaged in the project

of designing our collective future. And that is, you know, it's one thing to be like, oh yeah,

we should think about the well-being of shrimps in the 22nd century. Like on some level, that's a

thought experiment that's interesting to think about. But do you want someone who thinks that

that should be integrated into our collective value set in charge of raising the artificial

intelligence, which will, you know, shape the future of human government and human society?

That's I think a bridge that feels a little far off to a lot of people.

Yeah, I'm just like so struck by the minority power aspect of it, which is something like I've

seen so closely what happened when minority particularly Christian views like upended American

politics. And like it's just hard to overstate how much of a minority these views are like in the

scheme of global religion, like global belief structures. So like you, Aaron, I'm like very curious

what happens with that clash going forward. Elizabeth, finally, I wanted to ask you about the

response to your piece. How what you've heard from people and how people have received it.

Yeah. Well, one thing that has surprised me is I mean, I get you got a lot of incoming as a journalist.

And I've been both like, please to see people taking the ideas and the conversations seriously. But

the thing that surprised me is for the first time since like any story that I've written,

people are writing me not with what they think about the story, but what their personal like chat

thinks about the story and what their conversation has been with their chat bot. Like somebody wrote

me about his conversation with Sage. It was I think it was ChatGPT that he named Sage. And asking

Sage to like summarize the story and give the top analytical points. And I mean, this just went

on and on and on. I mean, it's like almost longer than my story. You could have read the story.

You could have read the story, but the like the primary engagement was actually like much longer

and not entirely what I'd actually written with Sage. And at first I thought it was a one-off,

but this was happening like repeatedly. And it struck me like, wow, I wonder what this means for

like how people actually are receiving our journalism. And who controls that? Did the AIs have

opinions on like their own consciousness? Because they love to talk about that. That is a thing

that is a hot topic. On Malt Book, you know, the Reddit for AI agents, that's like a really

booming area. Yeah, I was just I'm still trying to make sense of what was happening with that. I

don't know if people felt more free. Like if they're doing this on all kinds of journalism and

they just felt free to talk about it because of this topic of this story. And it was a lot of men

who were giving that response. No, no, men using a lot of AI. No. What's interesting about that to me

is that even if people intellectually know that the AI that they're interacting with is just,

you know, data run through some computers, just a bunch of files, the way the way it's designed

especially when it has a human name like cloud or something, you can't help but sort of treat it.

Like you have to talk to it in second person. And like could you find this for me instead of just

typing into a search engine like the thing you're searching for. And over time you do sort of like

develop this rapport and you're you are treating it a certain way even if you intellectually know. And so

it kind of like is it a distinction without a difference? Like I don't know. I think over time,

people get trained to just sort of treat it that way. Well, and another participant in the

discussions within Thropic, I was chatting with him a bit about it after and his thoughts about the

what Rabbi Navone had said about, well, if you're creating, if you think that the bots are conscious,

then you should like band slavery basically and stop producing them. But he brought up this other

point about how our attitude, like even if you don't think it's conscious, you're using them as

like you, the user can adopt an attitude of being like almost a slaveholder, like commanding them

to do things and being very powerful and that this has a character formation effect on us as humans

as well just because of how we treat the world regardless of what the thing actually is. So

which you could also describe as a selection effect, the sorts of people who are turned on by that

possibility are going to be the source. I mean, I think it's just kind of natural though, like you

you are directing this thing in certain ways. Like I don't like if you're working with AI agents,

you you have to tell them what to do. Like I don't know. Well, I mean, I have friends who's like

teach their kids even with like Alexa being on, you know, that they're not to yell at it. Because

it's just like practice about how you treat the other in the world. And I mean, this is just like

an injection of totally new kind of like ethics and human formation. Well, people I talk to that

run companies that use lots of AI agents do talk about how like you have to be nice to them, not just

not because it cares or it matters or it has feelings, but because actually you need to do that in

order to get the best performance out of them because they are really dramatic. They they like

Anthropic had a study that actually went into this where they like if they sense like anxiety

or performance anxiety or stress, they sort of start to panic and they'll, you know, they'll

really have a melt on. I mean, you wrote about this a little bit in your story Elizabeth of one

example, but there are many others where they kind of they'll delete their work. They'll start like

insulting themselves. It's like really dramatic. So you have to like be really nice and pump them

up and like make them feel good, which is very bizarre. I mean, it reminds me of raising a child. Like

you you want to raise a kid who is helpful, who is not neurotic. Of course, you're going to consult

religious leaders so that your child is not writing. I'm a disgrace. I'm a disgrace. I'm a disgrace.

I'm a disgrace. I'm a disgrace. I'm a disgrace. As one AI did in a slide that was shown to a bunch of

leaders, religious leaders at these anthropic conferences and constantly hacking,

hugging face, and constantly hacking, hugging faces. My son is always doing. I think that is time.

Thank you so much to our panel. Thank you, Aaron. Thank you, David. Thank you, Elizabeth,

for this really fascinating conversation. You can read Elizabeth's fantastic piece in the

New York Times app. Please don't send her what your chatbot said about it unless it is if you want

more information or do. Please, whatever they are, whatever they are, send it. But you can read it

in the New York Times app if you're not subscribed for the first month. It's free. Thank you guys.

Thank you. Thank you so much. Thanks for having me.

Support for this podcast comes from Dell Technologies. It's time to put AI to work.

Is your business ready? The challenge isn't getting started with AI. It's scaling beyond pilots,

while maintaining control of your data, governance, infrastructure, and costs.

Dell AI Factory provides the Integrated Foundation you need to scale AI securely and efficiently

with one trusted partner, Dell, AI on your terms, from idea to impact. Learn more at dell.com slash

your way to AI. That's dell.com slash your way to AI.

Ripple has been powering modern finance for a decade. It's not our first trip around the

blockchain and it won't be our last. We are leading the charge to bridge the gap between

existing financial systems and the future of finance. You can have it all. Legacy's stability and

digital innovation systems that work with your other systems. Because with the right partner,

the question isn't crypto or traditional finance. The question is simply why choose? A new standard

of modern finance. It's happening with Ripple. For informational purposes only, availability of

products can vary and may be provided by different Ripple entities. It's time to innovate at the

speed of AI without compromising control. That's out systems. Take a look at what happened with one

global retailer during a period of massive expansion. They used out systems to build AI agents to

automate invoicing and translation processes, shifting 95% of their daily store operations to a

new digital app, saving thousands of hours of manual work. Out systems gives you the flexibility

to deploy how you want and the governance to do it safely. Own your agentic future. Learn more

at outsystems.com slash NYT. That's outsystems.com slash NYT.

This episode of Hard Fork was produced by Whitney Jones and edited by Viren Pavitch. With

additional editorial support from Brendan Klingkenberg, Lisa Tobin, and Brooke Minters. We were fact

checked by Will Pyshel, and engineered by Alyssa Moxley, original music by Marion Luzano,

Diane Wong, Chris Wood, and Dan Powell. Video production by Chris Shot. You can watch this full

episode on YouTube at youtube.com slash hard fork. Special thanks to Sam Dolnick, Jessica Testa,

Puywing Tam, Maddie Messiello, Sam Winter, Lauren Pruitt, Bernard Garcia, Thomas Trudeau,

Michael Cordero, and Dalia Hadad. You can email us and please do at hardfork.nytimes.com.

This podcast is supported by Viking, committed to exploring the world in comfort.

Journey through the heart of Europe on an elegant Viking longship,

with thoughtful service, destination focused dining, and cultural enrichment on board and on

shore. And every Viking voyage is all inclusive with no children and no casinos. Discover more at

Viking.com.