Peter Thiel on Technological Stagnation and Out of Touch Rationalists

Thiel's argument against Bostrom's Vulnerable World Hypothesis is basically "Well, Science might cause bad things, but totalitarianism might cause even worse stuff!", which, sure, but Bostrom's whole point is that we seem to be confronted with a choice between two very undesirable outcomes: either technology kills us or we become totalitarian. Either we risk death from cancer, or we risk death from chemotherapy. Thiel implicitly agrees with this frame, it's just that he thinks the cure worse than the disease, he doesn't offer some third option or argue that science is less dangerous than Bostrom believes.

He also unfortunately doesn't offer much against Elizer's "Death With Dignity" post, no specific technical counterarguments, just some sneering and "Can you believe these guys?" stuff. I don't think Thiel would be capable of recognizing the End of The World as such 5 years before it happens. However his point about the weirdness of bay area rationalists is true, though not especially new.

calls out Bostrom as out of touch

I think he actually said that Bostrom represents the current zeitgeist, which is kind of the opposite of "out of touch"? (Unless he also said "out of touch"? Unfortunately I can't find a transcript to do a search on.)

It's ironic that everyone thinks of themselves as David fighting Goliath. We think we're fighting unfathomably powerful economic forces (i.e., Moloch) trying to build AGI at any cost, and Peter thinks he's fighting a dominant culture that remorselessly smothers any tech progress.

Here's a transcript. Sorry for the slight innacuracies, I got Whisper-small to generate it using this notebook someone made. Here's the section about MIRI and Bostrom.

But but I, you know, I was involved peripherally with some of these sort of
East Bay rationalist futuristic groups.
There was one called the Singularity Institute in the 2000s and the sort of the
self-understanding was, you know, building an AGI, it's going to be this most,
the most important technology in the history of the world.
We better make sure it's friendly to human beings and we're going to work on
making sure that it's friendly.
And you know, the vibe sort of got a little bit stranger and I think it was
around 2015 that I sort of realized that, that they weren't really, they didn't
seem to be working that hard on the AGI anymore and they seemed to be more
pessimistic about where it was going to go and it was kind of a, it sort of
devolved into sort of a Burning Man, Burning Man camp.
It was sort of, had gone from sort of transhumanist to Luddite in, in 15 years.
And some, something had sort of gone wrong.
My, and it was finally confirmed to me by, by a post from Mary, Machine
Intelligence Research Institute, the successor organization in April of this
year.
And this is again, these are the people who are, and this is sort of the cutting
edge thought leaders of the, of the people who are pushing AGI for the last 20
years and, and you know, it was fairly important in the whole Silicon Valley
ecosystem.
Title, Mary announces new death with dignity strategy.
And then the summary, it's obvious at this point that humanity isn't going to
solve the alignment problem.
I, how is AI aligned with humans or even try very hard or even go out with much
of a fight.
Since survival is unattainable, we should shift the focus of our efforts to helping
humanity die with slightly more dignity.
And, and then anyway, it goes on to talk about why it's only slightly more dignity
because people are so pathetic and they've been so lame at dealing with this.
And of course you can, you know, there's probably a lot you can say that, you
know, this was, there's somehow, this was somehow deeply in the logic of the whole
AI program for, for decades that it was, was potentially going to be very dangerous.
If you believe in Darwinism or Machiavellianism, there are no purely
self-interested actors.
And then, you know, if you get a superhuman AGI, you will never know that
it's aligned.
So there was something, you know, there was a very deep problem.
People have had avoided it for 20 years or so.
At some point, one day they wake up and the best thing we can do is, is, is just
hand out some Kool-Aid a la People's Temple to everybody or something like this.
And, and if we, and then I think, unless we just dismiss this sort of thing as, as
just, as just the kind of thing that happens in a, in a, in a post-COVID
mental breakdown world.
I found another article from Nick Bostrom who's sort of an Oxford academic.
And, you know, most of these people are sort of, I know there's, there's somehow,
they're interesting because they have nothing to say.
They're interesting because they're just mouthpieces.
There's, it's like the mouth of Sauron.
It's, it's just sort of complete sort of cogs in the machine, but they are, they're
useful because they tell us exactly where the zeitgeist is in some ways.
And, and, and this was from 2019 pre-COVID, the vulnerable world hypothesis.
And that goes through, you know, a whole litany of these different ways where, you
know, science and technology are creating all these dangers for the world.
And what do we do about them?
And it's the precautionary principle, whatever that means.
But then, you know, he has a four-part program for achieving stabilization.
And I will just read off the four things you need to do to make our world less
vulnerable and achieve stabilization in the sort of, you know, we have this
exponentiating technology where maybe it's not progressing that quickly, but
still progressing quickly enough.
There are a lot of dangerous corner cases.
You only need to do these four things to, to stabilize the world.
Number one, restrict technological development.
Number two, ensure that there does not exist a large population of actors
representing a wide and recognizably human distribution of motives.
So that's a, that sounds like a somewhat incompatible with the DEI, at least in
the, in the ideas form of diversity.
Number three, establish extremely effective preventive policing.
And number four, establish effective global governance.
Since you can't let, you know, even if there's like one little island somewhere
where this doesn't apply, it's no good.
And, and so it is basic, and this is, you know, this is the zeitgeist on the other
side.

This feels like a conflict theory on corrupted hardware argument: AI risk people think they are guided by technical considerations, but the norm encompassing their behavior is the same as with everything else in technology, smothering progress instead of earnestly seeking a way forward, navigating the dangers.

So I think the argument is not about the technical considerations, which could well be mostly accurate, but a culture of unhealthy attitude towards them, shaping technical narratives and decisions. There's been a recent post [LW · GW] making a point of the same kind.

I watched that talk on youtube. My first impression was strongly that he was using hyperbole for driving the point to the audience; the talk was littered with the pithiest versions his positions. Compare with the series of talks he gave after Zero to One was released for the more general way he expresses similar ideas, and you can also compare with some of the talks that he gives to political groups. On a spectrum between a Zero to One talk and a Republican Convention talk, this was closer to the latter.

That being said, I wouldn't be surprised if he was skeptical of any community that thinks much about x-risk. Using the 2x2 for definite-indefinite and optimism-pessimism, his past comments on American culture have been about losing definite optimism. I expect he would view anything focused on x-risk as falling into the definite pessimism camp, which is to say we are surely doomed and should plan against that outcome. By the most-coarse sorting my model of him uses, we fall outside of the "good guy" camp.

He didn't say anything about this specifically in the talk, but I observe his heavy use of moral language. I strongly expect he takes a dim view of the prevalence of utilitarian perspectives in our neck of the woods, which is not surprising because it is something we and our EA cousins struggle with ourselves [LW · GW] from time to time.

As a consequence, I fully expect him to view the rationality movement as people who are doing not-good-guy things and who use a suspect moral compass all the while. I think that is wrong, mind you, but it is what my simple model of him says.

It is easy to imagine outsiders having this view. I note people within the community have voiced dissatisfaction with the amount of content that focuses on AI stuff, and while strict utilitarianism isn't the community consensus it is probably the best-documented and clearest of the moral calculations we run.

In conclusion, Thiel's comments don't cause me to update on the community because it doesn't tell me anything new about us, but it does help firm up some of the dimensions along which our reputation among the public is likely to vary.

To me it sounds like Thiel is making a political argument against... diversity, wokeness, the general opposition against western civilization and technology... and pattern-matching everything to that. His argument sounds to me like this:

*

A true libertarian is never afraid of progress, he boldly goes forward and breaks things. You cannot separate dangerous research from useful research anyway; every invention is dual-use, so worrying about horrible consequences is silly, progress is always a net gain. The only reason people think about risks is political mindkilling.

I am disappointed that Bay Area rationalists stopped talking about awesome technology, and instead talk about dangers. Of course AI will bring new dangers, but it only worries you if you have a post-COVID mental breakdown. Note that even university professors, who by definition are always wrong and only parrot government propaganda, are agreeing about the dangers of AI, which means it is now a part of the general woke anti-technology attitude. And of course the proposed solution is world government and secret police controlling everyone! Even the Bible says that we should fear the Antichrist more than we fear Armageddon.

*

The charitable explanation is that he only pretends to be mindkilled, in order to make a political point.

It seems like a cached speech from him. He echoes the same words at the Oxford Union earlier this month. I'm unsure how much this needs updating on. He constantly pauses and is occasionally inflammatory so my impression was he was measuring his words carefully for the audience.

dude has been funding trumpism, I wouldn't really read much into what he says

edit 4mo later: https://johnganz.substack.com/p/the-enigma-of-peter-thiel