Simulacra and Subjectivity

Frames that describe perception can become tools for controlling perception.

The idea of simulacra has been generative here on LessWrong, used by Elizabeth in her analysis of negative feedback [LW · GW], and by Zvi in his writings on Covid-19 [LW · GW]. It appears to originate in private conversations between Benjamin Hoffman and Jessica Taylor. The four simulacra levels or stages are a conception of Baudrillard’s, from Simulacra and Simulation. The Wikipedia summary quoted on the original blog post between Hoffman and Taylor has been reworded several times by various authors and commenters.

We should approach this writing with a gut-level understanding of what motivates these authors to so passionately defend “level one” speech, and why they find simulacra to be so threatening as to describe it in Biblical terms. For Hoffman and Taylor, simulacra are “corruption of discourse,” “destroying the language,” “wireheading,” comparable to a speculative bubble, a “fall from grace.” For Elizabeth, they cause “more conflict or less information in the world,” and she “wanted a button [she] could push to make everyone go to level one all the time.” The “game” is an “enemy,” a “climate” that “punishes” honesty. Zvi’s relentlessly cynical attitude about speech he figures as simulacra is well-known to readers of his Covid-19 analysis.

It is transparently obvious to me that there are some problems in which perceptions matter enormously, others in which perceptions matter not at all, and still others in which both are equally important and dependent upon one another. Problems of perceptions operate in a complex but ultimately lawful manner. It is possible to develop a correct gears-level model of perceptions and use it to predict and control them to advantage. As Elizabeth points out, however, this perpetuates the necessity of doing so. It may be more advantageous to build relationships and institutions that at least resemble the sort that we would build if we placed a great deal of value on preserving our capacity for honesty.

Many people claim to have a bullshit detector, but nobody has one that I can borrow. Most analysis of simulacra is a description of untruths. While this can be important, it mostly motivates me to seek predictive models, identifying honest and reliable experts, make factual and logical arguments, and to advocate for perceptions that allow us to focus on solving practical problems. It’s never wise to overanalyze nonsense, and I hope that the rationalist community can continue to focus less on the thousands of things that are not and should not be, and more on what should be, what can be, and what is.

I'll add that this is a cycle; Stage 5 is Stage 1. People operating in Stage 4 are paying very little attention to objective reality. Accordingly, their objective situation is usually deteriorating; competitors operating at lower levels gradually eat their lunch without them really noticing. The cycle restarts when objective conditions deteriorate to the point that they can no longer be ignored and the complicated games of social signaling are abandoned. To extend Strawperson's comment:

Level 1: "There's a lion across the river." = There's a lion across the river.
Level 2: "There's a lion across the river." = I don't want to go (or have other people go) across the river.
Level 3: "There's a lion across the river." = I'm with the popular kids who are too cool to go across the river.
Level 4: "There's a lion across the river." = A firm stance against trans-river expansionism focus grouped well with undecided voters in my constituency.

Level 5/Level 1: "There's a lion right here" = There's a lion right here (We really should have been paying more attention to the actual lion and focus groups no longer seem important).

Promoted to curated: It took me a really long time to wrap my head around the simulacra-level idea, but after a few weeks of engaging with it passively, it clicked, and it's now something that I've been using quite a bit in my modeling, and have brought up in conversations quite a few times. 

This post is still not amazing at explaining the different levels, but it seems better than the posts that came before it. And I do think the concept is important, and worth pointing more people at, so I am curating it.

Let me know if this matches — the way I understand it is that level 3 is often about signaling belonging to a group, and level 4 is about shaping how well different belonging signal works.

So:

Level 1: "Believe all women" = If a woman accuses someone of sexual assault, literally believe her.

Level 2: "Believe all women" = I want accusations of sexual assault to be taken more seriously.

Level 3: "Believe all women" = I'm part of the politically progressive tribe that takes sexual assault seriously.

Level 4: "Believe all women" = Taking sexual assault seriously should be a more important signal of political progressivism than other issues.

Level 5: "Believe all women" = But actually take sexual assault seriously even if it becomes opposed to political progressivism because Biden.

1-3 I can wrap my head around. But can you provide some examples of what level 4 looks like?

I really like these levels. In addition to viewing this as a progression of knowledge, I think it's also possible to treat people as existing at different levels in terms of both how they can see the board, and how they act within various local incentive structures. For instance, there are people who will take everything at face value, people who will always lie, and people who view everything as a game piece.

One of the reasons it's so important to create robust sociopath repellents is that without them, the default is that people playing at higher levels will locally outcompete people playing at lower levels, whereas groups that can play internally at lower levels will outcompete groups that play internally at higher levels. You need organizations that provide incentives, deterrents, and screening mechanisms such that people at Kegan 3, 4 and 5 are all incentivized to play at level 1 internally, even if they can see the board at level 4.

I think a lot of Zvi's recent sequence on Moral Mazes [? · GW] was asking if we can overcome the local incentives that cause individuals who play at higher levels to beat individuals who play at lower levels.

This post went in a similar direction as Daniel Kokotajlo's 2x2 Simulacrum grid [LW(p) · GW(p)]. It seems to have a "medium amount of embedded worldmodel", contrasted with some of Zvi's later simulacra writing (which I think bundle a bunch of Moral-Maze-ish considerations into Simulacrum 4) and Daniel's grid-version (which is basically unopinionated about where the levels came from)

I like that this post notes the distinction between domains where Simulacrum 3 is a degenerate form of level 1, vs domains where Simulacrum 3 is the "natural" form of expression.

A topic that came up last time and I didn't see a great answer for: where does truth fit in here? Does the idea of rent-paying beliefs have any intersection with this framework?