LessWrong 2.0 Reader

Even for serious intellectual conversations, something I appreciate in this kind of advice is that it often encourages computational kindness [LW · GW]. E.g. it's much easier to answer a compact closed question like "which of these three options do you prefer" instead of an open question like "where should we go to eat for lunch". The same applies to asking someone about their research; not every intellectual conversation benefits from big open questions like the Hamming Question.

AFAICT, approximately every "how to be good at conversation" guide says the same thing: conversations are basically a game where 2+ people take turns free-associating off whatever was said recently. (That's a somewhat lossy compression, but not that lossy.) And approximately every guide is like "if you get good at this free association game, then it will be fun and easy!". And that's probably true for some subset of people.

But speaking for myself personally... the problem is that the free-association game just isn't very interesting.

I can see where people would like it. Lots of people want to talk to other people more on the margin, and want to do difficult thinky things less on the margin, and the free-association game is great if that's what you want. But, like... that is not my utility function. The free association game is a fine ice-breaker, it's sometimes fun for ten minutes if I'm in the mood, but most of the time it's just really boring.

For example, most US school children recite the Pledge of Allegiance every day (or at least they used to). I can remember not fully understanding what the words meant until I was in middle school, but I just went along with it. And wouldn't you know it, it worked! I do have an allegiance to the United States as a concept.

Can you explain how it caused that, and maybe what it feels like?

(I find it alarming that being forced to recite a pledge as a child can actually have that effect -- I knew humans were culturally programmable, but not that {forcing someone to say "I endorse x!" when they don't know what it means nor want to say it} every day would actually cause them to endorse x later on. Actually, I notice I'm skeptical that that was the real cause in your case; what's your reason for believing it was the cause?)

(No pressure to answer my questions of course - interpret them as statements of curiosity rather than requests in the human/social sense)

Now using julia with Claude to look at further aspects of the data, particularly in view of other commenters' observations:

Two thoughts about the role of quining in IBP:

• Quine's are non-unique (there can be multiple fixed points). This means that, viewed as a prescriptive theory, IBP produces multi-valued prescriptions. It might be the case that this multi-valuedness can resolve problems with UDT such as Wei Dai's 3-player Prisoner's Dilemma [LW(p) · GW(p)] and the anti-Newcomb problem [LW · GW]^[1]. In these cases, a particular UDT/IBP (corresponding to a particular quine) loses to CDT. But, a different UDT/IBP (corresponding to a different quine) might do as well as CDT.
• What to do about agents that don't know their own source-code? (Arguably humans are such.) Upon reflection, this is not really an issue! If we use IBP prescriptively, then we can always assume quining: IBP is just telling you to follow a procedure that uses quining to access its own (i.e. the procedure's) source code. Effectively, you are instantiating an IBP agent inside yourself with your own prior and utility function. On the other hand, if we use IBP descriptively, then we don't need quining: Any agent can be assigned "physicalist intelligence" (Definition 1.6 in the original post [LW · GW], can also be extended to not require a known utility function and prior, along the lines of ADAM [LW · GW]) as long as the procedure doing the assigning knows its source code. The agent doesn't need to know its own source code in any sense.

1. ^{^}

   @Squark [LW · GW] is my own old LessWrong account.

...Does that hold together as a potential explanation for why our universe is so young? Huh.

By "psychopath" I mean someone with the cluster B personality disorder.

There isn't a cluster B personality disorder called psychopathy. Psychopathy has never been a formal disorder and the only time we've ever been close to it is way back in 1952 when the DSM-1 had a condition called "Sociopathic Personality Disturbance". The closest you'll get these days is Antisocial Personality Disorder, which is a garbage bin diagnosis that covers a fairly broad range of antisocial behaviours, including the thing most people have in mind when they say "psychopath", but also plenty of other personality archetypes that don't seem particularly psychopathic, like adrenaline junkies and people with impulse control issues.

I misread a small bit, but I still stand by my answer. It is however still unclear to me if you value truth or not. You mention moral frameworks and opinions, but also sound like you want to get rid of biases? I think these conflict.

I guess I should give examples to show how I think:

• Suppose that climate change is real, but that the proposed causes and solutions are wrong. Or that for some problem X, people call for solution Y, but you expect that Y will actually only make X worse (or be a pretend-solution which gives people a false sense of security and which is only adopted because it signals virtue)
• Suppose that X is slightly bad, but not really worth bothering about, however, team A thinks that X is terrible and team B thinks that A is the best thing ever.
• Suppose that something is entirely up to definition, such that truth doesn't matter (for instance, if X is a mental illness or not). Also, suppose that whatever definition you choose will be perceived as either hatred or support.
• I don't think it's good to get any opinions from the general population. If actual intelligent people are discussing an issue, they will likely have more nuanced takes than both the general population and the media.
• Lets say that X personality trait is positively correlated with both intelligence and sexual deviancy. One side argues that this makes them good, another side argues that this makes them bad. Not only is this subjective, people would be confusing the utilitarian "good/bad" with the moral "good/bad" (easy example: Breaking a leg is bad, but having a broken leg does not make you a bad person). 

I think being rational/unbiased results in breaking away from socities opinions about almost everything. I also think that being biased is to be human. The least biased of all is reality itself, and a lot of people seem really keen on fixing/correcting reality to be more moral. In my worldview, a lot of things stop making sense, so I don't bother with them, and I wonder why other people are so bothered by so many things.

I might be unable to respond for a little while myself, sorry about that

Some cultures used to, and maybe still do, have a solution to the hostile telepaths problem you didn't list: perform rituals even if you don't mean them.

If a child breaks their mom's glasses, the mom doesn't care if they are really sorry or not. All she cares about is if they perform the sorry-I-broke-your-glasses ritual, whatever that looks like. That's all that's required.

The idea is that the meaning comes later. We have some non-central instances of this in Western culture. For example, most US school children recite the Pledge of Allegiance every day (or at least they used to). I can remember not fully understanding what the words meant until I was in middle school, but I just went along with it. And wouldn't you know it, it worked! I do have an allegiance to the United States as a concept.

The world used to be more full of these rituals and strategies for appeasing hostile telepaths, who just chose not to use their telepathy because everyone agreed it didn't matter so long as the rituals were performed. But the spread of Christianity and Islam has brought a demand for internalized control of behaviors to much of the world, and with it we get problems like shame and guilt.

Now I'm not saying that performing rituals even if you don't mean them is a good solution. There are a lot of tradeoffs to consider, and guilt and shame offer some societal benefits that enable higher trust between strangers. But it is an alternative solution, and one that, as my Pledge of Allegiance example suggests, does sometimes work.

Technically the connection between the computability levels of AIT (estimability, lower/upper semi-computability, approximability) and the Turing degrees has not been worked out properly. See chapter 6 of Leike's thesis, though there is a small error in the inequalities of section 6.1.2. It is necessary to connect the computability of real valued functions (type two theory of effectivity) to the arithmetic hierarchy - as far as I know this hasn't been done, but maybe I'll share some notes in a few months. 

Roughly, most classes don't have a universal distribution because they are not computably enumerable, but perhaps there are various reasons. There's a nice table in Marcus Hutter's original book, page 50.

It says that (negative log) universal probability is about the same as the (monotone) Kolmogorov complexity - in the discrete case up to a constant multiple. Basically, the Bayesian prediction is closely connected to the shortest explanation. See Li and Vitanyi's "An Introduction to Kolmogorov Complexity and its Applications."

Last question is a longer story I guess. Basically, the conditionals of the universal distribution are not lower semi-computable, and it gets even worse when you have to compare the expected values of different outcomes because of tie-breaking. But a good approximation of AIXI can still be computed in the limit.