LessWrong 2.0 Reader

Believing you may be in a simulation by the other party, or believing that some of your copies are simulated by the other party, is a type of acausal trade, and likely the simplest type of acausal trade.

For CDT agents, it is the only type of acausal trade, since the other party can only influence the CDT agent's incentives by simulating the CDT agent in a closed loop where its behaviour directly causes its outcomes.

I admit that it is new framing to describe the malign agent simulating the other agent, as offering it a bad acausal trade. I believe it summarizes what I'm saying elegantly, but of course I can talk about things without this framing if you think it's the cause of disagreement.

When reality deviations from the simulations (promises are not fulfilled), it may be too late. The promise may require the scammed agent to do something irreversible, like build a copy of the other agent, hand over all power to the other agent, and then expect to get the reward.

I agree the alien civilization does not necessarily have a high prior to begin with, but my point is that if you choose any system of priors (not just the Solomonoff universal prior) which is wrong by a few orders of magnitude, then even tons of observations and CDT won't save you.

If your prior is wrong by a few orders of magnitude, there isn't necessarily an alien civilization with a very high prior, but there very likely is. Either your prior overestimated the weight of slightly simpler universes with slightly fewer bits (maybe like the malign Solomonoff universal prior), or your prior overestimated the weight of more complex universes. And these universes might have an order of magnitude more weight than they should, and contain intelligent agents who realize you've overestimated their prior probability by an order of magnitude.

Regardless of whether you are a CDT agent or UDT agent, they will use your overestimation to create something very subjectively important to you, which you will trade much of your universe for due to you underestimating the prior probability of your actual universe.

If you are a UDT agent, they can offer the typical acausal trade. If you are a CDT agent, they can run so many simulations of you that it is functionally the same as an acausal trade. Instead of assigning too much "value" to their offer, you assign too much "probability" to their simulations of you.

Their simulations of you reward you for doing what they want, and you rationally take the reward (based on your prior), since getting that reward seems more probable than achieving anything in the case you are not in a simulation.

I occasionally get texts from journalists asking to interview me about things around the aspiring rationalist scene. A few notes on my thinking and protocols for this:

• I generally think it is pro-social to share information with relatively high-integrity journalists on topics of clear public interest.
• By-default I speak with them only if their work seems relatively high-integrity. I like journalists whose writing is (a) factually accurate, (b) boring, and (c) do not feel to me to have an undercurrent of hatred for their targets.
• By default I speak with them off-the-record, and then offer to send them write-ups of the things I said that they want to quote. This has gone quite well. I've felt comfortable speaking in my usual fashion without wondering about nailing each and every phrasing. Then I ask what they're interested in quoting, and I send them (typically a 1-2 page) google doc on those topics (largely re-stating what I already said to them, and making some improvemetns / additions). Then they tell me which quotes they want to use (typically cutting many sentences or paragraphs half-way). Then I make one or two slight edits and give them explicit permission to quote. I think this has gone quite well and they've felt my quotes were substantive and improvements.
• For the New York Times, I am currently running on the policy of "I am happy to chat off-the-record. I will also offer quotes by my usual protocol, but I will only give them conditional on you including a mention that I disapprove of the NYT's de-anonymization policies (which I bring up due to your reckless and negligent behavior that upturned the life of a beloved member of my community)." I am about to try this for the first time, and I expect they will thus not want to use my quotes, and that's fine by me.

It sounds like you're assuming the Copenhagen interpretation of QM, which is not strictly necessary. To the best of my understanding, initially but not solely from the learned hear on LW [LW · GW], QM works just fine if you just don't do that and assume the wave equations are continuous work exactly as written, everywhere, all the time, just like every other law of physics. You need a lot of information processing, but not sophisticated as described here.

There's a semi-famous, possibly apocryphal, story about Feynman when he was a student. Supposedly he learned about the double slit experiment and asked what would happen when you added a third, fourth, fifth, etc. slit. Then he asked about the limiting case - infinite slits - aka no barrier. The point was, there's never a moment when anything fundamental changes about what is being computed, whether there's a barrier with slits or not.

One last, even more speculative thought:

I recently watched (the 1997 version of) Twelve Angry Men, and found it fascinating from a Bayesian / confusion-noticing perspective.

My (spoilery) notes (cw death, suspicion, violence etc):

From all the above, I conclude:

and

However

if and only if

(trigger warning for detailed discussion of that thing I just mentioned)

I'm curious what other LW users think.

The average reader has gotten dumber and prefers shorter, simpler sentences.

I suspect that the average reader is now getting smarter, because there are increasingly ways to get the same information that require less literacy: videos, text-to-speech, Alexa and Siri, ten thousand news channels on youtube. You still need some literacy to find those resources, but it's fine if you find reading difficult and unpleasant, because you only need to exercise it briefly. And less is needed every year.

I also expect that the average reader of books is getting much smarter, because these days adults reading books are nearly always doing so because they like it.

It'll be fascinating to see whether sentence length, especially in books, starts to grow again over the coming years.

Believing you may be in a simulation is not the same as making an acausal trade, so it seems you were at least using language in an unconventional way to suggest a particular intuition about how badly things might go with a typical prior. I think that intuition is mostly wrong.

For reasons discussed in this post, the alien civilization that runs a trillion simulations of you does not necessarily have a high prior (certainly it doesn’t get a factor of a trillion versus if it had run one simulation of you).

Learning takes place as soon as the simulation model makes different predictions than the “default” model. For instance, when those promises are not in fact fulfilled (for this reason threats may remain credible for longer than incentives, since a threat might only manifest itself off policy).

Yes diminishing returns. For a purely theoretical field say pure mathematics this is even clearer. Take a fixed number of humans with a fixed intelligence (both average and outliers) then let mathematics advance. It will advance to the point that there is a vanishingly small number of people who can even understand the state of the art, with them then making little to no further progress. All else being equal the promising ideas are explored first etc.

More people just pushes you along the asymptote faster. If we took things even further, some knowledge would not even be live at one time. For example if there happened to be a few genius in a sub-field and it was in fashion, then humanities understanding of that sub field would advance, and be written down. However in 50 years or so no-one would actually understand the work, and it would require an unusual genius to be born to even take humanity knowledge to the edge of what is written down.

Given theoretical physics has such low useful bandwidth coming from experiments (Higgs is real, what else, countless null results upholding  the standard model don't help) then the same situation applies. We won't solve quantum gravity this side of the Singularity or massive human enhancement.

related: https://forum.effectivealtruism.org/posts/wAbYKBzEJBJpHMrDh/book-rec-the-war-with-the-newts-as-ea-fiction [EA · GW]

It felt productive. My day job (running a policy nonprofit) involves a lot of vibes/reacting to Current Thing and not a great deal of rigorously solving hard problems, and the exercise usefully... crystallized? a vague, vibes-based framework that I follow when I do strategy planning - set timers, generate plans, set probabilities, check surprise, go meta, iterate, etc. It's nice to have that operationalized!