LessWrong 2.0 Reader

On the same day I posted my original comment I later realized what I said was wrong, and I'll soon edit it to reflect that.

Regarding your response: I think I have a guess on the important difference you're referring to. They both seem to be equivalent to an Incubator Sleeping Beauty, but see consideration 2 bellow.

1

I think another useful (at least to me) way of seeing/stating what is happening here is that all of the following sentences are true, in an ISB and your two experiments:

• The probability (from an external POV) that the coin was Heads or Tails is 1/2.
• Each individual "me" (however many there are) will experience the coin being Heads or Tails one half of the time.
• If every "me" always predicts Heads, all of my mes will be correct 1/3 of the time and wrong 2/3 of the time. Each individual me will only be able to notice this if we get together after the experiments to compare notes.

I think this is equivalent to the difference in scoring methods you used in Anthropical Motte and Bailey in two versions of Sleeping Beauty.

2

With the two experiments in your response, the only significant difference I can see is that, in experiment 1, there are two identical copies of me, and in 2, there are two different people. I don't know if you're implying that this changes any probabilities, and I'm not sure that it does. What I can say is that experiment 2 is, AFAICT, equivalent to the Doomsday argument in it's setup: two theories on the amount of people that will come to be, with 1:1 prior odds between them, and the question is "should you update on your existing". I have more reflection to make before I can give any firm answer here, but I'm inclined toward "no".

3

I have a feeling that, even though we agree with the final probabilities, we disagree on some of the internal details of how these experiments work. What would you say is the significant difference between the experiments, and does it change the numbers?

P(GPT-5 Release)

What is the probability that OpenAI will release GPT-5 before the end of 2025? "Release" means that a random member of the public can use it, possibly paid.

Does this require a product called specifically "GPT-5"? What if they release e.g "OpenAI o2" instead, and there will never be something called GPT-5?

Number of Current Partners
(for example, 0 if you are single, 1 if you are in a monogamous relationship, higher numbers for polyamorous relationships)

This is a confusing phrasing. If you have 1 partner, it doesn't mean your relationship is monogamous. A monogamous relation is one in which there is a mutually agreed understanding that romantic or sexual interaction with other people is forbidden. Without this, your relationship is not monogamous. For example:

• You have only one partner, but your partner has other partners.
• You have only one partner, but you occasionally do one night stands with other people.
• You have only one partner, but both you and your partner are open to you having more partners in the future.

All of the above are not monogamous relationships!

And also, I do not personally want to be running into any writing that AI had a hand in.

(My guess is the majority of posts written daily on LW are now written with some AI involvement. My best guess is most authors on LessWrong use AI models on a daily level, asking factual questions, and probably also asking for some amount of editing and writing feedback. As such, I don't think this is a coherent ask.)

I don't think this kind of surface-level naive popularity optimization gives rise to a good comms strategy. Evolution is true, and mostly we should focus on making arguments based on true premises. 

Starting your introduction with

A few million years ago, something very strange happened. 

seems likely to turn away roughly 40% of the US population, along with any leaders who need the goodwill of that 40% to keep their positions.

The point I understand you to be making (tripling the brain size of a chimp gives more than triple the impact) could be easily made without this sentence to introduce it. Given the importance of the US in addressing the existential threat of AI, and assuming one of the goals of this article is to be a general call to action, potentially alienating nearly half the target audience seems counterproductive.

I'm planning to run the unofficial LessWrong Community Census again this year. There's a post with a link to the draft and a quick overview of what I'm aiming for here [LW · GW], and I'd appreciate comments and feedback. In particular, if you

• Have some political questions you want to get into detail with or
• Have experience or opinions on the foundational skills of rationality and how to test them on a survey

then I want to hear from you. I care a lot about rationality skills but don't know how to evaluate them in this format, but I have some clever ideas if I had a signal I could sift out of the survey. I don't care about politics, but lots of people do and I don't want to spoil their fun.

You can also propose other questions! I like playing with survey data :) 

I should probably get into the habit of splitting my comments up. I keep making multiple assertions in a single response, which means when people add (dis)agreement votes I have no idea which part(s) they're (dis)agreeing with.

Hi Aksh, I'm looking to reproduce the CAI process in the HuggingFace tutorial myself. Did you publish your code for this? I'm having a hard time getting LLMSwarm to work so I'm eager to see your setup.

Typo in the linked document:

There is no one is coming to save us.