LessWrong 2.0 Reader

I noticed the most successful people, in the sense of advancing their career and publishing papers, I meet at work have a certain belief in themselves. What is striking, no matter their age/career stage, it is like they are already taking certain their success and where to go in the future.

I also noticed this is something that people from non-working class backgrounds manage to do.

Second point. They are good at finishing projects and delivering results in time.

I noticed that this was somehow independent from how smart is someone.

While I am very good at single tasks, I have always struggled with long term academic performance. I know it is true for some other people too.

What kind of knowledge/mentality am I missing? Because I feel stuck.

yes very lukewarm take

also nice product placement nina

Here's an argument that alignment is difficult which uses complexity of value as a subpoint:

• A1. If you try to manually specify what you want, you fail.

• A2. Therefore, you want something algorithmically complex.

• B1. When humanity makes an AGI, the AGI will have gotten values via some process; that process induces some probability distribution over what values the AGI ends up with.

• B2. We don't understand how to affect the values-distribution toward something specific.

• B3. If we don't affect the value-distribution toward something specific, then the values-distribution probably puts large penalties for absolute algorithmic complexity; any specific utility function with higher absolute algorithmic complexity will be less likely to be the one that the AGI ends up with.

• C1. Because of A2 (our values are algorithmically complex) and B3 (a complex utility function is unlikely to show up in an AGI without us skillfully intervening), an AGI is unlikely to have our values without us skillfully intervening.

• C2. Because of B2 (we don't know how to skillfully intervene on an AGI's values) and C1, an AGI is unlikely to have our values.

I think that you think that the argument under discussion is something like:

• (same) A1. If you try to manually specify what you want, you fail.

• (same) A2. Therefore, you want something algorithmically complex.

• (same) B1. When humanity makes an AGI, the AGI will have gotten values via some process; that process induces some probability distribution over what values the AGI ends up with.

• B2. We'll affect the values-distribution, somehow.

• B3. The work or difficultly involved in B2 is greater, the greater the complexity of our values.

• C1. Because of A2 (our values are algorithmically complex) and B3 (a complex utility function requires more work to put in an AGI), it would take a lot of work to make an AGI pursue our values.

But these are different arguments, which make use of the complexity of values in different ways.

Thanks for the feedback! working on refining the writeup.

(2) ✅ ... The first is from Chief of Staff at Anthropic.

The byline of that piece is "Avital Balwit lives in San Francisco and works as Chief of Staff to the CEO at Anthropic. This piece was written entirely in her personal capacity and does not reflect the views of Anthropic."

I do not think this is an appropriate citation for the claim. In any case, They publicly state that it is not a matter of “if” such artificial superintelligence might exist, but “when” simply seems to be untrue; both cited sources are peppered with phrases like 'possibility', 'I expect', 'could arrive', and so on.

aiming directly for achieving some goal is not always the most effective way of achieving that goal.

there is an obvious utilitarian reason of not getting sick

I'd be interested to know what the numbers on UV in ductwork look like over the past 5 years. When I had to get a new A/C system installed in 2020, they asked whether I wanted a UVC light installed in the air handler. I had, before then, been using a 70w UVC corn light I bought on Amazon to sterilize the exterior of groceries (back when we thought fomites might be a major transmission vector), and in improvised ductwork with fans and cardboard boxes taped together.
Getting a proper bulb--an optimal wavelength source--seemed like a big upgrade. Hard to come up with quantitative efficacy numbers, but we did have a friend over for the day, who turned out to have been in the early stages of covid, without getting infected. Our first infection was years later, at a music event.

This seems very interesting, but I think your post could do with a lot more detail. How were the correlations computed? How strongly do they support PRH? How was the OOD data generated? I'm sure the answers could be pieced together from the notebook, but most people won't click through and read the code.

More reasons: people wear sunglasses when they’re doing fun things outdoors like going to the beach or vacationing so it’s associated with that, and also sometimes just hiding part of a picture can cause your brain to fill it in with a more attractive completion than is likely.