LessWrong 2.0 Reader

Imagine, you have a function f that takes a_1, a_2, ..., a_n and returns b_1, b_2, ... b_m.  a_1, a_2, ..., a_n are boolean states of the known world and b_1, b_2, ... b_m boolean states of the world you don't yet know. Because f uses predicate logic internally you can't modify it to take values between 0 and 1 and have to accept that it can only take boolean values.

When you do your probability augmentation you can easily add probabilities to a_1, a_2, ..., a_n and have P(a_1), P(a_2), ..., P(a_n), as those are part of the known world.

On the other hand, how would you get P(b_1), P(b_2), ... , P(b_m)?

Follow-up question: If sunglasses are so cool, why do relatively few people wear them? Perhaps they aren't that cool after all?

I imagine a character (Alice) is constantly used as the rational actor in scenarios. We make Alice a likeable character, give her a personality, a series of events and decisions that lead her to the present.

Then, when the user has been around for a sufficient amount of time. Alice starts to slip. She makes mistakes that harm others, perhaps she has disputes with ‘Stupidus’, Maybe she just begins to say untrue things.

How long will it take a user to pry themself out of the rose tinted glasses, and update on Alice?

"if they don't understand, they will ask"

A lot of people have to write for audiences with narcissism, who never ask, because asking constitutes an admission that there might be something important that they don't understand. They're always looking for any reason, however shallow, to dismiss any view that surprises them too much.
So these writers feel like they have to pre-empt every possible objection, even the stupid ones that don't make any sense.

It's best if you can avoid having to write for audiences like that. But it's difficult to avoid them.

You should be more curious about why, when you aim at a goal, you do not aim for the most effective way.

 If we stand by while OpenAI violates its charter, it signals that their execs can get away with it. Worse, it signals that we don’t care.

what signals you send to OAI execs seems not relevant.

in the case where they really can't get away with it, e.g. where the state will really arrest them, then sending them signals / influencing their information state is not what causes that outcome.

if your advocacy causes the world to change such that "they can't get away with it" becomes true, this also does not route through influencing their information state.

OpenAI is seen as the industry leader, yet projected to lose $5 billion this year

i don't see why this would lead them to downsize, if "the gap between industry investment in deep learning and actual revenue has ballooned to over $600 billion a year"

I’m not quite sure how to answer your question, but at least I have similar feelings: that my conscientiousness is relatively low ; and that many people who do cooler stuff than me appear to be more driven, with clearer goals and a better ability to actually go and pursue them. I have various thoughts on this:

• To an extent, it’s just an impression. Many people will struggle to do more than a fraction of what they wanted, and yet because they still do quite a lot and remain very upbeat, you don’t notice than they achieve relatively little compared to what they want, but they certainly notice that. Similarly, many people are working on cool projects and apparently having tons of fun doing it, but if you asked you’d learn that they have no clue  about "what they want to do with their lives" or similar super long-term goals.
• In fact, I suspect that most people feel at least a little like that at least sometimes, and that we grossly underestimate how likely others are to feel that way.
• Yet, some people genuinely are better able to get stuff done and stay relentlessly focused on tasks than others. It can be built from habit, it can come from being really really into the one specific thing you’re working on, etc. If you struggle with that anyway, it might be something to do with mental health: famously ADHD, but also autism, or depression/anxiety can impact conscientiousness, and all these seem somewhat more common among LW readers than in the general population, so I dunno, maybe?
• And some people are also better than others at being optimistic, enthusiastic, eager to do cool stuff. I guess there are many causes, and therefore many potential ways of dealing with it, but I personally like the explanation from low self-confidence, fear of failure, etc., making you less willing to try ambitious stuff (notice how you said "it’s like they’re already taking their success for certain", when, yeah that might be the case, but it might also be that they’re aware they can fail, but they think it’s likely they could easily recover from that failure anyway). It’s quite well described (imho) here.
• But I’m pretty sure I’m covering only a relatively narrow part of the space of all the things that could be said on that topic, so I hope other people write other replies with completely different takes on the problem :-)

Thanks, it does clarify, both on separating the instantiation of an empathy mechanism in the human brain vs in AI and on considering instantiation separately from the (evolutionary or training) process that leads to it.

The argument goes through on probabilities of each possible world, the limit toward perfection is not singular. given the 1000:1 reward ratio, for any predictor who is substantially better than chance once ought to one-box to maximize EV. Anyway, this is an old argument where people rarely manage to convince the other side.

how? edit: maybe you meant just the first kind