LessWrong 2.0 Reader

In my experience, the first step in reconciling conflict is to understand one's own values, before listening to those of others.  There are multiple reasons for this step, but the one relevant to your point is that by reflecting on the tradeoffs that I accept or reject and why, I can feel secure in listening to someone else's point of view.  If their approach addresses my own concerns, then I can recognize it and that dissolves the disagreement.  If it doesn't, then I know enough about what I really want to suggest modifications to their approach that would address my concerns.  Either way, it keeps me safe from value-drift, especially on important principles like ethics.  

Just because someone else has valid concerns doesn't mean I have to give up any of my own, but it doesn't mean we're at an impasse either.  Humans have a habit of turning disagreements into false dichotomies.  When they listen to each other, the conversation becomes, "alright, I understand your concerns, but you understand why mine are more important, right?"  They are so quick to ask other people to sacrifice their values that they don't think of exploring alternative approaches, ones that can change the situation to fulfill the values of all the stakeholders.  That's what I'm working on changing.  

Does that all make sense?  

I also found this hard to parse. I suggest the following edit:

Omega will send you the following message whenever it is true: "Exactly one of the following statements is true: (1) you will not pull the lever (2) the stranger will not pull the lever " You receive the message. Do you pull the lever?

I’ve reread the comment thread and I think I’ve figured out what went wrong here. Starting from a couple posts ago, it looks like you were assuming that the reason I thought you were wrong was that I disagreed with your reasons for believing that people sometimes feel that way, and were trying to offer arguments for that point. I, on the other hand, found it obvious that the issue was that you were privileging the hypothesis, and was confused about why you were arguing the object-level premises of the post, which I hadn’t mentioned; this led me to assume it was a non-sequiter and respond with attempted clarifications of the presumed misunderstanding.
To clarify, I agree that some people view old things negatively. I don’t take issue with the claim that they do; I take issue with the claim that this is the likeliest or only possible explanation. (I do, however, think disagree-voting Anders' comment is a somewhat implausible way for someone to express that feeling, which for me is a reason to downweight the hypothesis.) I think you’re failing to consider sufficient breadth in the hypothesis-space, and in particular the mental move of assuming my disagreement was with the claim that your hypothesis is possible (rather than several steps upstream of that) is one which can make it difficult to model things accurately.

Thanks for the feedback! I’ll forward it to our team.

I think I basically agree with you that from reading the RFP page, this project doesn’t seem like a central example of the projects we’re describing (and indeed, many of the projects we do fund through this RFP are more like the examples given on the RFP page). 

Some quick reactions:

• FWIW, our team generally makes a lot of grants that are <$100k (much more so than other Open Phil teams).
• I agree the application would probably take most people longer than the description that Gavin gave on Manifund. That said, I think it’s still relatively lean considering the distribution of projects we fund, though I agree it’s slightly long for projects as small as this one (but I think Gavin could have filled it out in <<2 days). For reference, this is our form.
• Regarding turnaround time, my guess is for this project, we would have taken significantly less than 3 months, especially if they had indicated that receiving a decision was time-sensitive. For reference, the form currently says: 

We expect to make most funding decisions in 3 months or less (assuming prompt responses to any follow-up questions we may have), and we may or may not be able to accommodate requests for greater time-sensitivity. Applicants asking for over $500K should expect a decision to take the full 3 months (or more, in particularly complex cases), and apply in advance accordingly. We’ll let you know as soon as we can if we anticipate a longer than 3-month decision timeline. [emphasis in original]

• For $500k+ projects, I think a 3-month turnaround time is more defensible, though I do personally wish we generally had faster response times. 

Is separate evaluation + context-insensitive aggregation actually how helpful + harmless is implemented in a reward model for any major LLM? I think Anthropic uses finetuning on a mixture of specialized training sets (plus other supervision that's more holistic) which is sort of like this but allows the model to generalize in a way that compresses the data, not just a way that keeps the same helpful/harmless tradeoff.

Anyhow, of course we'd like to use the "beneficial for humanity" goal, but sadly we don't have access to it at the moment :D Working on it.

... No, I mean I'm discussing your statement "I'm curious why you were downvoted.... I will just assume that they're rationalists who dislike (and look down on) traditional/old things for moral reasons. This is not very flattering of me but I can't think of better explanations." I think the explanation you thought of is not a very likely one, and that you should not assume that it is true, but rather assume that you don't know and (if you care enough to spend the time) keep trying to think of explanations. I'm not taking any position on Anders' statement, though in the interests of showing the range of possibilities I'll offer some alternative explanations for why someone might have disagree-voted it.
-They might think that stuff that works is mixed with stuff that doesn't
-They might think that trial and error is not very powerful in this context
-They might think that wisdom which works often comes with reasonably-accurate causal explanations
-They might think that ancient wisdom is good and Anders is being unfairly negative about it
-They might think that ancient wisdom doesn't usually apply to real problems
Et cetera. There are a lot of possible explanations, and I think being confident it's the specific one you thought of is unwarranted.

Yes, red is perhaps the most useful to color to be able to see! That's why I chose to use it in this example.

I would argue that personal identity is based on a destructible body.

You don't think the entire western world is biased in favor of science to a degree which is a little naive? In addition to this, I think that people idolize intelligence and famous scientists, that they largely consider people born before the 1950s to have repulsive moral values, that they dislike tradition, that they consider it very important to be "educated", that they overestimate book smarts and underestimate the common sense of people living simple lives, and that they believe that things generally improve over time (such that older books are rarely worth bothering with), and I believe that social status in general make people associate with newer ideas over older ones. There's also a lot of people who have grown up around old, strict and religious people and who now dislike these. It doesn't help it that more intelligent people are higher in openness in general, and that rationalism correlates with a materialistic and mechanical worldview.

Many topics receive a lot more hostility than they deserve because of these biases, and usually because they're explained in a crazy way (for instance, Carl Jungs ideas are often called pseudoscience, and if you take the bible literally then it's clearly wrong) or because people associate them with immorality (say, the idea that casual sex is disliked by traditional because they were mean and narrow-minded, and not because casual sex caused problems for them, or because it might cause problems for us)

A lot of things are disliked or discarded despite being useful, and a lot of wisdom is in this category. All of this was packed in the message that "people dislike old things because it sounds irrational or immoral" (people tend to dislike long comments)

who seems to think that first-person perspective is illusion and only third-person perspective is real

The taste of cheese is quite real, it's just not a technical consideration relevant for chip design. Concepts worth noticing are usually meaningful in some way, but most of them are unclear and don't offer a technical foothold in any given endeavor.