We must be very clear: fraud in the service of effective altruism is unacceptable

LW's stated mission, IIRC, is roughly "accelerate intellectual progress on the important problems facing humanity", and I think that's a basically-accurate description of the value LessWrong provides. The primary utility of LessWrong is in cultural norms and a user base conducive to that mission.

For example, comment boxes on every frontpage post have these guidelines:

• Aim to explain, not persuade
• Try to offer concrete models and predictions
• If you disagree, try getting curious about what your partner is thinking
• Don't be afraid to say 'oops' and change your mind

LessWrong's primary utility is a culture which makes things like that part of a natural, typical communication style.

I would say that a core part of that culture is to generally try to stay on low simulacrum levels [? · GW] - talk literally and directly about our actual models of the world, and mostly not choose our words as moves in a social game. Insofar as simulacrum level 3 is a coordination strategy [LW · GW], that means certain kinds of coordination need to happen somewhere else besides LessWrong. And at current margins, that's a very worthwhile tradeoff! By default, humans tend to turn every available communication channel into a coordination battleground, so there are few spaces out there which stay at low simulacrum levels, and the marginal value of such spaces is therefore quite high. Thus the value of LessWrong: it's primarily a forum for intellectual progress, i.e. improving our own understanding, not a forum for political coordination.

Yes, and to expand only slightly: Coordinating against dishonest agents or practices is an extremely important part of coordination in general; if you cannot agree on removing dishonest agents or practices from your own group, the group will likely be worse at accomplishing goals; groups that cannot remove dishonest instances will be correctly distrusted by other groups and individuals.

All of these are important and worth coordinating on, which I think sometimes means "Let's condemn X" makes sense even though the outside view suggests that many instances of "Let's condemn X" are bad. Some inside view is allowed.

I agree that there is value in common-knowledge building, but there is a difference between doing something that feels social-miasma or simulacrum-level 3 shaped where you assert that "WE ALL AGREE THAT X" vs. you argue that something is a good idea, and you currently believe lots of other people believe the same.

I think coordinating against dishonest practices is important, but I don't think in order to do that we have to move away from just making primarily factual statements or describing your own belief state, and have to invent some kind of group-level belief.

↑ comment by evhub · 2022-11-11T02:43:52.557Z · LW(p) · GW(p)

Where do you think I make any claims that “everyone agrees X” as opposed to “I think X”? In fact, rereading my own writing, I think I was quite clear that everything therein was my view and my view alone.

Replies from: habryka4

↑ comment by habryka (habryka4) · 2022-11-11T03:49:09.148Z · LW(p) · GW(p)

I think the title is the biggest problem here:

We must be very clear: fraud in the service of effective altruism is unacceptable

There is no "I think" here, no "I believe". At least to me it feels very much like a warcry instead of a statement about the world.

to make clear that we don't support fraud in the service of effective altruism.

This is also a call to action to change some kind of collective belief. I agree that you might have meant "we individually don't support fraud", but the "in the service of effective altruism" gives me a sense of this being a reference to a collective belief of effective altruism.

I do agree you have overall been pretty clear, and I appreciate the degree to which you ground things in your personal beliefs, but I do think the title as well as the central call to action of the post goes against that.

Replies from: evhub

↑ comment by evhub · 2022-11-11T03:57:34.284Z · LW(p) · GW(p)

I agree that the title does directly assert a claim without attribution, and that it could be misinterpreted as a claim about what all EAs think should be done rather than just what I think should be done. It's a bit tricky because I want the title to be very clear, but am quite limited in the words I have available there.

I think the latter quote is pretty disingenuous—if you quote the rest of that sentence, the beginning is “I think the best course of action is”, which makes it very clear that this is a claim about what I personally believe people should do:

Right now, I think the best course of action is for us—and I mean all of us, anyone who has any sort of a public platform—to make clear that we don't support fraud in the service of effective altruism.

To be clear, “in the service of effective altruism” there is meant to refer to fraud done for the purpose of advancing effective altruism, not that we have an obligation to not support fraud and that obligation is in the service of effective altruism.

Edit: To make that last point more clear, I chainged “to make clear that we don't support fraud in the service of effective altruism” to “to make clear that we don't support fraud done in the service of effective altruism”.

Replies from: habryka4

↑ comment by habryka (habryka4) · 2022-11-11T05:03:57.784Z · LW(p) · GW(p)

I still get a strong feeling of group think every time I see the title of the post, and feel a strong sense of something invading into my thought-space in a way that feels toxic to me. For some reason this feels even stronger in the Twitter post you made:

I don't know, I just feel like this is some kind of call-to-action that is trying to bypass my epistemic defenses. 

Replies from: evhub, lahwran

↑ comment by evhub · 2022-11-11T05:08:44.952Z · LW(p) · GW(p)

The Twitter post is literally just title + link. I don't like Twitter, and don't want to engage on it, but I figured posting this more publicly would be helpful, so I did the minimum thing to try to direct people to this post.

From my perspective, I find it pretty difficult to be criticized for a “feeling” that you get from my post that seems to me to be totally disconnected from anything that I actually said.

Replies from: habryka4, WilliamKiely, TekhneMakre

↑ comment by habryka (habryka4) · 2022-11-11T05:33:19.894Z · LW(p) · GW(p)

Yeah, I am sorry. Like, I don't think I currently have the energy to try to communicate all the subtle things that feel wrong to me about this, but it adds up to something I quite dislike.

I wish I had a more crystallized quick summary that I expect to cross the inferential distance quickly, but I don't currently.

↑ comment by WilliamKiely · 2022-11-15T02:08:04.415Z · LW(p) · GW(p)

FWIW when I first saw the title (on the EA Forum) my reaction was to interpret it with an implicit "[I think that] We must be very clear: fraud in the service of effective altruism is unacceptable".

Things generally don't just become true because people assert them to be--surely people on LW know that. I think habryka's concern that not including "I think" in the title is a big deal is overblown. Dropping "I think" from the title is reasonable IMO to make the title more concise; I don't anticipate it degrading the culture of LW. I also don't see how it "bypasses epistemic defenses." If the lack of inclusion of an "I think" in your title will worsen readers' epistemics, then those readers seem to be at great risk of getting terrible epistemics from seeing any news headlines.

I don't mean to say that there's not value in using more nuanced language, including "I think" and similar qualifications to be more precise with ones words, just that I think the karma/vote ratio your post received is an over-reaction to concern about posts of your style degrading the level one "Attempt to describe the world accurately" culture of LW.

↑ comment by TekhneMakre · 2022-11-11T10:32:17.430Z · LW(p) · GW(p)

IDK where habryka is coming from, but to me the post is good, and the title is fine but gives a twinge from the words "We" and "must" and those words together. (Also the phrase "is unacceptable" also is implicitly speaking from a social-collective-objective perspective, if you know what I mean. Which is fine, but it contributes to the twinge.) Things that would, to me, decrease the twinge:

• EAs should be....
• EA must unambiguously not accept fraud...

That's a low-character-count way to be a bit more specific about who We is, to whom something Is Unacceptable. It's maybe not what you really mean, maybe you really mean something more complicated like "people who want to ambitiously do good in the world" or something, and you don't have a low-character way to say that, and "We" is aspirationally pointing at that. 

In the post you clarify

we--as people who unknowingly benefitted from it and whose work for the world was potentially used to whitewash it

and say

Right now, I think the best course of action is for us—and I mean all of us, anyone who has any sort of a public platform—to make clear that we don't support fraud done in the service of effective altruism.

Which is reasonable. The title though, by touching on the We, seems to me to "make it" a "decision that is the group's decision". 

↑ comment by the gears to ascension (lahwran) · 2022-11-12T23:00:25.782Z · LW(p) · GW(p)

it sure is a call to action, your epistemic defenses had better be good enough to figure out that it is a good one, because it is, and it is correct to pressure you about it. the fact that you're uncertain about whether I am right does not mean that I am uncertain. it is perfectly alright to say you're not sure if I'm right. but being annoyed at people for saying you should probably come to this conclusion is not reasonable when that conclusion is simply actually objectively justified - instead say you will have to think about it because you aren't sure if you see the justification yet, or something, and remember that you don't get to exclude comments from affecting your reputation, ever. if there's a way you can encode your request for courtesy about required updates that better clarifies that you are in fact willing to make updates that do turn out to be important and critical moral cooperation policy updates, then finding that method of phrasing may in fact be positive expected value for the outside world because it would help people request moral updates of each other in ways that are not pushing too hard. but it is often correct to push. do not expect people to make an exception because the phrasing was too much pressure.

I think it's useful for that as well, but I think it's primarily a place to pursue truth and rationality ("the nameless virtue").

I think the main issue here is that Less Wrong is not Effective Altruism, and that many (at a guess, most) LW members are not affiliated with EA or don't consider themselves EAs. So from that perspective, while this post makes sense in the EA forum, it makes relatively little sense on LW, and to me looks roughly like being asked to endorse or disavow some politician X. (And if I extend the analogy, it's inevitably about a US politician even though I live in another country.)

So this specific EA forum post is just a poor fit for reposting on LW without a complete rewrite.

That said, the core sentiment (ethics; fraud is bad; the ends don't justify the means; etc.) obviously does have a place on LW, so there's probably a way to write a dispassionate current-events take on e.g. this post [LW · GW] from the Sequences.

Yeah I'm definitely not sure, but I'm also doubting much anything I read on twitter will make me as sure about any of this as many others seem to be (although that thread is interesting). I have no stake in this saga; for all I know SBF is guilty of all those claims and more, but I just don't think the public info is very reliable in general and especially right at this moment.

fwiw I think gears' comment is sort of directionally right.

I think there is something important Oli and John were correct to be defending on LessWrong, and epistemic culture, and prevent things from moving towards higher simulacra levels, etc. But, also the LessWrong-style way of doing things feels kinda stunted at coordination. 

There's sort of a package deal that (much of) society offers on how to do moral coordination (see: "Simulacrum 3 As Stag-Hunt Strategy [LW · GW]"), which has a lot of problems, both epistemically, strategically and morally. My sense is that LW-er types are often trying to roll their own coordination schemes, and this will (hopefully) eventually result in something better and epistemic/lawfully grounded. But in the meantime it means there's a lot of obvious tools we don't have access to (including "interfacing morally/coordinationaly with much of the rest of the world", which is one of the key points of, well, morality and coordination). 

I endorse making the overall tradeoff, but it seems like it should come with more awareness that... like, we're making a tradeoff by having our memetic immune system trigger this hard. Not just uniformly choosing a better option. 

...

Followup note: there's a distinction between "what is right for LessWrong" and "what is right for the broader rationalsphere on EA Forum and Twitter and stuff." I think Oli had criticized both, separately. LessWrong is optimizing especially hard for epistemics/intellectual-progress as much as possible, and I think that's correct. It's less obvious to me whether it's bad that this post got 600 karma on EA Forum. In my dream world, the whole EAcosystem has better coordination and/or morality tech that doesn't route with Simulacrum 3 signaling games that are vulnerable to co-option. But I think we're still in an uncanny valley of coordination theory/practice and not sure what the right approach is for non-LW discourse, in the meantime.

I don't actually have a strong belief that the OP is good at the goal it's trying to accomplish. Just, the knee-jerk reaction to it feels like it has a missing mood to me.

I am not being asked to do something because it is moral. I am being asked to do something because it is signaling. Evan is primarily telling me I'm obligated to do PR-control for EA, but that is something I do not actually care that much about and do not believe I am obligated to do, and that's why I strong-downvoted the post.

Seems like a pretty blatant misrepresentation of what I wrote. In justifying why I think you have an obligation to condemn fraud in the service of effective altruism, I say:

Assuming FTX's business was in fact fraudulent, I think that we—as people who unknowingly benefitted from it and whose work for the world was potentially used to whitewash it—have an obligation to condemn it in no uncertain terms.

That's pretty clearly a moral argument and not about PR at all.

[oh man, my underuse of punctuation makes this hard to read. editing that now, sorry it's hard with voice recognition to add enough punctuation.]

I think I actually have a different view of moral communication than you - clarifying your stance after someone casts doubt on your personal commitment to honesty, via their demonstration that stances you have expressed yourself have lead them to concerning conclusions, warrants communicating clearly yourself by action and word that you disagree with their concerning conclusions, in order to ensure that the word bindings of your own moral philosophy are actually connected by example to the behaviors you wish your moral philosophy to refer to in the territory. map-words lose their meaning if not used to associate with the territory they're intended to; this "signaling", as you refer to it, is the task of showing-not-telling that your moral philosophy means what it's intended to after a major implementation of that moral philosophy reveals that it was corrupt.

To put it another way: someone has cast doubt on the algorithms "utilitarianism" and "ea", and so to the degree you share those algorithms, you now need to check for bugs in your implementation of them. Someone published an incident of a CVE in utilitarianism getting exploited (by SBF); it's important, after an error, to actually run the exception handler. one of the steps of the exception handler should in fact be to figure out what you wish to demand of others as part of your moral coprotection-establishment communication process; your insistence that it is not reasonable for others to have an edge of demand when they ask you to participate in clarifying ground rules of morality is understandable because of the pressure that can imply, but despite that I understand why you hesitate, I expect you to figure out how to concisely say that you are not ends-justify-the-means pure utilitarian.

For further discussion, I'd suggest reading the EA forums post - it goes into this stuff in detail in the comments and there are great discussions being had.

And to be clear, I do expect that there is some form of this moral self-description that you would already attempt to describe yourself as being bound to follow by honor and promise. I don't think that the allergic reaction folks have to someone being like "we must clarify our moral stance" is completely unwarranted, attempts to clarify moral stance overconfidently can in fact cause harm and themselves become risks to community security. But I think the level of allergic response is overcalibrated, and y'all should consider that the level of allergy to clarifying your viewpoint is in fact an example of a bad pattern in intelligent-being group social behavior. Verifying co-protection is hard, and its understandable for your immune system to have reactions to others' immune systems, but it is important to figure out how to participate in the multi-agent immune system in ways that are grounded, predictive, honest, accurate. in a significant sense, this sort of error checking is the multi-agent safety problem we need to solve for agi.

Did they have a document where they described the promise they made about what they did?

I think how bad what they did happened to be depends a lot on the specific promises they made.