What is malevolence? On the nature, measurement, and distribution of dark traits

Edit: we are not going to technically curate this post since it's an EA Forum crosspost and for boring technical reasons that breaks the curation email. I will leave this notice up though.

Curated. This piece definitely got me thinking. If we grant that some people are unusually altruistic, empathetic, etc., it stands to reason that there are others on the other end of various distributions. And then we should also expect various selection effects on where they end up.

It was definitely a puzzle piece clicking for me that these traits can coexist with [genuine] moral conviction and that the traits are egodystonic. This rings true but somehow hasn't been an explicit model for me, but yes. Combine with this the difficult of detecting these traits and resultant behaviors...and yeah, there's stuff here to think about.

I appreciate that the authors were thorough in their research but don't especially love the format. This was pretty dense and I think a post that pulled out the most key pieces of info and argued for some conclusions would be a better read, but I much prefer this to no post.

To the extent I should add my own opinions to curation notices, my thought is this makes me update against "benefit of the doubt" when witnessing concerning behaviors. I don't know that everyone beginning to scrutinize everyone else for having big D vibes would be good, but I do think scrutinizing behaviors for being high-integrity, cooperative, transparent, etc. might actually be a good direction – with the understanding that good norms around acceptable behaviors prevents abuses that anyone (however much D) is tempted towards. Something like we want to build "robust-to-malevolence" orgs and community that make it impractical or too costly to manipulate, etc.

I'm not sure focusing on individual evil is the right approach. It seems to me that most people become much more evil when they aren't punished for it. A lot of evil is done by organizations, which are composed of normal people but can "normalize" the evil and protect the participants. (Insert usual examples such as factory farming, colonialism and so on.) So if we teach AIs to be as "aligned" as the average person, and then AIs increase in power beyond our ability to punish them, we can expect to be treated as a much-less-powerful group in history - which is to say, not very well.

I think this is an important topic and am glad to see substantial scholarship efforts on it.

Wrt AI relevance: I think the meme that it matters a lot who builds the self activating doomsday device has done potentially quite a bit of harm and may be a main contributor to what kills us.

Wrt people detecting these traits: I personally feel that the self domestication of humans has made us easier targets for such people, and undermined our ability to even think of doing anything about them. I don't think this is entirely random.

shame—no need to exacerbate such feelings if it can be avoided

Shame [LW · GW] may be an important tool that people with dark traits can leverage to overcome those traits. Exacerbating it may in some cases be salutary. 

Its interesting to note the compability of malevolence with strong moral convictions, and in fact, is a personal conundrum that I have had for long time. In certain situations, malevolent personalities are not only conducive to power, but also instrumental to obtaining it. And in a utilitarian framework, if you discount the future rewards against your present acts of malevolence and find the former >> later, then would it not be imperative for you to pursue the malevolent path? The trickiest part is when these people are proven to be effective in outcomes. In modern corporate, Jobs is the epitomous example, single-handedly leading Apple after his return to one of the most valuable companies in the world. In 20th century history, Stalin bootstrapped an industrial powerhouse from an agrarian society, eventually becoming the dominant challenger to the post-WW2 world order. Malevolence in both of them are not only correlated but seems to be instrumental with their success, and I don't think we need to reach very far for other examples, both comtemporary and historical. So I guess the conundrum is, are bad people nessescary to do good things?

Amazingly detailed article covering malevolence, interaction with power, and the other nuances! Have been thinking of exploring similar topics, and found this very helpful. Besides the identified research questions, some of which I highly agree with, one additional question I was wondering is: do self-awareness of one's own malevolence factors help one to limit the malevolence factors? if so how effective would that be? how would this change when they have power? 

I took the Dark factor test and got a very low score, but I kept second-guessing myself on the answers. I did that because I wasn't sure what my actions in a real-life scenario would be. Even though I had good intentions and I believe that other people's well-being has inherent value, I would put a high probability that I would get at least a slightly higher score if this sort of test was a real-world test that I didn't know I was taking. That makes me pessimistic about the data that the authors cite in this article. If (for example) "over 16% of people agree or strongly agree that they 'would like to make some people suffer even if it meant that I would go to hell with them'" when they know they are being tested for malevolent traits, how many people actually would do that given the choice? Also - for people who believe in hell, I hope this question is scale insensitivity problem, since infinite time being tortured seems to me to have infinite negative utility, so you would need to value harming others more than helping yourself to agree with that statement.

This seems directly important and actionable in a way that most Less Wrong posts, while well considered and informative, are not.  I have not acquitted myself well in the rationalist community so I would like to offer engagement with this article and its concepts as a surrogate for any engagement with me.