LessWrong 2.0 Reader

I think all of those words would be better used less. Really, actually, fundamentally, basically, essentially, ultimately, underneath it all, at bottom, when you get down to it, when all's said and done, these are all lullaby words, written in one's sleep, to put other people to sleep. When you find yourself writing one, try leaving it out. If the sentence then seems to be not quite right, work out what specifically is wrong with it and put it right instead of papering over the still, small voice of reason.

There is also the stereotypical "Well, actually," that so often introduces a trifling nitpick. I believe there was an LW post on that subject, but I can't find it. The search box does not appear to support multi-word strings.

Note ML systems are way more interpretable than humans, so if they are replacing humans then this shouldn't make that much of a difference.

I guess you mean here that activations and weights in NNs are more interpretable to us than neurological processes in the human brain, but if so this comparison does not seem relevant to the text you quoted. Consider that it seems easier to understand why an editor of a newspaper placed some article on the front page than why FB's algorithm showed some post to some user (especially if we get to ask the editor questions or consult with other editors).

Overall I'd guess that for WFLL1 it's closer to "replacing humans" than "replacing institutions".

Even if so (which I would expect to become uncompetitive with "replacing institutions" at some point) you may still get weird dynamics between AI systems within an institution and across institutions (e.g. between a CEO advisor AI and a regulator advisor AI). These dynamics may be very hard to interpret (and may not even involve recognizable communication channels).

How many people here remember Usenet's kill files?

Another difference beween "company is valued at X at a round" and "company is worth X" is that the valuation is effected by other terms of the deal as well. 

I agree with the idea that we empathetically simulate people, ("simulation theory") ... and I think we have innate social emotions if and only if we use that module when thinking about someone. So I brought that up here [LW · GW] as a possible path to "finding human goals" in a world-model, and even talked about how dehumanization and anthropomorphization are opposite errors, in agreement with you. :-D

I think "weakening EH" isn't quite the right perspective, or at least not the whole story, at least when it comes to humans. We have metacognitive powers, and there are emotional / affective implications to using EH vs not using EH, and therefore using EH is at least partly a decision rather than a simple pattern-matching process. If you are torturing someone, you'll find that it's painful to use EH, so you quickly learn to stop using it. If you are playing make-believe with a teddy bear, you find that it's pleasurable to use EH on the teddy bear, so you do use it.

So dehumanization is modeling a person without using EH. When you view someone through a dehumanization perspective, you lose your social emotions. It no longer feels viscerally good or bad that they are suffering or happy.

But I do not think that when you're dehumanizing someone, you lose your ability to talk coherently about their motivations. Maybe we're a little handicapped in accurately modeling them, but still basically competent, and can get better with practice. Like, if a prison guard is dehumanizing the criminals, they can still recognize that a criminal is "trying" to escape. 

I guess the core issue is whether motivation is supposed to have a mathematical definition or a normal-human-definition. A normal-human-definition of motivation is perfectly possible without any special modules, just like the definition of every other concept like "doorknob" is possible without special modules. You have a general world-modeling capability that lumps sensory patterns into categories/concepts. Then you're in ten different scenarios where people use the word "doorknob". You look for what those scenarios have in common, and it's that a particular concept was active in your mind, and then you attach that concept to the word "doorknob". Those ten examples were probably all central examples of "doorknob". There are also weird edge cases, where if you ask me "is that a doorknob?" I would say "I don't know, what exactly do you mean by that question?".

We don't need a special module to get an everyday definition of doorknobs, and likewise I don't think we don't need a special module to get an everyday definition of human motivation. We just need to be exposed to ten central examples, and we'll find the concept(s) that are most activated by those examples, and call that concept "motivation". Here's one of the ten examples: "If a person, in a psychologically-normal state of mind, says 'I want to do X', then they are probably motivated to do X."

Of course, just like doorknobs, there are loads of edge cases where if you ask a normal person "what is Alice's motivation here?" they'll say "I don't know, what exactly do you mean by that question?".

A mathematical definition of human motivation would, I imagine, have to be unambiguous and complete, with no edge cases. From a certain perspective, why would you ever think that such a thing even exists? But if we're talking about MDPs and utility functions, or if we're trying to create a specification for an AGI designer to design to, this is a natural thing to hope for and talk about.

I think if you gave an alien the same ten labelled central examples of "human motivation", plus lots of unlabeled information about humans and videos of humans, they could well form a similar concept around it (in the normal-human-definition sense, not the mathematical-definition sense), or at least their concept would overlap ours as long as we stay very far away from the edge cases. That's assuming the aliens' world-modeling apparatus is at least vaguely similar to ours, which I think is plausible, since we live in the same universe. But it is not assuming that the aliens' motivational systems and biases are anything like ours.

Sorry if I'm misunderstanding anything :-)

Publishing under a pseudonym may end up being counterproductive due to the Streisand effect. Identities behind many pseudonyms may suddenly be publicly revealed following a publication on some novel method for detecting similarities in writing style between texts.

I guess the thing I'm questioning is how well does Approval actually reflect the preferences of the electorate. Let's use Melbourne as an example: it was a safe Labor seat, but is now controlled by the Greens (there a useful summary of the seat's history at https://www.abc.net.au/news/federal-election-2016/guide/melb/ ). Under Approval, Liberal voters (who know their candidate isn't going to win, and want to deny Labor the seat) would vote for the Greens over Labor (e.g. approve Liberal and Greens); Greens voters would only vote Greens (they definitely won't vote Liberal); Labor vote Labor (as adding the Greens to the their vote only weakens their candidate). If the Greens win, Liberal voters can switch to Labor next election (causing more division); or if Labor wins can continue forcing Labor to spend resources on the Melbourne electorate—in both cases the result isn't reflecting what people actually want, rather the number of people following how-to-vote cards (are how-to-vote cards a thing in the US/outside Australia?). IRV has the same problem, except due to the problem of choosing whether to approve a candidate or not (vs giving all preferences), people are more likely to follow the how-to-vote card from the appropriate party (which is going to be governed by strategic voting, meaning you're getting more false information about the elecorate's preferences). It's also much harder to vote against someone under Approval, such as happened against Stephen Conroy ( https://www.pcworld.idg.com.au/article/355744/what_would_it_take_unseat_conroy_/ has some information, but I recall there were websites devoted to providing information about how to vote against Conroy — this would be much easier now since there were changes in the senate ballot in 2016).

Well, yes, the "policy" is meant to be entirely voluntary, since we don't have a way to enforce it. At most, there can be a mechanism of "soft enforcement" in the sense that, if you pledge to follow the "policy", this somewhat increases the trust in you, and presumably people who are sufficiently trustworthy will be privy to some unpublished knowledge and decision making. However, even then, the rules will inevitably be somewhat given to interpretation and personal judgement.

I second this sentiment.

...Although maybe I would say we need "AI infohazard guidance, options, and resources" rather than an "AI infohazard policy"? I think that would better convey the attitude that we trust each other and are trying to help each other—not just because we do in fact presumably trust each other, but also because we have no choice but to trust each other... The site moderators can enforce a "policy", but if the authors don't buy in, they'll just publish elsewhere.

I was just talking about it (in reference to my own posts) a few days ago—see here [LW(p) · GW(p)]. I've just been winging it, and would be very happy to have "AI infohazard guidance, options, and resources". So, I'm following this discussion with interest. :-)

I'm a fan of solving problems with technology. One way to solve this problem of people not liking an author's content is to allow users to put people on an ignore list (and maybe for some period of time).