LessWrong 2.0 Reader

There are public examples. These ones are famous because something went wrong, at least from a security perspective. Of course there are thousands of young adults with access to sensitive data who don't become spies or whistleblowers, we just don't hear about them.

• Theodore Hall, who worked at age 18 on the Manhatten Project.
• Edward Snowden who worked from age 22 for the NSA.
• Chelsea Manning who worked from age 22 as a US Army Intelligence Analyst.

Self-coordination might also be easier if AIs can recognize themselves by being good at recognizing features that survive paraphrasing and that vary very consistently between answers

Yeah, this is the case I'm thinking about. I think on longer outputs it will be possible for the AI to use a more nebulous sense of "style", rather than only relying on specific features like "using X secret phrase" (which are presumably easier to redact away).

I think this could be countered by paraphrasing at a higher level - i.e. rewriting entire paragraphs from scratch rather than just rewording sentences. I'm noting this down as a potential topic if I have time to do another project.

Thanks for the discussion, it's helped me clear up my thoughts on this topic.

I do see some security risk.

Although Trump isn't spearheading the effort I expect he will have access to the results.

Looks like this is actually a link to their second talk on May 4th 2009. Not sure if the first talk still exists.

Neither of them is exactly what you're looking for, but you might be interested in lojban, which aims to be syntactically unambiguous, and Ithkuil, which aims to be extremely information-dense as well as to reduce ambiguity. With regards to logical languages (ones which, like lojban, aim for each statement to have a single possible interpretation), I also found Toaq and Eberban just now while looking up lojban, though these have fewer speakers.

I think you're assuming these minds are more similar to human minds than they necessarily are. 

I don't assume similarity to human minds so much as you assume universal dissimilarity.
 

its not something were even aiming for with current alignment attempts.

Indeed

oh not at all, I think I'm failing on both fronts

I see. Well, in case one of them reads this: to the extent it might matter to you, as a fellow radical vegan I request you not kill anyone trying to figure out how to make the first takeover-capable AI be benevolent.

(I'm aware that humans are such that it's not obvious that designer-chosen-alignment → benevolence, and that one does actually have to reason about that instead of just assuming the hopeful option, but after having reasoned about it, I think most would choose benevolence in the end.^[1] At least, I do not expect important exceptions to be alignment researchers, as opposed to selected-for power-seekers in high positions.)

1. ^{^}

   It seems to me that actually reflectively endorsing evil (objectively describing what's happening and then being like "this is good", instead of euphemizing and coping with arguments that often secretly amount to "if you're right then the world would be really bad, and that's scary" like "but nature has it") is rare, and most people are actually altruistic on some level but conformism overrides that and animal abuse is so normal that they usually don't notice it, but then when they see basic information like "cows have best friends and get stressed when they are separated" they seemingly get empathetic and reflective (check the comments).

I disagree but not confident I could write an explanation that's both legible and not losing lots of info by simplifying into "oppressed people more likely to want to oppose oppression". When I saw the question I was looking forwards to you writing a good answer to it, actually. To hint at some starting points, why is queer anarchism a thing? How do different minds decide who they are?

i agree but think its solvable and so human content will be duper valuable. these are my additional assumptions

3. for lots of kinds of content (photos/stories/experiences/adr), people'll want it to be a living being on the other end

4. insofar as that's true^, there will be high demand for ways to verify humanness, and it's not impossible to do so (eg worldcoin)