The first thing I did after reading your comment was ask my AI how Wei_Dai would have replied to this post.

My simulated Wei_Dai deleted his comment.

You are one of the people I am least confident in simulating accurately.

I do not empathize with this!

I think my eliezerish-detector is very similar to (the negation of) my GPT-ish detector. Eliezer is characteristically very coherent, so (to my eyes) his style falls into a GPT blindspot.

It was trained on the Internet (among other sources); I would be unsurprised to find out that it has read most of the Sequences.

That is correct.

The responses are cherry-picked, so this is way better than what GPT-3 is capable of. See the discussion in the downvoted subthread.

So, how were they cherry-picked exactly ? Did you take the best one out of 2 ? Out of 10 ? Out of 100 ?

I wasn't counting. Rarely more than 10. Sometimes the first answer just worked. Never did I come anywhere close to 100.

More important than throwing out answers was how often I changed prompts. Some prompts prompt much better input than others.

Did you picked half an answer, then completed with half an answer from an other prompt ?

No. Though I did sometimes keep half an answer I liked and then autocompleted from there instead of rerolling the dice from scratch. Sometimes the answer kept going and going and I truncated it early.

There were lots of edge cases. In one instance, Robin Hanson butted in, which I edited out.

…your comment with the un-curated answers. How was it obtained ?

I didn't keep the answers I threw out. The uncurated answers were created specially for that comment.

This is true inasmuch as posts written with help from GPT-3 are meant to be evidence about the capabilities of GPT-3. 

Sometimes posts are primarily intended to be fun, and success is measured by how fun they are, and then I don't care how much iteration you put into it, I just want it to be fun.

I guess this was a combo, because it's about simulation? So your question is reasonable. FYI from having played with GPT-3 myself, I assumed something like Lsusr had run multipl (3-15) iterations and partial iterations on each segment and thrown bits out and thrown whole other segments out. That said it was probably clearer to me because I've written with GPT-3 myself, and someone who hasn't could've been under the impression this was just the first pass.

I predict it will, for at least a small number of people, within the next 15 years.

I upvoted you because you caused this response to be generated [LW(p) · GW(p)], which was informing to read, and I like informative things, and whatever generates informative things can't be all bad <3

Thank you for that! :-)

...

However, I strongly disagree with your claim that LW's audience is "uninformed" except in the generalized sense that nearly all humans are ignorant about nearly all detailed topics, and: yes, nearly all of the contributors to Lesswrong are humans and thus ignorant in general by default.  

Based on my personal experiences, however, most people on Lesswrong are unusually well informed relative to numerous plausible baselines on a variety of topics relevant to good judgement and skilled prediction and computer science and similar topics.

...

Also, it seems like you used the word "alarmist" as though it deserved negative connotations, whereas I feel that having well designed methods for raising alarm and responding to real emergencies is critical to getting good outcomes in life, overall, in light of the non-Gaussian distribution of outcomes over events that is common to real world dynamical processes. So... "alarmism" could, depending on details, be good or bad or in between.

I think the generally disastrously incompetent response, by the world, to covid-19's escape from a lab, and subsequent killing of millions of people, is a vivid illustration of a lack of competent admirable "alarmism" in the ambient culture. Thus I see Lesswrong as helpfully "counter-culture" here, and a net positive.

...

Also, even if the typical reader on Lesswrong is "more than normally uninformed and unskillfully alarmist" that does not coherently imply that exposing the audience to short, interesting, informative content about AI advances is a bad idea. 

I think, in this sense, your model of discussion and decision making and debate assumes that adults can't really discuss things productively, and so perhaps everything on the internet should proceed as if everyone is incompetent and only worthy of carefully crafted and highly manipulative speech? 

And then perhaps the post above was not "cautiously manipulative enough" to suit your tastes?

Maybe I'm wrong in imputing this implicit claim to you? 

And maybe I'm wrong to reject this claim in the places that I sometimes find it?

I'd be open to discussion here :-)

Finally, your claim that "you" (who actually? which people specifically?) somehow "have an AGI death cult going here" seems like it might be "relatively uninformed and relatively alarmist"?  

Or maybe your own goal is to communicate an ad hominem and then feel good about it somehow? If you are not simply emoting, but actually have a robust model here then I'd be interested in hearing how it unpacks!

My own starting point in these regards tends to be the Bainbridge & Stark's sociological model of cults from The Future Of Religion. Since positive cultural innovation has cult formation as a known negative attractor [LW · GW] it is helpful, if one's goal is to create positive-EV cultural innovations, to actively try to detect and ameliorate such tendencies.

For example, it is useful and healthy (in my opinion) to regularly survey one's own beliefs and those of others using a lens where one ASSUMES (for the sake of exploratory discovery) that some of the beliefs exist to generate plausible IOUs for the delivery of goods that are hard-to-impossible to truly acquire and then to protect those beliefs from socially vivid falsification via the manipulation of tolerated rhetoric and social process. I regularly try to pop such bubbles in a human and gentle way when I see them starting to form in my ambient social community. If this is unwelcome I sometimes leave the community... and I'm here for now... and maybe I'm "doing it wrong" (which is totally possible) but if so then I would hope people explain to me what I'm doing wrong so I can learn n'stuff.

Every couple years I have run the Bonewits Checklist and it has never returned a score that was so high as to be worrisome (except for maybe parts of the F2F community in Berkeley two or three years on either side of Trump's election maybe?) and many many many things in modern society get higher scores, as near as I can tell :-(

For example, huge swaths of academia seem to be to be almost entirely bullshit, and almost entirely to exist to maintain false compensators for the academics and those who fund them. 

Also, nearly any effective political movement flirts with worryingly high Bonewits scores. 

Also, any non-profit not run essentially entirely on the interest of a giant endowment will flirt with a higher Bonewits score.

Are you against all non-engineering academic science, and all non-profits, and all politics? Somehow I doubt this...

In general, I feel your take here is just not well formed to be useful, and if you were going to really put in the intellectual and moral elbow grease to sharpen the points into something helpfully actionable, you might need to read some, and actually think for a while?

Finally finally, the "death cult" part doesn't even make sense... If you insist on using the noun "cult" then it is, if anything an instance of an extended and heterogeneous community opposed to dangerous robots and in favor of life.

Are you OK? A hypothesis here is that you might be having a bad time :-(

It feels to me like your comment here was something you could predict would not be well received and you posted it anyway. 

Thus, from an emotional perspective, you have earned a modicum of my admiration for persisting through social fear into an expression of concern for the world's larger wellbeing! I think that this core impulse is a source of much good in the world. As I said at the outset: I upvoted!

Please do not take my direct challenges to your numerous semi-implicit claims to be an attack. I'm trying to see if your morally praiseworthy impulses have a seed of epistemic validity, and help you articulate it better if it exists. First we learn, then we plan, then we act! If you can't unpack your criticism into something cogently actionable, then maybe by talking it out we can improve the contents of our minds? :-)

I strongly agree that the methodology should have presented up front. lsusr's response is illuminative and gives invaluable context.

But my first reaction to your comment was to note the aggressive tone and what feels like borderline name-calling. This made me want to downvote and ignore it at first, before I thought for a minute and realized that yes, on the object level this is a very important point. It made it difficult for me to engage with it.

So I'd like to ask you what exactly you meant (because it's easy to mistake tone on the internet) and why. Calling the LW audience (i.e. including me) 'alarmist and uninformed' I can understand (if not necessarily agree with) but 'an AGI death cult'? That seems to mean a cult that wants to bring about death through AGI but that's the opposite of what LW is about and so I'm pretty sure you didn't mean that. Please clarify.

Source code: None. I used OpenAI's GPT-3 playground.

"Wrote both sides": No. I only wrote my side.

"Cherry picked": Yes. I frequently generated several responses and then kept the best ones.

This comment did not deserve the downvotes; I agree with asking for disclosure.

It does deserve criticism for tone. "Alarmist and uninformed" and "AGI death cult" are distractingly offensive.

The same argument for disclosure could could have been made by "given that LW's audience has outsized expectations of AI performance" and "it costs little, and could avoid an embarrasing misunderstanding".