Posts
Comments
I mean, really? It's not like we asked 270 random people. We basically asked 270 people, each one of which had already invested many hundreds of hours into participating on LessWrong, many of which I knew personally and considered close friends. Like, I agree, if you message 270 random people you don't get to expect anything from them, but the whole point of networks of trust is that you get to expect things from each other and ask things from each other.
If any of the people in that list of 270 people had asked me to spend a few minutes doing something that was important to them, I would have gladly obliged.
I want to point out a few things in particular. Firstly, the email was sent out to 270 users which from my perspective made it seem that the website was almost guaranteed to go down at some time, with the only question being when (I was aware game was played last year, but I had no memory of the outcome or the number of users).
I mean, this is a fine judgement to make, but also a straightforwardly wrong one. Last year we had ~150 people, and the site did not go down, with many people saying that we really have to add more incentives if we want to have any substantial chance of the site going down. I do think it's a pretty understandable mistake to make, but also one that is actually really important to avoid in real-life unilateralist situations.
Obviously, someone pressing the button wouldn't damage the honor or reputation of Less Wrong and so it seemed to indicate that this was just a bit of fun..
Of course it damaged our reputation! How could it not have? Being able to coordinate on this is a pretty substantial achievement, and failing on this is a pretty straightforwardly sad thing to happen. I definitely lost a good amount of trust in LessWrong, and I know of at least 10 other people who independently expressed similar things. Again, it's an understandable mistake to make, but also straightforwardly a retrospectively wrong statement.
Now Habryka is annoyed because he was trying to run a specific experiment and that experiment wasn't, "Can people who kind of care about the game, but don't care too much get fooled into taking down the site". I can understand that, I imagine that this experiment took a lot of time to set up and he was probably looking forward to it for a while.
To be clear, I think in real-life situations, people taking the consequences of their actions not seriously and treating things as just a game to be played is a serious path towards real-life risks! I don't think you destroyed the setup for this experiment at all. Indeed, someone not thinking for very long about the consequences of their actions, and taking an action with pretty serious consequences out of carelessness is one of the primary ways in which I expected the frontpage to get nuked, and was an intentional part of the test that I wanted to perform. Indeed, usually norms deteriorate by people disassociating from them, and saying that they never felt they were real in the first place, and brushing things off as inconsequential.
To clarify this some more, in a bit of a rambling way: different people have different values. While it is obvious to us in this community that destroying civilization via a nuclear war is pretty bad, there are many people who when asked "if you could wipe out humanity with the press of a button" would happily go and press it, especially if they didn't think much about it, because they have the cached belief that civilization is probably overall bad and that life would be better off without humanity. Or many people believe in an afterlife and that the apocalypse would overall cause there to be less sin, or whatever.
I assign substantial probability to the world being destroyed not by someone who wants to destroy the world, but by someone who just doesn't really think that their actions will have substantial consequences. Like Petrov could have just been a normal bureaucrat, doing his job, following the protocols that were set out by him, and it's really not hard to imagine a Petrov who just didn't really care about his job. Who realized in the abstract that nuclear war was a thing, but didn't really care about it viscerally, and when given the read from the instruments, just didn't think about it very hard and forwarded the signals to his superiors. That's how bad things happen. The most likely world in which civilization ended because Petrov didn't intervene, seems to me to be one where Petrov's attitude was overall pretty similar to your attitude here. That doesn't mean your attitude is wrong, I believe that you would actually care about the real case of the nuclear weapons, and am not at all saying that you specifically wouldn't have done the right thing if the real deal was on the line, but that the reference class of the reason why you didn't do the right thing here (by my lights) is pretty representative of the reference class by which I expect things to go wrong in reality.
Communicating and coordinating on shared priorities and values is really hard. It's a way lots of things break. In this case, we clearly failed at that. But also, that's part of the challenge of building a real and important thing. In real-life, you don't get to assume that everyone who is working with you actually really cares about avoiding nuclear war with your enemies. You don't get to assume that everyone has a shared understanding of humanity being really important to preserve, and that being cautious with humanity's future is of utmost importance. Most people don't viscerally believe those statements, so you can't just build coordination on that assumption, and if you do it anyways, I think things will fail in pretty analogous ways to how they failed on Saturday.
Added a spoiler block
I added a spoiler block
I added a spoiler block.
Epistemic status: Haven't read the book, so take it with some piles of sand.
Everything I heard about the Three Body Problem gets the sociology wrong, and seems to fail to model what humans actually do in crises. At least what I heard so far, is that it really reinforces the "humans fall into despair when faced with crises" when that's really the opposite of what we know happens in real humanitarian crises. People usually substantially increase the amount of work they do, and generally report higher levels of engagement and very rarely just give up.
See also this pretty extended critique of the Three Body Problem by Jacobian: https://putanumonit.com/2018/01/07/scientist-fiction/
Oh, yeah, totally. I had understood Zack to make an ontological argument in the first paragraph that such an entity cannot coherently exist, or alternatively that "it is not deserving of anyone's trust", both of which seem like statements that are too strong to me, and I think neither correspond to the thing you are saying here. The rest of the comment seems pretty good and I agree with most of it.
Eh, it's pretty obvious that there is a thing that corresponds to "beliefs of the rationality community" or "broad consensus of the rationality community", and also pretty obvious that those broadly get a lot of things more right than many other sources of ideas one could listen to. Of course, it might still be fine advice to try really hard to think through things for yourself, but like, calling the existence of such a thing as something that one could even hypothetically assign trust to a "delusion" just seems straightforwardly wrong.
Nah, I don't think that's a real concern. Or at least I really don't see much danger in the things in there, and have worked a lot with it in the past.
To be clear, while there is obviously some fun intended in this tradition, I don't think describing it as "just a game" feels appropriate to me. I do actually really care about people being able to coordinate to not take the site down. It's an actual hard thing to do that actually is trying to reinforce a bunch of the real and important values that I care about in Petrov day. Of course, I can't force you to feel a certain way, but like, I do sure feel a pretty high level of disappointment reading this response.
Like, the email literally said you were chosen to participate because we trusted you to not actually use the codes.
Promoted to curated! I held off on curating this post for a while, first because it's long and it took me a while to read through it, and second because we already had a lot of AI Alignment posts in the curation pipeline, and I wanted to make sure we have some diversity in our curation decisions. But overall, I really liked this post, and also want to mirror Rohin's comment in that I found this version more useful than the version where you got everything right, because this way I got to see the contrast between your interpretation and Paul's responses, which feels like it helped me locate the right hypothesis more effective than either would have on its own (even if more fleshed out).
Oh, sorry, you are totally correct. We originally accidentally linked to the 2019 post, and I fixed it this morning.
Just to clarify, which two posts would you like us to link to?
I did realize my comment came of a bit too snarky, now that I am rereading it. Just to be clear, no snark intended, just some light jest!
Sorry for the unfortunate timing of this post and Petrov Day! When the frontpage goes back up tomorrow, I will bump this post to make sure it gets some proper time on the frontpage.
Yep, seems like the Nash Equlibrium is pretty stably at everyone not pressing the button. Really needed some more incentives, I agree.
I can confirm that this message was not sent by any admin.
(For what it's worth, the post made it not at all clear to me that we were talking about a nontrivial amount of funding. I read it as just you thinking a bit through your personal finance allocation. The topic of divesting and impact investing has been analyzed a bunch on LessWrong and the EA Forum, and my current position is mostly that these kinds of differences in investment don't really make much of a difference in total funding allocation, so it doesn't seem worth optimizing much, besides just optimizing for returns and then taking those returns and optimizing those fully for philanthropic impact.)
Yeah, I agree we should add that somewhere. But don't want to clutter up the new-comment form. But I think we can make something work (currently the bottom left corner is basically completely empty, so seems like a cheap option).
We do directly expose a markdown export via the graphQL API, so that sure would be a sad roundtrip :P
Yeah, I also noticed this. Should be easy to fix. The previous placeholder text implementation was a hack to deal with the old editor, but the new editor actually allows us to do this much more elegantly, so I will see when I get around to fixing this.
Yeah, sorry, the image uploader is currently connected to the WYSIWYG editor, so for markdown you still have to host them somewhere else. The markdown editor is currently really a very minimal HTML textfield with nothing fancy happening, so it's not super obvious how to make image-upload work without a bunch of additional work.
Yep, we have markdown translation for all documents, and it should be basically fully interoperable (LaTeX is a bit janky, but I think everything else should work).
You can get it either via the API, or you can activate the markdown editor in your user profile, and then edit any post that you edited in a non-markdown format. It will offer you to display the content in Markdown instead.
Yeah, I don't know. Don't take this as a moderator warning (yet), but usually when discussions reach the "one-sentence accusation of fallacy" stage it's usually best to disengage. I haven't had time to read this whole thread to figure out exactly what happened, but I don't want either of you to waste a ton of time in unproductive discussion.
Ruby went through all the pages and decided whether to import them or not. I think it's unlikely we are going to import most of the remaining pages (some of which were pretty random and low-quality), but we will make sure they stay accessible, and if there is any individual post that isn't covered by the import that you feel is missing, there is a good chance we can just add it to the import. Which specific ones we should import is Ruby's call.
Promoted to curated: This post made a point that I have been grasping at for a while, and made it quite well. For better or for worse, I use Prisoner's Dilemma analogies at least 5 times a week, and so understanding the dynamics around those dilemmas is quite important to me. This post felt like it connected a number of ideas in this space in a way that I expect to refer back to in the future at least a few times.
Oops, you are right that for some reason we had the verbatim search feature deactivated on some of the indexes. Thank you for helping me notice this! This should now be fixed! (Because of caching it might take a while for it to work for the exact "well, actually" query, but you can try using quotes for some other queries, and it should now work as expected).
The search box does not appear to support multi-word strings.
The search box definitely supports multi-word strings. See this screenshot.
Huh, weird. I do notice that I don't like the word "really" because it is super ambiguous between being a general emphasis "this is really difficult" or being a synonym to "actually", i.e. in "do you really mean this?". The first usage feels much more common to me, i.e. in more than 80% of the sentences I could come up with the word "really" in it while I was writing the comment, I used it as general emphasis, and not as a synonym to "actually".
(Just to be clear, I understood Ruby's comment to be a joke)
Yeah, we have an open PR that adds online events to the Community section and the navigation menu on the left. Currently all events need a physical location, which is obviously pretty dumb during a global pandemic where most events are online, but it obviously made sense in the pre-pandemic world, so we've been encouraging a number of online meetup organizers to post them as normal posts instead.
Yeah, I've been thinking the same. It feels like there are a number of action-coordination dimension where we could have done substantially better (a substantial number of which will still be relevant for a while, so there is still time to improve).
Alas, the best I have usually been able to do is "<Name of the paper> replication" or "<Name of the author> replication".
That means your judgement is based on past behaviour that was already punished.
I don't understand this sentence at all. How has he already been punished for his past behavior? Indeed, he has never been banned before, so there was never any previous punishment.
significant parts of habryka's post were factually incorrect.
I am not currently aware of any factual inaccuracies, but would be happy to correct any you point out.
The only thing you pointed out was something about the word "threat" being wrong, but that only appears to be true under some very narrow definition of threat. This might be weird rationalist jargon, but I've reliably used the word "threat" to simply mean signaling some kind of intention of inflicting some kind punishment in response to some condition by the other person. Curi and other people from FI have done this repeatedly, and the "list of people who have evaded/lied/etc." is exactly one of such threats, whether explicitly labeled as such or not.
The average LessWrong user would pretty substantially regret having engaged with curi if they later end up on that list, so I do think it's a pretty concrete punishment, and while there might be some chance you are unaware of the negative consequences, this doesn't really change the reality very much that due to the way I've seen curi active on the site, engaging with him is a trap that people are likely to regret.
Yeah, almost everyone who we ban who has any real content on the site is warned. It didn't feel necessary for curi, because he has already received so much feedback about his activity on the site over the years (from many users as well as mods), and I saw very little probability of things changing because of a warning.
This is the definition that I had in mind when I wrote the notice above, sorry for any confusion it might have caused.
Additionally, I think that while a ban is sometimes necessary (e.g. harassment), a 2-year ban seems like quite a jump. I could think of a number of different sanctions, e.g. blocking someone from commenting in general; giving users the option to block someone from commenting; blocking someone from writing anything; limiting someone's authority to her own shortform; all of these things for some time.
I am not sure. I really don't like the world where someone is banned from commenting on other people's posts, but can still make top-level posts, or is banned from making top-level posts but can still comment. Both of these end up in really weird equilibria where you sometimes can't reply to conversations you started and respond to objections other people make to your arguments, and that just seems really bad.
I also don't really know what those things would have done. I don't think those things would have reduced the uncertainty of whether curi is a good fit for LessWrong super much, and feel like they could have just dragged things out into a long period of conflict that would have been more stressful for everyone.
The "blocking someone from writing anything" does feel like an option. Like, at least you can still vote and read. I do think that seems potentially like the better option, but I don't think we currently actually have the technical infrastructure to make that happen. I might consider building that for future occasions like this.
"I don't want others to update on this as being much evidence about whether it makes sense to have curi in their communities" seems a bit weird to me. "a propensity for long unproductive discussions, a history of threats against people who engage with him" and "I assign too high of a probability that old patterns will repeat themselves" seem like quite a judgement and why would someone else not update on this?
The key thing I wanted to communicate is that it seems quite plausible to me that these patterns are the result of curi interfacing specifically with the LessWrong culture in unhealthy ways. I can imagine him interfacing with other cultures with much less bad results.
I also said "I don't want others to think this is much evidence", not "this is no evidence". Of course it is some evidence, but I think overall I would expect people to update a bit too much on this, and as I said, I wouldn't be very surprised to see curi participate well in other online communities.
Yep, after we are done with the import, we are going to redirect all the pages we imported. And then probably make all the remaining pages on the old wiki read-only, so we don't have to maintain a whole separate wiki system forever.
This sentence really makes no sense to me. The proof that it can have an incentive to allow itself to be switched off even if it isn't uncertain is trivial.
Just create a utility function that assigns intrinsic reward to shutting itself off, or create a payoff matrix that punishes it really hard if it doesn't turn itself off. In this context using this kind of technical language feels actively deceitful to me, since it's really obvious that the argument he is making in that chapter cannot actually be a proof.
In general, I... really don't understand Stuart Russell's thoughts on AI Alignment. The whole "uncertainty over utility functions" thing just doesn't really help at all with solving any part of the AI Alignment problem that I care about, and I do find myself really frustrated with the degree to which both this preface and Human Compatible repeatedly indicate that it somehow is a solution to the AI Alignment problem (not only like, a helpful contribution, but both this and Human Compatible repeatedly say things that to me read like "if you make the AI uncertain about the objective in the right way, then the AI Alignment problem is solved", which just seems obviously wrong to me, since it doesn't even deal with inner alignment problems, and it also doesn't solve really any major outer alignment problems, but that requires a bit more writing).
The sailing ships one sounds fun. GWP as terrible metric also sounds interesting. The others also seem good, but those two seem marginally better.
Today we have banned two users, curi and Periergo from LessWrong for two years each. The reasoning for both is bit entangled but are overall almost completely separate, so let me go individually:
Periergo is an account that is pretty easily traceable to a person that Curi has been in conflict with for a long time, and who seems to have signed up with the primary purpose of attacking curi. I don't think there is anything fundamentally wrong about signing up to LessWrong to warn other users of the potentially bad behavior of an existing user on some other part of the internet, but I do think it should be done transparently.
It also appears to be the case that he has done a bunch of things that go beyond merely warning others (like mailbombing curi, i.e. signing him up for tons of email spam that he didn't sign up for, and lots of sockpupetting on forums that curi frequents), and that seem better classified as harassment, and overall it seemed to me that this isn't the right place for Periergo.
Curi has been a user on LessWrong for a long time, and has made many posts and comments. He also has the dubious honor of being by far the most downvoted account in all of LessWrong history at -675 karma.
The biggest problem with his participation is that he has a history of dragging people into discussions that drag on for an incredibly long time, without seeming particularly productive, while also having a history of pretty aggressively attacking people who stop responding to him. On his blog, he and others maintain a long list of people who engaged with him and others in the Critical Rationalist community, but then stopped, in a way that is very hard to read as anything but a public attack. It's first sentence is "This is a list of ppl who had discussion contact with FI and then quit/evaded/lied/etc.", and in-particular the framing of "quit/evaded/lied" sure sets the framing for the rest of the post as a kind of "wall of shame".
Those three things in combination, a propensity for long unproductive discussions, a history of threats against people who engage with him, and being the historically most downvoted account in LessWrong history, make me overall think it's better for curi to find other places as potential discussion venues.
I do really want to make clear that this is not a personal judgement of curi. While I do find the "List of Fallible Ideas Evaders" post pretty tasteless, and don't like discussing things with him particularly much, he seems well-intentioned, and it's quite plausible that he could me an amazing contributor to other online forums and communities. Many of the things he is building over on his blog seem pretty cool to me, and I don't want others to update on this as being much evidence about whether it makes sense to have curi in their communities.
I do also think his most recent series of posts and comments is overall much less bad than the posts and comments he posted a few years ago (where most of his negative karma comes from), but they still don't strike me as great contributions to the LessWrong canon, are all low-karma, and I assign too high of a probability that old patterns will repeat themselves (and also that his presence will generally make people averse to be around, because of those past patterns). He has also explicitly written a post in which he updates his LW commenting policy towards something less demanding, and I do think that was the right move, but I don't think it's enough to tip the scales on this issue.
More broadly, LessWrong has seen a pretty significant growth of new users in the past few months, mostly driven by interest in Coronavirus discussion and the discussion we hosted on GPT3. I continue to think that "Well-Kept Gardens Die By Pacifism", and that it is essential for us to be very careful with handling that growth, and to generally err on the side of curating our userbase pretty heavily and maintaining high standards. This means making difficult moderation decision long before it is proven "beyond a reasonable doubt" that someone is not a net-positive contributor to the site.
In this case, I think it is definitely not proven beyond a reasonable doubt that curi is overall net-negative for the site, and banning him might well be a mistake, but I think the probabilities weigh heavily enough in favor of the net-negative, and the worst-case outcomes are bad-enough, that on-net I think this is the right choice.
Sorry for the delay! Here it is: https://www.facebook.com/bshlgrs/posts/10218388194790943
A recently released paper that seems kind of relevant: https://www.researchgate.net/publication/337275911_Taking_a_disagreeing_perspective_improves_the_accuracy_of_people%27s_quantitative_estimates
:D Glad to hear that!
Yep, fixed. Thank you!
Judging from the URL of those links, those images were hosted on a domain that you could access, but others could not, namely they were stored as Gmail image attachments, to which of course you as the recipient have access, but random LessWrong users do not.
Oh no! The two images starting from this point are broken for me:
This is great! It's also been on my to-do list for a while to look more into how exactly contracts work and what the relevant abstractions are, and this feels like it gives me a decent framework to start from.
Hmm, I guess maybe I am more lucky. It has happened reasonably frequently to me that someone gets an ugh-field around a task that some other person doesn't find stressful (examples: organizing spreadsheets, calling businesses, having meetings, writing long explanatory blogposts).
But I do agree that reassignment is definitely much less frequent than just talking through whatever is aversive and usually one can find some other solution to the problem (in my experience pair-programming or pair-writing is often pretty successful here).