Follow-Up to Petrov Day, 2019

Even ignoring the above problem, I'm confused why it's valuable to build up a "real tradition" among LW users, given that the wider unilateralist curse problem that our world faces can't possibly be solved by LW users having such a tradition.

A few points.

First, I don't think it's clear that in the Rationalist / EA community, there is enough reinforcement of this, and I routinely see issues with people "going rogue" and unilaterally engaging in activities that others have warned them would be dangerous, net negative, etc.

Second, it's valuable even as an exemplar; we should be able to say that there is such a community, and that they are capable of exercising at least this minimal level of restraint.

Third, I think it's clear that in the next decade the number of people in the Rationalist-sphere that are in actual positions of (relatively significant) power will continue to grow, and we have already seen some such people emerge in government and in the world of NGOs. For AI, in particular, there are many people who have significant influence in making decisions that could significantly affect Humanity's future. Their active (i.e. passive) participation in this seems likely to at least give them a better understanding of what is needed when they are faced with these choices.

I think you are overestimating the degree to which there was "no real tradeoff". I bet with you that many people felt tempted to enter the launch codes, for reasons that Said mentioned, but also as a resource to be sold (like Jeff Kauffman considered).

Like, it's obvious that if you didn't do the thing with the codes, and just had a button available to every LessWrong user, someone would have pressed it "just for the lulz", which isn't a very good reason to press the button, but as you scale the number of people who are trying to balance the pros and cons, someone will think that the lulz are more important than not pressing the button (and I think "for the lulz" can be pretty validly described as someone doing something for the public good).

I was quite uncertain whether anyone would press the button over the course of the day, and don't think that uncertainty was unjustified. And I would be surprised if you were confident at 90%+ that no one would press the button over the course of the day.

I do think that given how it worked out this time, adding a more concrete ritualized incentive seems maybe good to me, like giving $500 to a charity chosen by whoever presses the button.

I also don't think the constraint of "what matters are only considerations about the public good" is a good constraints for a tradition like this, because in reality obviously many people will not only think about the public good, which actually significantly worsens the problems that the unilateralist curse is talking about, and it seems bad to not incorporate those into this tradition (and the UC is the best abstraction we currently have for talking about this reference class).

I've had pretty much the same objection to making Petrov day about UC and overall haven't felt entirely satisfied with the counters. (Same with Petrov kind of being a unilateralist, though I moderately buy Habryka's view that he was unilateralist with respect to local command structure but non-unilateralist with respect to humanity.) I generally have felt averse to what feels like dilution of the meaning of UC to something much closer to simple defection.

However, there is a way in which there is kind of disagreement about public benefits in the recent LW button scenario if you can sell your ability to nuke the site to someone who is a troll or motivated by spite, and then use the funds for something you think is good that outweighs the harm. At last Jeff K considered this but was dissuaded by others thinking it was a bad idea.

I like your donation/anonymity suggestion. The public shaming seems like a strong disincentive that disanalogizes the scenario from real world scenarios without this element (or where action would at least draw obvious praise from many).

The other direction is having an "opposing" site that our button takes down and vice versa plus chance of false alarm to more accurately resemble the history (though that is still more what I'd call a standard game theory/cooperation/defection thing than UC).

If you had a valid code you could have just created a new account or given it to someone else . . .

You know what, I think LessWrong has collectively been worth more than $1,672 to me — especially after the re-launch. Heck, maybe even Petrov Day alone was. Incredibly insightful and potentially important.

I'd do this privately, but Eliezer wrote that story about how the pro-social people are too quiet and don't announce it. So yeah, I'm in for $1,672. Obviously, I wouldn't have done this if some knucklehead had nuked the site.

Now for the key question —

What kind of numbers do we need to put together to get another Ben Pace quality dev on the team? (And don't tell us it's priceless, people were willing to sell out your faith in humanity for less than the price of a Macbook Air! ;)

And yeah, mechanics for donating to LW specifically? Can follow up on email but I imagine it'd be good to have in this thread.

Edit: Before anyone suggests I donate to some highly-ranked charity, after I'd had some success in business I was in the nonprofit world for years and always 100% volunteer, have spent an immense amount of hours both understanding the space and getting things done, and was reasonably effective though not legendarily so or anything. By my quick back of the envelope math, I imagine any given large country's State Department would have paid $50,000 to $100,000 to have Petrov Day happen successfully in such a public way. Large corporations — I've worked with a few — maybe double that range. It was a really important thing and while "budget for hiring developers on a site that facilitates discussion of rationality" has far more nebulous and hard-to-pin-down value than some very worthy projects, it's first a threshold-break thing where a little more might produce much more results, and I think this site can be really important. If I might suggest something, though, perhaps an 80/20 eng-driven growth plan for the site that prioritizes preserving quality and norms would also make sense? We should have 10x the people here. It's very doable. I'm really busy but happy to help if I can. I think a lot of us would be happy to help make it happen if y'all would make it a little easier to know how. Something special is happening here.

Edit2: Okay, my donation is now conditional on banning whoever downvoted this ;) - just kidding. But man, what a strange mix of really great people and total idiots here huh? "I liked this a lot and I'd like to give money." WTF who does this guy think he is. Oh, me? Just someone trying to support the really fucking cool thing that's happening and asking for the logistics of doing so to be posted in case anyone else thinks it's been really cool and great for their life.

Lizardman's Constant is an observation seen in polls of unfiltered groups of people, but the people who were given the launch codes were selected for trustworthiness.