LessWrong 2.0 Reader

I know this comment was 17 years ago, but nearly half of modern US politics in 2024 is strongly influenced by the idea "don't trust experts". I have listened to a lot of cranks (ex), and the popular ones are quite good at what they do, rhetorically, so most experts have little chance against them in a debate. Plus, if the topic is something important, the public was already primed to believe one side or the other by whatever media they happen to consume, and if the debate winner would otherwise seem close (to a layman) then such preconceptions will dominate perceptions about who won.

One thing these winning cranks take advantage is that the audience (like the cranks themselves!) tend to believe they are "knowledgeable enough to understand all the technical arguments" when they're not. Also, most experts have much worse teaching skills than these cranks (who are popular for a reason) so the expert tends to provide worse explanations, while also overlooking key counterarguments they could make against the crank's case.

So: there's some truth in this article, but I wouldn't want the public to read it.

I called this a long time ago, though I'm not sure I wrote it down anywhere. But it doesn't mean faster is safer. That's totally wrong - scaling A.I. actually motivates building better GPU's and energy infrastructure. Regardless of compute overhang there was always going to be a "scaling up" period, and the safety community is not prepared for it now.

This is importantly wrong because the example is in the context of an analogy

getting some pawns : Stockfish : Stockfish's goal of winning the game :: getting a sliver of the Sun's energy : superintelligence : the superintelligence's goals

The analogy is presented as forceful and unambiguous, but it is not. It's instead an example of a system being grossly more capable than humans in some domain, and not opposing a somewhat orthogonal goal

I think there could be an interesting game that leaned even harder into the creation and testing of contracts and rules. What if players each controlled a state within a nation, and had to agree on Federal laws. They could also make state laws. After a round of law making and voting, a computer simulation of economy and population growth and whatnot would play out. The effects of the interplay of all the laws would become obvious. Then, another round of law making.

My favourite way to deal with late assignments is to have the option to not do them in the first place. My psychology professor gave each of us three options: 1) do two assignments that count X%, and have the tests count Y% (default option); 2) do one assignment, and have the tests count proportionally more; 3) do zero assignments, and have the tests count for everything.

My ADHD ass chose the last option, and it was the only course that I scored 100% in.

I might be biased about this approach by the fact that I was the student in this scenario, but I'm willing to bet the professor was also happy with this arrangement, since it meant he had fewer assignments to grade.

That professor also had a policy about grading tests, where if almost everybody got a question wrong, he would drop that question from scoring altogether, but still score it for the few people who got it right, who would thus have the possibility of scoring more than 100% (i.e. 50/49).

He also was the only professor I had who always made sure to upload class materials at least a week before the associated class (if not earlier), and always organized them well and labelled them clearly, and at the beginning of each class gave everyone printouts of that day's PowerPoint to write notes on.

I really, really, really liked that professor.

Updated thoughts

Optimal Weave I played a trial run of a printed copy of Optimal Weave. It felt like it had a core of something workable, but needed polishing in order to be something I could pull others into playing with me and have them be satisfied by the experience. I should write up more thoughts on this.

Other Games I've been thinking about other games which could be modified to be cohabitative. It's fascinating how an iterated prisoners' dilemma naturally arises out of such patterns. For example, Go. Play it as a cohabitative game by giving every player a persistent record of territory won. The game ends when both players pass consecutively. You can examine your territory-per-game score. Two players who manage to agree to split the board perfectly evenly, and play against each other often, will obtain a good ratio. But the temptation to defect will be there. If you believe yourself to be a stronger player than the other, this temptation gets stronger. Struggle burns territory (intersections covered by either player's stones count for nobody, only empty points count for territory). Maybe you negotiate by telling the other player that you think you could win 60% of the board as territory, and if they believe you, then you both establish that boundary efficiently?

How about a similar iterated resource/territory score based diplomacy?

I think it would be really interesting to test AI on such games, over millions of iterations, and see which strategies did well.

I think I was thinking:

1. The war room transcripts will leak publicly
2. Generals can secretly DM each other, while keeping up appearances in the shared channels
3. If a general believes that all of their communication with their team will leak, we're be back to a unilateralist's curse situation: if a general thinks they should nuke, obviously they shouldn't say that to their team, so maybe they nuke unilaterally
   1. (Not obvious whether this is an infohazard)
4. [Probably some true arguments about the payoff matrix and game theory increase P(mutual destruction). Also some false arguments about game theory — but maybe an infohazard warning makes those less likely to be posted too.]

(Also after I became a general I observed that I didn't know what my "launch code" was; I was hoping the LW team forgot to give everyone launch codes and this decreased P(nukes); saying this would would cause everyone to know their launch codes and maybe scare the other team.)

I don't think this is very relevant to real-world infohazards, because this is a game with explicit rules and because in the real world the low-hanging infohazards have been shared, but it seems relevant to mechanism design.

I'd also love to have access!

The other part of "this is certainly not how it works" is that yes, in part of cases you are going to be able to predict "results on this benchmark will go up 10% with such-n-such increase in compute" but there is no clear conversion between benchmarks and ability to take over the world/design nanotech/insert any other interesting capability.

Your access should be activated within 5-10 minutes. Look for the button in the bottom right of the screen.