LessWrong 2.0 Reader

https://neuromorph365105.substack.com/p/the-ai-academy

You're phrasing this as though it's rebutting some remark I made; if so, I'm not sure what remark that is. I know that admissions offices are admitting students according to an intentional system.

University cohorts are basically setup to maximally benefit the people who do get admitted, not to admit the most qualified. For this purpose universities would rather have students that make the school more rewarding for other students, and not the smartest possible students. This is combined with a general tendency to do prestigous/donor wanted things. And donors want to have gone to a college that is hard to get into (even though they did not like applying). The challenge of application (and with that admit rates over yield rates) is a signal. 

I think you might fundamentally misunderstand the purpose of admission systems. To be frank, admissions is set up to benefit the university and the university alone. If getting good test scores was the bottleneck, you would see shifts in strategic behaviour until the test became mostly meaningless. For instance, you can freely retake the SAT, so if you just selected based on that people would just retake till they got a good result. 

The university has strong preferences about the distribution of students in classes. They have decided that they want different things from their applicants than "just" being good at tests.

They get this exactly through account race-based affirmative action, athletic recruitment, “Dean’s Interest List”-type tracking systems for children of donors or notable persons, and legacy preference for children of alumni, and a bunch of ill articulated selection actions in admissions offices and in other various places. 

stable-marriage system would require a national system, which would require universities as distinct and competing organizations (mostly for prestige) to coordinate for the benefit of students. They obviously should do things wiht that general description, but they tend not to. 

I miss the days where when someone asked me for my political opinion I could say something interesting, maybe even something theoretical or abstract.
Nowadays my opinions are all "tariffs bad" "free trade good" "due process good"
I'm involuntarily being NPC'd.

Things that used to be obvious as fact are now 'opinions' that must therefore be Debated, leaving true opinions in the realm of farfetched theory best left to quiet personal conversations - certainly not to real political commentary.

Not sure what Richard would say, but off the cuff, I'd distinguish 'imparting information that happens to induce guilt' from 'guilting', based on intent to cooperatively inform vs. psychologically attack.

My read of the post is that some degree of "being virtuously willing to contend with guilt as a fair emergent consequence of hearing carefully considered and selected information" is required for being well-founded or part of being a well-founded system (receive criticism without generating internal conflict, etc).

A "no" to either would mean this work falls under milling behavior, and will not meaningfully contribute toward keeping humanity safe from DeepMind's own actions.

I think it's probably possible greatly improve safety given a moderate budget for safety and not nearly enough buy in for (1) and (2). (At least not enough buy-in prior to a large incident which threatens to be very costly for the organization.)

Overall, I think high quality thinking about AI safety seems quite useful even if this level of buy-in is unlikely.

(I don't think this report should update us much about having buy-in needed for (1)/(2), but the fact that it could be published at all in it's current form is still encouraging.)

A sufficiently high band in the CCP could work. 

I don't know, maybe nothing. (I just meant that on current margins, maybe the quality of the safety team's plans isn't super important.)

Cool work. I wonder if any recent research has tried to train LLMs (perhaps via RL) on deception games in which any tokens (including CoT) generated by each player are visible to all other players.

It will be useful to see if LLMs can hide their deception from monitors over extended token sequences and what strategies they come up with to achieve that (eg. steganography).