LessWrong 2.0 Reader

I think I saw someone arguing that their particular capability benchmark was good for evaluating the capability, but of limited use for training the capability because their task only covered a small fraction of that domain.

This is surprising to me. Is it possible that the kind of introspection you describe isn't whats happening here?

The first line is generic and could be used for any explanation of a pattern.
The second line might use the fact that the first line started with a "H" plus the fact that the initial message starts with "Hello" to deduce the rest.

I'd love to see this capability tested with a more unusual word than "Hello" (which often gets used as example or testing code to print "Hello World") and without the initial message beginning with the answer to the acrostic.

I'll probably play around with it a bit tomorrow.

Terrific, I'm excited to hear about your results! I definitely wouldn't be surprised if my results could be improved on significantly, although I'll be somewhat surprised if you get as high as 70% from Sonnet (I'd put maybe 30% credence on getting it to average that high in a day or two of trying).

A new air purifier is $150, but mine have been hanging around my house collecting dust and viruses; I don't think a used air purifier would have gone for $150 pre-emergency. Let's say the used value was $75. To get the same benefit as selling for $300 with no surcharge I'd need to charge $525: 2x my $300, less the $75 used value.

But I agree: the air purifiers situation is still improved when moving from the status quo (illegal) to the proposal (taxed). My point with that footnote is that the proposal still does some to discourage supply increases relative to a world without this regulation.

Always a bit embarrassing when you inform someone about their own paper ;)

My vague memories:

Thanks, a lot of that matches patterns I've seen as well. If you do anything further along these lines, I'd love to know about it!

What will you do if nobody makes a successful case?

This is a capabilities game. It is neither alignment or safety. To the degree it's forecasting, it helps cause the thing it forecasts.

Not me assuming kratom was a made-up word haha.

Awesome comic! You captured the recurring traits really really well.

I bet the person says “no”.

I agree, but I think it's important to mention issues like social desirability bias and strategic self-deception here, coupled with the fact that most people just aren't particularly good at introspection.

it’s conflicting desires, not conflicting values

It's both, our minds employ desires in service of pursuing our (often conflicting) values.

Insofar as different values are conflicting, that conflict has already long ago been resolved, and the resolution is: the action which best accords with the person’s values, in this instance, is to get up.

I'd rather put it as a routine conflict eventually getting resolved in a predictable way.

Another example: if someone says “I want to act in accordance with my values” or “I don’t always act in accordance with my values”, we recognize these as two substantive claims. The first is not a tautology, and the second is not a self-contradiction.

Indeed, but I claim that those statements actually mean "I want my value conflicts to resolve in the way I endorse" and "I don’t always endorse the way my value conflicts resolve".

This seems possible - according to this article almost every model got crushed by the easiest Stockfish: https://dynomight.net/chess/
But at the end he links to his second attempt which experimented with fine tuning and prompting, eventually getting decent performance against weak Stockfish. Actually he notes that lists of legal moves are actively harmful, which may partially explain the original example with random agents. 

A cursory glance at publications on the topic seems to indicate that LLMs can make valid moves and somehow represent the board state (which seems to follow), but are still weak players even after significant effort designing prompts.

Can you share any more definitive evidence?