LessWrong 2.0 Reader

It seems like you are trying to convince me that intelligence exists, which is obviously true and many of my comments rely on it. My position is simply that consequentialism cannot convert intelligence into powerful agency, it can only use intelligence to bypass common obstacles.

Instance:

Someone (i.e. me) should look into video self modeling (that is, recording oneself & reviewing the recording afterwards, writing down what went wrong & iterating) as a rationality technique/sub-skill of deliberate practice [LW · GW]/feedbackloop-first rationality [LW · GW].

What is the best ratio of engaging in practice vs. reviewing later? How much to spend engaging with recordings of experts?

Probably best suited for physical skills and some social skills (speaking eloquently, being charismatic &c).

Law of one player: Any specific thing you just thought of will never happen unless you (yes, you specifically) make it happen.

This was a thought-provoking essay. I hope you consider full mirroring posts here in the future as I think you'll get more engagement.

I think you're interpreting the word "offer" too literally in the statement of IIA.

Also, any agent who chooses B among {A,B,C} would also choose B among the options {A,B} if presented with them after seeing C. So I think a more illuminating description of your thought experiment is that an agent with limited knowledge has a preference function over lotteries which depends on its knowledge, and that having the linguistic experience of being "offered" a lottery can give the agent more knowledge. So the preference function can change over time as the agent acquires new evidence, but the preference function at any fixed time obeys IIA.

See my other comment [LW(p) · GW(p)]. I find it distressing that multiple people here are evidently treating acknowledgements as implying that the acknowledged person endorses the end product. I mean, it might or might be true in this particular case, but the acknowledgement is no evidence either way.

(For my part, I’ve taken to using the formula “Thanks to [names] for critical comments on earlier drafts”, in an attempt to preempt this mistake. Not sure if it works.)

Huh.  Could you try copy-pasting the spoiler block I have below and see how it comes out for you?  It looks like somehow you've ended up with quote blocks instead of spoiler blocks.  If this doesn't work, we can go harass LW tech people :P

An obvious first idea is to switch between 4.1 and 4o in the chat interface and see if the phenomenon we've been investigating occurs for both of them

Oh, switching models is a great idea. No access to 4.1 in the chat interface (apparently it's API-only, at least for now). And as far as I know, 4o is the only released model with native image generation.

• 4o -> 4.5: success (in describing the image correctly)
• 4o -> o4-mini-high ('great at visual reasoning'): success

o4-mini-high's reasoning summary was interesting (bolding mine):

The user wants me to identify both the animals and their background objects in each of the nine subimages, based on a 3x3 grid. The example seems to incorrectly pair a fox with a straw hat, but the actual image includes different combinations. For instance, the top left shows a fox in front of a straw sun hat, while other animals like an elephant, raccoon, hamster, and bald eagle are set against varying objects like bicycles, umbrellas, clapperboards, and a map. I'll make sure to carefully match the animals to their backgrounds based on this.

Chiang and Rajaniemi are on board

Let’s all keep in mind that the acknowledgement only says that Chiang and Rajaniemi had conversations with the author (Nielsen), and that Nielsen found those conversations helpful. For all we know, Chiang and Rajaniemi would strongly disagree with every word of this OP essay. If they’ve even read it.

Relevant: Alignment as a Bottleneck to Usefulness of GPT-3 [LW · GW]

between alignment and capabilities, which is the main bottleneck to getting value out of GPT-like models, both in the short term and the long(er) term?