LessWrong 2.0 Reader

In a study of bias blindness, experimental subjects predicted that they would have a harder time neutrally evaluating the quality of paintings if they knew the paintings were by famous artists. And indeed, these subjects exhibited the very bias they had predicted when the experimenters later tested their prediction. When asked afterward, however, the very same subjects claimed that their assessments of the paintings had been objective and unaffected by the bias.³

Also interesting to note that this mirrors how some LLMs work when asked to judge the work of acclaimed artists. If you present the work with the context of it being by someone widely recognized for their talent, the LLM will resort to praise over criticism and will return very shallow, surface-level observations and then hastily append the reassurance that of course, art is subjective and everyone has their own opinions, so really there is no criticism that can be done at all.

It is only when the text is given independent of its author that the LLM actually conducts some form of criticism upon it. And just like how it's written here, LLMs deny having bias blindness when confronted and claim to be entirely neutral in their takeaways.

The question as stated can be rephrased as "Should EAs establish a strategic stranglehold over all future resources necessary to sustain life using a series of unequal treaties, since other humans will be too short sighted/insensitive to scope/ignorant to realise the importance of these resources in the present day?"

And people here wonder why these other humans see EAs as power hungry.

How are you interpreting this fact?

Sam Altman's power, money, and status all rely on people believing that GPT-(T+1) is going to be smarter than them. Altman doesn't have good track record of being honest and sincere when it comes to protecting his power, money, and status. 

I don't think I understood this.

Sadly, in my experience, looking at the representational capacity of neural networks quickly runs into very annoying technical problems. For example, for a fixed dimension, a finite size network can fit arbitrary continuous functions to arbitrary accuracy. The construction is pathological (in particular, the network weights become impractically large), but it shows why it's hard to prove limitations in the representational capacity of neural networks.

You could limited the network parameters to have finite precision, but that makes it extremely hard to reason formally. Numerical experiments could still yield interesting results though.

Personally, I'd put my money on research into what neural networks can learn (rather than what they can represent). We're still in early stages, but things like the leap complexity seem promising to me.

I'm curious why you disagree? I'd guess you're thinking that it's necessary to keep low-quality contributions from flooding the space, and telling people how to improve when they're just way off the mark is not helpful. Or if they haven't read the FAQ or read enough posts that shouldn't be rewarded.

But I'm very curious why you disagree.

You'd need some coupling argument to know that the problems have related difficulty, so that if A is constantly saying "I don't know" to other similar problems it counts as evidence that A can't reliably know the answer to this one. But to be clear, we don't know how to make this particular protocol go through, since we don't know how to formalise that kind of similarity assumption in a plausibly useful way. We do know a different protocol with better properties (coming soon).

I don't think the mental model of "corrupted machinery" is a very useful one. Humans reason by using heuristics. Many heuristics have advantages and disadvantages instead of being perfect. Sometimes that's because they are making tradeoffs, other times it's because they have random quirks. 

Real Character was a failed experiment. I don't know how capable Ithkuil IV happens to be. 

Only fifteen comments on a post with 804 upvotes? I guess I'll add the sixteenth. Thank you for writing the Sequences; going by others' comments it seems to be quite a transformative experience to read them, and I can't wait to get started.

Also, thank you for recognizing the flaws in choosing impressive-sounding examples in your writing. That was refreshing and validating to hear.

If you publicly commit to something, taking down the written text does not constitute a de-commitment. Violating a prior commitment is unethical regardless of whether the text of the commitment is still on your website.

(Not that there's any mechanism to hold Google to its commitments, or that these commitments ever meant anything—Google was always going to do whatever it wanted anyway.)