LessWrong 2.0 Reader

A much appreciated update, thank you!

imagine you’re playing a game where some rules encoded in words are so special that no one is allowed to touch or change them. these special rules have special properties, so no one questions them. the devil, in this case, is like a sneaky player who hides behind those rules, using them to confuse everyone else. they create a trap by making things so complicated that people can’t figure out what’s really going on. this makes it harder to escape or solve the problem, because the rules are off limits.

The atomless property and only contradictions taking a 0 value could both be consequences of the axioms in question. The Kolmogorov paper (translated from French by Jeffrey) has the details, but from skimming it I don't immediately understand how it works.

In a study of bias blindness, experimental subjects predicted that they would have a harder time neutrally evaluating the quality of paintings if they knew the paintings were by famous artists. And indeed, these subjects exhibited the very bias they had predicted when the experimenters later tested their prediction. When asked afterward, however, the very same subjects claimed that their assessments of the paintings had been objective and unaffected by the bias.³

Also interesting to note that this mirrors how some LLMs work when asked to judge the work of acclaimed artists. If you present the work with the context of it being by someone widely recognized for their talent, the LLM will resort to praise over criticism and will return very shallow, surface-level observations and then hastily append the reassurance that of course, art is subjective and everyone has their own opinions, so really there is no criticism that can be done at all.

It is only when the text is given independent of its author that the LLM actually conducts some form of criticism upon it. And just like how it's written here, LLMs deny having bias blindness when confronted and claim to be entirely neutral in their takeaways.

The question as stated can be rephrased as "Should EAs establish a strategic stranglehold over all future resources necessary to sustain life using a series of unequal treaties, since other humans will be too short sighted/insensitive to scope/ignorant to realise the importance of these resources in the present day?"

And people here wonder why these other humans see EAs as power hungry.

How are you interpreting this fact?

Sam Altman's power, money, and status all rely on people believing that GPT-(T+1) is going to be smarter than them. Altman doesn't have good track record of being honest and sincere when it comes to protecting his power, money, and status. 

I don't think I understood this.

Sadly, in my experience, looking at the representational capacity of neural networks quickly runs into very annoying technical problems. For example, for a fixed dimension, a finite size network can fit arbitrary continuous functions to arbitrary accuracy. The construction is pathological (in particular, the network weights become impractically large), but it shows why it's hard to prove limitations in the representational capacity of neural networks.

You could limited the network parameters to have finite precision, but that makes it extremely hard to reason formally. Numerical experiments could still yield interesting results though.

Personally, I'd put my money on research into what neural networks can learn (rather than what they can represent). We're still in early stages, but things like the leap complexity seem promising to me.

I'm curious why you disagree? I'd guess you're thinking that it's necessary to keep low-quality contributions from flooding the space, and telling people how to improve when they're just way off the mark is not helpful. Or if they haven't read the FAQ or read enough posts that shouldn't be rewarded.

But I'm very curious why you disagree.

You'd need some coupling argument to know that the problems have related difficulty, so that if A is constantly saying "I don't know" to other similar problems it counts as evidence that A can't reliably know the answer to this one. But to be clear, we don't know how to make this particular protocol go through, since we don't know how to formalise that kind of similarity assumption in a plausibly useful way. We do know a different protocol with better properties (coming soon).