LessWrong 2.0 Reader

If we know nothing about them, the statements could equally be true or false, and positively or negatively dependent. The same argument which makes us assume 50% probability to individual statements would also make us assume independence between statements. The possibilities cancel out, so to speak.

Computationally tractable is Yudkowsky's framing and might be too limited. The kind of thing I believe is for example, an animal without a certain brain complexity will tend not to be a social animal and is therefore unlikely to have the sort of values social animals have. And animals that can't do math aren't going to value mathematical aesthetics the way human mathematicians do.

Relativity to Newtonian mechanics is a warp in a straightforward sense. If you believe the layout of a house consists of some rooms connected in a certain way, but there are actually more rooms connected in different ways, getting the maps to line up looks like a warp. Basically, the closer the mapping is to a true homomorphism (in the universal algebra sense), the less warping there is, otherwise there are deviations intuitively analogous to space warps.

Right, but... it's a combined issue of tokenization and data cleaning. I agree you can't properly do one without the other. I just think you should do both! 

Clearly the root cause is the messy data, and once you've cleaned the data it should be trivial to fix the tokenizer.

I checked Lunary's predicted tokenization for Gemma, Mistral, and Grok as well. These three models all seem to handle number tokenization cleanly, a single digit at a time. This suggests that there isn't some well considered valid reason that digits need uneven lumpy tokenization, that it's really just a mistake.

I am unsure whether I've grasped the mechanism behind this inner work stuff. You feel your limbs weird, and what? What implication are you trying to point to? How should your experience or insight contribute to rationality and moral philosophy? Yes, we have inherent biases but, as far as I've managed to get acquainted with the rationality discourse, they are not resolved by merely being aware of them or expanding consciousness. I cannot clearly see the added value of this inner work. Why is experimental psychology insufficient? Thanks for your answer.

I strongly believe that step 1 is sufficient or almost sufficient for step 2, i.e., that it's impossible to give an adequate account of human phenomenology without figuring out most of the computational aspects of consciousness.

Apologies for nitpicking, but your strong belief that step 1 is (almost) sufficient for step 2 would be more faithfully re-phrased as: it will (probably) be possible/easy to give an adequate account of human phenomenology by figuring out most of the computational aspects of consciousness. The way you phrased it (viz., "impossible...without") is equivalent to saying that step 1 is necessary for step 2, an importantly different claim (on this phrasing, something besides the computational aspects may be required). Of course, you may think it is both necessary and sufficient, I'm just pointing out the distinction. 

Why would pulling the lever make you more responsible of the outcome than not pulling the lever? Both are options you decide to take once you have observed the situation.

Her main motivation was to make Thomas’ ideas more accessible to people.

thanks Chi!!

Right. Pure ignorance is not evidence.

Close, but not quite what this post is saying. The core claim in this post is that our brains model the world as though there's a thing called "our values", and tries to learn about those values in the usual epistemic way.

That does not necessarily imply that there actually is a thing called "our values". A fairly precise analogy: consider a Bayesian reasoner which is hardcoded to believe it's in the Game of Life, but is in fact hooked up to my webcam. That reasoner will, for instance, try to learn about the positions of rocks and oscillators and gliders, but there are not actually any rocks or oscillators or gliders in its environment.

You are exactly right that if I say “I’m learning about blah”, then I’m strongly implying that there’s a fact-of-the-matter about blah, and that I think I'm updating in such a way that my beliefs approximately-monotonically converge towards the fact-of-the-matter. But whether there actually exists a fact-of-the-matter, and whether I'm actually converging towards it, is a separate question from whether I think there is, or whether I model the world as though there's a fact-of-the-matter.

And that's the main next question which this post sets up: when, and to what extent, is there a referent of the "values" which humans think they're learning about? Insofar as this all works like ordinary epistemics, we should be able to answer that question in much the same way that we answer any other question about when a human who thinks they're learning about blah is learning about an actual thing. Just like the example of the position of the dog in my apartment.