LessWrong 2.0 Reader

While I was typing this, quetzal_rainbow made the same point

What have you tried?

In my experience, meditation works wonders to improve focus. Some people might recommend exercise as a way to improve energy. (Meditation and exercise also takes time, of course.)

With that in mind, the real hot possibility is the inverse of what Shai and his coresearchers did. Rather than start with a toy model with some known nice latents, start with a net trained on real-world data, and go look for self-similar sets of activations in order to figure out what latent variables the net models its environment as containing. The symmetries of the set would tell us something about how the net updates its distributions over latents in response to inputs and time passing, which in turn would inform how the net models the latents as relating to its inputs, which in turn would inform which real-world structures those latents represent.
 

Thank you, this was very much the paragraph I was missing to understand why this and other computational mechanics work might be useful for interpretability.

Ascertainment bias, of course, because we only see the cases where this did not work, and do not know exactly how many members of e.g. Delta Force were originally in doubt as to their gender. We can know it doesnt work sometimes.

I mean, the problem is if it works we won't hear about such people - they just live happily ever after and don't talk about uncomfortable period of their life.

Well  there's this frequently observed phenomenon where someone feels insecure about their gender, and then does something hypermasculine like joining Special Forces or becoming a cage fighter or something like that. They are hoping that it will make them feel confident of their birth-certificate-sex. Then they discover that nope, this does not work and they are still trans.

People should be aware that there are copious examples of people who are like -- nope, still trans --- after hoping that going hard on their birth-certificate-gender will work,

That's a nice graphical illustration of what you do. Thanks.

Is there even anybody claiming there is an experiential difference?

Yep! Ask someone with this view whether the current stream of consciousness continues from their pre-uploaded self to their post-uploaded self, like it continues when they pass through a doorway. The typical claim is some version of "this stream of consciousness will end, what comes next is only oblivion", not "oh sure, the stream of consciousness is going to continue in the same way it always does, but I prefer not to use the English word 'me' to refer to the later parts of that stream of consciousness".

This doesn't show they believe there is a difference in experience. It can be simply a different analysis of the meaning of "the current stream of consciousness continuing". That's a semantic difference, not an empirical one.

Utilitarianism is just an approximate theory. I don’t think it’s truly possible to compare happiness and pain, and certainly one can not balance the other. The Repugnant Conclusion should be that Utilitarianism is being stretched outside of its bounds. It’s not unlike Laplace’s demon in physics: it’s impossible to know enough about the system to make those sorts of choices.

You would have to look at each individual. I order to get a sufficiently detailed picture of their life, it takes a lot of time. Happiness isn’t a number. It’s more like a vector in high-dimensional space, where it can depend on any number of factors, including the mental state of one’s neighbors. Comparing requires combinatorics, so again, these hypothetical computations would blow up to impracticality.

Utilitarianism is instead an approximate theory. We are accepting the approximation that happiness and pain are a one-dimensional. It’s not real, but it makes the math easier to deal with. It’s useful, because that approximation works for most cases, without knowing the details, similar to statistical mechanics, but once you start getting into edge cases, the wheels fall off. That shouldn’t be surprising, as we are collapsing a high-dimensional vector into a single point. We’re losing fidelity, to gain computability.

I think it’s fair to say that humans are incapable of truly understanding each other. Relationships of that approximate level of knowledge take years to develop, and in most cases never do. Without that you don’t know their preferences, and without that you can’t know the vectors of their mental state, and therefore you can’t really compare for the level of detail needed to truly know if the world would be better in one state or another.

So, we approximate. Which is fine, as long as you remember that it is an approximation. I don’t think that it is possible to have a perfect ethical system with no contradictions. The best we can do is hold several ethical models, and see how they compare as a guide for our actions in an uncertain world.

Ever since NIST put a backdoor in Dual Elliptic Curve Deterministic Random Bit Generator, they have the problem that many people no longer trust them.

I guess it might be possible to backdoor AI Safety Evaluations (e.g. suppose there is some  know very dangerous thing that National Security Agency is doing with AI, and NIST deliberately rigs their criteria to not stop this very dangerous thing).

But apart from the total loss of public trust in them as an institution, NIST has done ground-breaking work in the computer security field in the past, so it wouldnt be so unusal for them to develop AI criteria.

The whole dual elliptic curve fiasco is possibly a lesson that criteria should be developed by international groups, because a single country;s standards body, like NIST, can be subverted by their spy agencies.