LessWrong 2.0 Reader

This is neat.

Did you write all that or who did?

One underlying idea comes from how AI mis-generalization is intended to work. If superintelligent AI systems are misaligned, does this misalignment look like an inaccurate generalization from what their overseers wanted or a deceptive goal that's entirely unrelated to anything their overseers intended to train. Levels 1-4 vs levels 5+, [LW · GW] more or less. 

If the AI that's misaligned ends up 'egregiously' misaligned and doesn't care at all about anything human valuable, as Eliezer thinks, it places zero terminal value on human welfare and only trade, threats or compromise would get it to be nice.

If it's super-intelligent and you aren't, none of those apply. Hence, nothing is left for humans. If the AI is misaligned in any other way that's not an entirely arbitrary value system, then it values human survival at least a bit.

Could there not be AI value drift in our favor, from a paperclipper AI to a moral realist AI?

But some of the writing in those old sacred texts is actually really good.

The Halo Effect (pun not intended) is very strong here.

I seldom see people who say thibnngs like this also say that the writing in the Koran is actually really good, for some reason. And not because there's some difference in how good the writing in the Koran and in the Bible is.

The "one weird trick" to getting the right answers is to discard all stuck, fixed points. Discard all priors and posteriors. Discard all aliefs and beliefs. Discard worldview after worldview. Discard perspective. Discard unity. Discard separation. Discard conceptuality. Discard map, discard territory. Discard past, present, and future. Discard a sense of you. Discard a sense of world. Discard dichotomy and trichotomy. Discard vague senses of wishy-washy flip floppiness. Discard something vs nothing. Discard one vs all. Discard symbols, discard signs, discard waves, discard particles. 

All of these things are Ignorance. Discard Ignorance.

Is this the same principle as "non-attachment"?

I'm not sure about the rest of it, but this caught my eye:

if moral realism was true, and one of the key roles of religion was to free people from trapped priors so they could recognize these universal moral truths, then at least during the founding of religions, we should see some evidence of higher moral standards before they invariably mutate into institutions devoid of moral truths.

I had a similar thought, and was trying to figure out if I could find a single good person to formally and efficiently coordinate with in a non-trivial pre-existing institution full of "safely good and sane people".

I'm still searching. If anyone has a solid lead on this, please DM me, maybe?

Something you might expect is that many such "hypothetically existing hypothetically good people" would be willing to die slightly earlier for a good enough cause (especially late in life when their life expectancy is low, and especially for very high stakes issues where a lot of leverage is possible) but they wouldn't waste lives, because waste is ceteris paribus bad, and so... so... what about martyrs who are also leaders?

This line of thinking is how I learned about Martin The Confessor, the last Pope to ever die for his beliefs.

Since 655 AD is much much earlier than 2024 AD, it would seem that Catholicism no longer "has the sauce" so to speak?

Also, slightly relatedly, I'm more glad that I otherwise might be that in this timeline the bullet missed Trump. In other very nearby timelines I'm pretty sure the whole idea of using physical courage to detect morally good leadership in a morally good group would be much more controversial than the principle is here, now, in this timeline, where no one has trapped priors about it that are being actively pumped full of energy by the media, with the creation of new social traumas, and so on...

...not that elected secular leaders of mere nation states would have any obvious formal duties to specifically be the person to benevolently serve literally all good beings as a focal point.

To get that formula to basically work, in a way that it kinda seems to work with US elections, since many US Presidents are assassinated in ways they could probably predict were possible (modulo this currently only working within the intrinsically "partial" nature of US elections, since these are merely elections for the leader of a single nation state that faces many other hostile nation states in a hobbesian world of eternal war (at least eternal war... so far! [? · GW]) ) I think one might need to hold global elections?

And... But... And this... this seems sorta do-able?!? Weirdly so!

We have the internet now. We have translation software to translate all the political statements into all the languages. We have internet money that could be used to donate to something that was worth donating to.

Why not create a "United Persons Alliance" (to play the "House of Representatives" to the UN's "Senate"?) and find out what the UPA's "Donation Weighted Condorcet Prime Minister" has to say?

I kinda can't figure out why no one has tried it yet.

Maybe it is because, logically speaking, moral realism MIGHT be true and also maybe all humans are objectively bad?

If a lot of people knew for sure that "moral realism is true but humans are universally fallen" then it might explain why we almost never "produce and maintain legibly just institutions".

Under the premises entertained here so far, IF such institutions were attempted anyway, and the attempt had security holes [? · GW], THEN those security holes would be predictably abused and it would be predictably regretted by anyone who spent money setting it up, or trusted such a thing.

So maybe it is just that "moral realism is true, humans are bad, and designing secure systems is hard and humans are also smart enough to never try to summon a real justice system"?

Maybe.

This does seem kind of correct to me?

Maybe you could see the fixed points that OP is pointing towards as priors in the search process for frames.

Like, your search is determined by your priors which are learnt through your upbringing. The problem is that they're often maladaptive and misleading. Therefore, working through these priors and generating new ones is a bit like relearning from overfitting or similar.

Another nice thing about meditation is that it sharpens your mind's perception which makes your new priors better. It also makes you less dependent on attractor states you could have gotten into from before since you become less emotionally dependent on past behaviour. (there's obviously more complexity here) (I'm referring to dependent origination for you meditators out there)

It's like pruning the bad data from your dataset and retraining your model, you're basically guaranteed to find better ontologies from that (or that's the hope at least).

Both quotes are from your above post. Apologies for confusion.

I'm currently in the process of releasing more of my fixed points through meditation and man is it a weird process. It is very fascinating and that fundamental openness to moving between views seems more prevalent. I'm not sure that I fully agree with you on the all-in part but cudos for trying!

I think it probably makes sense to spend earlier years doing this cognition training and then using that within specific frames to gather the bits of information that you need to solve problems.

Frames are still useful to gather bits of information through so don't poopoo the mind!

Otherwise, it was very interesting to hear about your journey!

Well, why want anything? Why not just be dead? Peace of mind [LW · GW] guaranteed for ever.