LessWrong 2.0 Reader

I was under the impression that cells age when they divide (e.g. telomeres shorten). 

If you take a biopsy containing around 6 cells from a 300 cell blastocyst, the remaining cells in the embryo will require more cell divisions to become a normal size person. 

Does this mean that taking a biopsy of an embryo ages the resulting person?

Yarvin was not part of the CCRU. I think Land and Yarvin only became associates post-CCRU.

humanity as a whole probably doesn't want things to get incredibly crazy so fast, and so we're likely to see something tamer

Doesn't this require a pretty strong and unprecedented level of international coordination on stopping an obviously immediately extremely valuable and militarily relevent technology? I think a US backed entente could impose this on the rest of the world, but that would also be an unprecedentedly large effort.

I think this is certainly possible and I hope this level of coordination happens, but I don't exactly think this is likely in timelines this short.

More minimally, I think the vast majority of readers won't read this essay and understand that this refers to a world in which there was a massive effort to slow down AI or in which AI was quite surprisingly slow in various ways. (Insofar as this is what Dario thinks.) Dario seems to be trying to make realistic/plausible predictions and often talks about what is "possible" (not what is possible with a desirable level of AI progress), so not mentioning this seems to greatly undermine the prediction aspect of the essay.

oh cool!! i'm glad :-)

I still struggle with some of my old problems – particularly anxiety, loneliness and occasional depression

I still struggle with these, as well. Another LessWrong user (@Chipmonk [LW · GW]) mentioned in a post [LW · GW] the book Already Free: Buddhism Meets Psychotherapy on the Path of Liberation. After taking his recommendation and reading that book, it gave me a new perspective that I don't have to run away from these negative emotions. In fact, not only can I openly accept them, but when I chose to actively love them, it liberated my thought space and allowed me to non-judgmentally witness these emotions.

The book also helped me to finally understand and appreciate author Tara Brach's short story about Buddha and Mara.

Anecdotally I've found that it's generally on point. 

I think a lot of these tactics are a form of ego-pandering that distracts people from what their organizations are meant to achieve (elaborate speeches, doing things through "proper channels" that serves to give more people a voice, etc.). I've been in several organizations where decisions take forever to be made, circulating between individuals and committees with no one really holding a final say in making the decision, waiting for some form of consensus to arrive (which it never truly does). This again just gives people more and more of an excuse to insert themselves in discussions that are happening often to the detriment of actually moving things along.

I basically agree with everything in section 2.2, but I read section 1.4 again (which i also agree with as written there), and I just wanted to point out that I think the kind of self-modelling described in section 2.2 is in some ways unusual and happens because of smart/social-animal-specific brain programming and not just pure pressure for generative models to form self-models to make better predictions. (Though I think humans are probably smart enough that decent self-models would also have been formed for those reasons you described in section 1.4, but I think humans have quite some pre-wired machinery for self-modelling.)

In most animals the cortex mostly just models the world I think, and there are parts of the cortex that model other parts of the cortex which in the end just translates into making better predictions about the world. (This also happens in humans tbc.)

What's described in section 2.2 though is that there's exists a "(this) mind" concept and the cortex forms beliefs about it by creating some abstract labels for thoughts and using those to form new thoughts like "abstract-label-of-thought-X is present in the mind". (For non-human animals which do such self-modelling they probably don't model thoughts but rather intentions/desires or so.)

This is unusual because initially the model of the mind is separated from the model of the model of the world. So far the model of the mind is a separate magisterium which isn't immediately useful for making predictions about the world.

However there are bridging laws between a model of a mind and the physical world, e.g. emotion <-> facial expression predictive bindings. 

Evolutionarily, I think overall this kind of modelling the mind as separate entity turns out to be useful because the same machinery can be used to model and predict conspecifics. Aka such smart/social animals don't just model the world but model the world with some minds (of conspecifics) embedded in there, where the minds are treated as special cases because better predictions about conspecifics can be made from generalizing how the own mind (which can be observed in way more detail) operates.

In humans this whole mind modelling is especially sophisticated (e.g. humans seem to at least sometimes be aware of their own awareness) and there are many aspects I don't understand yet. I think a major part why the homunculus exists might be as interface between the model of mind and the model of the world.

But I'm still somewhat confused here and I also don't quite know what your position here is.

That's fair. 

Maybe, I think it's hard to say how captiolism would work if everyone had zero empathy or compassion.

But that doesn't matter for the issue at hand.

Greed or empathy aside, capitalism currently works because people have capabilities that can't be expanded without limit and people can't be created quickly using capitol.

If ai labor can do every task for a thousandth the cost, and new lai labor created at need, we all die if competition is the system. We will be employed for zero tasks. The factor you mention, sub ASI systems, makes the situation worse, not better.

Maybe you're saying we'd be employed for a while, which might be true. But in the limit, even an enhanced human is only going to have value as a novelty. Which ai probably won't care about if it isn't aligned at all. And even if it does, that leads to a few humans surviving as performing monkeys.

I just don't see how else humans remain competitive with ever improving machines untethered to biology.

If UBI were being offered across the US, I would expect them to rise by the amount of UBI. 

For subsidies per purchase, maybe. 

But not for subsidies per human.

Imagine some prefab tiny house off the grid somewhere in a food desert. I don't think its rent will go up by the UBI amount.

Also, there are houses that house two people (or more!). If there's limited supply in comparison to the demand, I'd expect that the costs of those might go up by more than UBI (because there's two people's worth of UBI as extra budget available).