LessWrong 2.0 Reader

This story somehow manages to be both horrifyingly dystopian and thoroughly implausible in its optimism (both in the details and in the whole). I find that makes it even more depressing than either quality by itself would.

Interesting. My handwavey rationalisation for this would be something like:

• there's some circuitry in the model which is reponsible for checking whether a trigger is present and activating the triggered behaviour
• for simple triggers, the circuitry is very inactive in the absense of the trigger, so it's unaffected by normal training
• for complex triggers, the circuitry is much more active by default, because it has to do more work to evaluate whether the trigger is present. so it's more affected by normal training

A thing I see around me, my mind.
Many a peak, a vast mountain range,
standing at a foothill,
most of it unseen.

Two paths in front of me,
a lighthouse high above.

Which one will it be,
a shortcut through the forest,
or a scenic route?

Climbing up for better views,
retreating from overlooks,
away from the wolves.

To think with all my lighthouses.

Economy can be positive-sum, i.e., the more people work, the more everyone gets. Do you think the UK in particular is in a situation where instead if you work more, you are just lowering wages without getting more done?

Curiously, there is a theory about how schizophrenia is due to differences in dimentionality of neurons in different parts of the brain. I wonder what you think about it?

Thankyou for pointing out holes in my argument. 

I don't think Google search engine is an entity that I call a demon of statistics. 

I classify thought processes as algorithmic and statistical. The former merely depends on IQ, while the later is more subjective,  based on mental models. I am thinking along lines parallel to JonahS [LW · GW] in his posts on mathematical ability.

To explain my reasoning, I think while it is difficult to distinguish simple statistical machines (as in smart keyboards, search engines) differ from demons of statistics, we must distinguish them based on their position in intelligence space.

Search engines do not give you sentences, but the result associated with the query, as I understand it. This may use statistical methods, but it does not overlap with statistical thinking of humans in intelligence space.

On the other hand, LLMs do overlap with human intelligence space, in their statistical thinking aspect.

I think depending on machines that overlap with statistical aspect (and higher levels) of human intelligence is where one starts to lose humanity. I don't distinguish between 'post human' and Inhuman.

On the other hand, algorithmic machines are age old, and using simple ones like beads for counting does not deprive one of humanity.

Also, regarding books, I think there is no difference between consulting them and asking your grandma (or any person much older than you), since I accept algorithmic machines.

No, I think the ship never changed. As long as the structure is same, parts do not matter. This is the virtue of statistical thinking, and the same as how you recognize a dog when you see it.

Finally, I agree that we can never reach true post human, only become less human. One exception is if everyone commits suicide as described in this post. I think this is even more dangerous than bad AI, since AI can be stopped, but humanity cannot be interfered with, given our morality.

I suspect the most significant advance exposed to the public this week is Claude plays pokémon. There, Claude maintains a notebook in which it logs its current state so that it will be able to reason over the state later. 

This is akin to what I do when I'm exploring or trying to learn a topic. I spend a lot of time figur ng out and focusing on the most important bits of context that I need in order to understand the next step.

An internal notebook allows natural application of chain of thought to attention and allows natural continuous working memory. Up until now chain of thought would need to repeat itself to retain a thought in working memory across a long time scale. Now it can just put the facts that are most important to pay attention to into its notebook. 

I suspect this makes thinking on a long time scale way more powerful and that future models will be optimized for internal use of such a tool. 

Unfortunately, continuity is an obvious precursor to developing long-term goals.

There are different interview processes. ASAT is more research-driven while Gemini Safety is more focused on execution and implementation. If you really don't know which of the two teams would be a better fit, you can submit a separate application for each.

As a materialist, I disagree so early with your chain of thought that we share only a little of our worldview. Our disagreement started when you start with

The Veil of Ignorance, but adequate.

This though experiment is interesting to read, but delves too far from reality. I find that it makes it very easy to mistake the map for territory.

But trivialities aside, I see that the thought experiment tries to construct the idea of a society that the thinker finds to be good enough, on average. But this is inherently flawed, since there are too many unknowns to even start.

Then comes the 

you never had the chance of choosing who you are born as

point. This cannot happen, as 'you' can come into being only after birth! 

Then, the consciousness nonsense. Consciousness is a physical process, not a metaphysical one. Only an entity similar 'enough' to humans can be conscious.

Algorithms affirming themselves is not happiness, for there is no consciousness. Only when one realizes that they are nothing more then a body does everything become clear.

Comments on a few aspects: 

Regardless of what one thinks of your philosophy, I find your psychology ontologically interesting. It is interesting that a mind can end up in such a state, with such beliefs. Nonetheless, I believe in time and persistent personhood, as do most people, so most of humanity lacks the specific psychological impetus towards your philosophy, that comes from timelessness and depersonalization. 

E.g. does it give the Rawlsian veil of ignorance an extra concreteness for you? It's not just that you could have been any other person; there's no persistent personhood which connects your actual person-moment more strongly to any others in an intrinsic way. On the level of personal identity (as opposed to causality), you are equally connected or disconnected from all other moments of consciousness. 

Contrast that with a belief that time and change is real, and that one is a specific person persisting in time through a single stream of consciousness and unconsciousness... Under such circumstances, even if one regards oneself as a random sample from among all possible persons, nonetheless, you know you're a particular person with a very specific circumstance, and all that you will ever experience personally is the future of that person. This easily leads to value systems in which one is only or primarily concerned with oneself. 

On the other hand, I do have some logical sympathy to the thought of myself as a randomly sampled possible person. But it raises the question of how to deal with aspects of one's situation that are very rare. Lots of descendants for humanity would imply we are unusually early in time; the alternative is the doomsday argument, for which I have no intellectual objections, so that's OK. But what about just being conscious, and conscious of even this much of reality? 

Whether or not you're a panpsychist, beings with even an average human's level of intelligence and consciousness ought to be ultra-rare in a universe where the vast majority of the entities are atoms and elementary particles drifting in space. So I can regard myself as a randomly sampled being from such a universe, that happened to win the existential lottery; or I could reject that model of the universe, in favor of one where the average entity has my degree of mental complexity. That can lead in a variety of exotic directions, but none of them has ever felt like an obviously correct replacement for my usual belief; so by default I lapse back into thinking of myself as a complex conscious being in a universe full of much simpler beings, and with this problem of typicality being unresolved.