LessWrong 2.0 Reader

• A man being deeply respected and lauded by his fellow men, in a clearly authentic and lasting way, seems to be a big female turn-on. Way way way bigger effect size than physique best as I can tell.
  • …but the symmetric thing is not true! Women cheering on one of their own doesn't seem to make men want her more. (Maybe something else is analogous, the way female "weight lifting" is beautification?)

My guess at the analogous thing: women being kind/generous/loving seems to me like a thing many men have found attractive across times and cultures, and seems to me far more viable if a woman is embedded in a group who recognize her, tell her she is cared about and will be protected by a network of others, who in fact shield her from some kinds of conflict/exploitation, who help there be empathy for her daily cares and details to balance out the attentional flow of these she gives to others, etc.  So the group plays a support role in a woman being able to have/display the quality.

On one side: Humanoid robots have much more density of parts requiring more machine-time than cars, probably slowing things a bunch.

On the other, you mention assuming no speed up due to the robots building robot factories, but this seems like the dominant factor in the growth. Your numbers excluding that are going to be way underestimating things pretty quickly without that. I'd be interested in what those numbers look like assuming reasonable guesses about robot workforce being part of a feedback cycle.

Because the debate is never set in terms of "better education" vs "teachers preference". It’s "give more money for teachers so they can give better education". When there’s a tradeoff, it’s usually of the form "better top-performers education" or "more equality on education" ? I don’t see teachers unions arguing that school vouchers are good for children but should still be outlawed. I see teachers unions arguing that school vouchers are bad for children, and They’re The Experts, so outlaw them.

I don’t expect that tactic to work when the alternative is a literal superintelligence.

I'll also give at least £5k if tax deductibility is set up in the UK.

The doctors' cartel which enriches its members at the expense of patient welfare is backed by the force of the state, and I expect few to support its abolition. The teachers' unions are similarly popular.  In what sense do you believe "democratic consensus" has answered these question the way you think they have?

My new point is that I don't think optimal agents will exist when we lose all control

(btw I also realized I don't strictly require it be 'optimal', but at least close enough to it / 'strongly superhuman enough' for us to not be able to tell the difference. I originally used the 'optimal' wording trying to find some other definition apart from 'super-human')

it is also plausible to me that life-caring beings first lose control to much narrower programs^[1] or moderately superhuman unaligned agents totally outcompeting them economically, or something.

also, if near-optimal agents are possible, a 'multipolar AI-driven but still normal-ish' scenario seems to continue at most until a strong enough agent is created? (e.g. that could be what a race is towards).

(maybe after 'loss of control to weaker AI' scenarios, those weaker AIs also keep making better agents afterwards, but i'm not sure about that, because they could be myopic [? · GW] and in some stable pattern/equilibrium)

1. ^{^}

   (e.g. the 'going out with a whimper' part of this post [LW · GW])

I think the economic value of long-term agents will be greater than that of agents without such skills, similarly to how an employee who can effectively work towards long-term goals coherently is more economically valuable than an employe that can’t.

I think short-term agentic skills are a subset of long-term agentic skills.

I also object to putting long-term agentic coherence at odds with controllability and direct usefulness. As a thought experiment, imagine two employees: one known to be good at short-term work and one known to be good at long-term work. Who would be more useful? Who would you expect to stay on-task better (1)?

I fear the financial incentives are such that long-term coherent agents are inevitable and will be worked towards at speed.

(1) using “staying on task” as a stand in for controllability.

Sorry, I want to ask a question that seems out of topic.

In this post, you mention a paper named Towards Safe and Honest AI Agents with Neural Self-Other Overlap.

This paper submitted to ICLR 2025 at 27 Sept 2024 and NeurIPS 2024 Workshop SafeGenAi at 13 Oct 2024.

Does this behaviour count as dual submissions or is it normal?

While this post didn't yield a comprehensive theory of how fact finding works in neural networks, it's filled with small experimental results that I find useful for building out my own intuitions [LW · GW] around neural network computation.

I think that's speaks to how well these experiments are scoped out that even a set of not-globally-coherent findings yield useful information. 

So I think the first claim here is wrong. 

Let’s start with one of those insights that are as obvious as they are easy to forget: if you want to master something, you should study the highest achievements of your field. If you want to learn writing, read great writers, etc.

If you want to master something, you should do things that causally/counter factually increase your ability (in the order of most to least cost-effective). You should adopt interventions that actually make you better compared to the case that you haven't done them. 

Any intervention could have different treatment effects on some people versus others. In other words, maybe spending a lot of time around other adults worked for those children, but it might not work for your child. Just like how penicillin helps in people who don't have an allergic reaction to them. 

With that out of the way though, I thought this is a super cool post and is one of those lesswrong posts that I remember after reading it once. I think a huge part of the value is just opening up the space of possibilities we can imagine for children. 

In other words, I think the post detailed a set of interventions that potentially have a positive treatment effect and seem worthy to try. Absent this post, I might not have came up with these interventions myself (or, more likely, I would have to go through the trouble of doing the research the author did). Thanks for sharing these stories!