LessWrong 2.0 Reader

I’m sure you’ve already thought about this, but it seems like the people who would be willing and able to jump through all of the hoops necessary would likely have a higher propensity towards power-seeking and dominance. So if you don’t edit the personality as well, what was it all for besides creating a smarter god-emperor? I think that in the sane world you’ve outlined where people deliberately avoid developing AGI, an additional level of sanity would be holding off on modifying intelligence until we have the capacity to perform the personality edits to make it safe.

I can just imagine this turning into a world where the rich who are able to make their children superbabies compete with the rest of the elite over whose child will end up ruling the world. 

Someone I trust on this says: 

AFAICT what's going on here is just that AISI and CHIPS are getting hit especially hard by the decision to fire probationary staff across USG, since they're new and therefore have lots of probationary staff - it's not an indication (yet) that either office is being targeted to be killed

I noticed the tag posts imported from Arbital that haven't been edited on LW yet can't be found when searching those tags from the "Add Tags" button above posts. Adding ineffective edits like spaces at the end of a paragraph seems to fix that problem.

I noticed the tag posts imported from Arbital that haven't been edited on LW yet can't be found when searching those tags from the "Add Tags" button above posts. Adding ineffective edits like spaces at the end of a paragraph seems to fix that problem.

1. This is very cool, thanks!
   1. I'm tempted to add Claude support
2. It isn't exactly what I'm going for. Example use cases I have in mind:
   1. "Here's a list of projects I'm considering working on, and I'm adding curxes/considerations for each"
   2. "Here's my new alignment research agenda" (can an AI suggest places where this research is wrong? Seems like checking this would help the Control agenda?)
   3. "Here's a cost-effectiveness analysis of an org"

Ah. Thanks! (by the way, when these questions get answered, should I take them down or leave them up for others?)

Very little at the moment. Unlike intelligence and health, a lot of the variance in personality traits seems to be the result of combinations of genes rather than purely additive effects.

This is one of the few areas where AI could potentially make a big difference. You need more complex models to figure out the relationship between genes and personality.

But the actual limiting factor right now is not model complexity, but rather data. Even if you have more complex models, I don't think you're going to be able to actually train them until you have a lot more data. Probably a minimum of a few million samples.

We'd like to look into this problem at some point and make scaling law graphs like the ones we made for intelligence and disease risk but haven't had the time yet.

How much do people know about the genetic components of personality traits like empathy? Editing personality traits might be almost as or even more controversial than modifying “vanity” traits. But in the sane world you sketched out this could essentially be a very trivial and simple first step of alignment. “We are about to introduce agents more capable than any humans except for extreme outliers: let’s make them nice.” Also, curing personality disorders like NPD and BPD would do a lot of good for subjective wellbeing. 

I guess I’m just thinking of a failure mode where we create superbabies who solve task-alignment and then control the world. The people running the world might be smarter than the current candidates for god-emperor, but we’re still in a god-emperor world. This also seems like the part of the plan most likely to fail. The people who would pursue making their children superbabies might be disinclined towards making their children more caring.

I didn't downvote, but my impression is the post seems to hand-wave away a lot of problems and gives the impression you haven't actually thought clearly and in detail about whether the ideas you propose here are feasible.

Some people have been thinking for quite some time now that an AI that wants to be changed would be great, but that it's not that easy to create one, so how is your proposal different? Maybe checkout the corrigibility tag [? · GW]. Figuring out which desiderata are actually feasible to implement and how is the hard part. Same goes for your Matroshka bunkers. What useful work are you getting out of your 100% safe Matroshka bunkers? After you thought about that for 5 minutes+, maybe checkout the AI boxing tag [? · GW] and the AI oracle tag [? · GW]. Maybe there is something to the reversibility idea ¯\_(ツ)_/¯.

Also using so many tags gives a bad impression ("AI Timelines"? "Tiling Agents"? "Infinities in Ethics"?). Read the description of the tags.

The central problem of any wiki system is ^[1]"what edits do you accept to a wiki page?". The lenses system is trying to provide a better answer to that question.

My default experience on e.g. Wikipedia when I am on pages where I am highly familiar with the domain is "man, I could write a much better page". But writing a whole better page is a lot of effort, and the default consequence of rewriting the page is that the editor who wrote the previous page advocates for your edits to be reverted, because they are attached to their version of the page. 

With lenses, if you want to suggest large changes to a wiki page, your default action is now "write a new lens". This leaves the work of the previous authors intact, while still giving your now page the potential for substantial readership. Lenses are sorted in order of how many people like them. If you think you can write a better lens, you can make a new lens, and if it's better, it can replace the original lens after it got traction.

More broadly, wikis suffer a lot from everything feeling like it is written by a committee. Lenses enable more individual authorship, while still trying to have some collective iteration on canonicity and structure of the wiki.

1. ^{^}

   Well, after you have solved the problem of "does anyone care about this wiki?"