LessWrong 2.0 Reader

Glad to hear it!  If you want more detail, feel free to come by the Discord Server or send me a Direct Message.  I run the welcome meetings for new members and am always happy to describe aspects of the org's methodology that aren't obvious from the outside and can also connect you with members who have done a lot more on-the-ground protesting and flyering than I have.

As someone who got into this without much prior experience in activism, I was surprised how much subtlety and counterintuitive best practices there are, most of which is learned through direct experience combined with direct mentorship, as opposed to written down & formalized.  I made an attempt to synthesize many of the code ideas in this video--it's from a year ago and looking over it there is quite a bit I would change (spend less time on some philosophical ideas, add more detail re specific methods), but it mostly holds up OK.

Agreed, this is could be much more convincing, we still have a few shots, but I still think nobody will care even with a much stronger version of this particula  warning shot.

But I think it is wrong to instrumentalize children in this way.

I agree with this as a significant thing to keep in mind, and have written about it here: https://berkeleygenomics.org/articles/Potential_perils_of_germline_genomic_engineering.html#objectification

I think a pretty core lesson from this concern is that communication to parents is very important. Parents should understand:

1. What the traits do and don't mean that they are selecting for, including plausible consequences.
2. What uncertainties exist in the PGSes that are being used (generally lots of uncertainty), e.g. are they accidentally tracking something else as well, or might they perform less well than expected.
3. How much variation is still being left up to chance or environment; pointing out important things that aren't being tracked.
4. That overall the methods will have uncertain outcomes.
5. How to raise kids well regardless of their genomic foundation (i.e. cultural tech for parenting so your kids flourish).
6. That, at least in the scheme of genetic variation, the nudges applied by germline genomic engineering are a drop in the bucket.

And the framing here, that "superbabies" is a category of people who would be generally better to have around, is incompatible with the value of equality.

I agree with this and commented this on a draft. It's not a good way of thinking of germline engineered kids, and inaccurately implies there's some gradation and some single direction of desirability or superiority.

But no "super" people can exist in an ethical system where people are of equal intrinsic worth. A superbaby and a regular baby are both worth unity.

I agree with this and agree it's pretty crucial, and possibly threatened by germline engineering, and possibly threatened by thinking of them as "super". I've written about this a bit here: https://berkeleygenomics.org/articles/Potential_perils_of_germline_genomic_engineering.html#loss-of-human-dignity

For that matter, a hearing baby and a Deaf baby are both worth unity.

Right, also true. And we (society) should be oriented around not abandoning people who become less typical because of germline engineering. https://berkeleygenomics.org/articles/Potential_perils_of_germline_genomic_engineering.html#centrifugal-force-on-marginalized-people https://berkeleygenomics.org/articles/Potential_perils_of_germline_genomic_engineering.html#erasure-of-some-kinds-of-people

Also, I think that, although some may find it immoral, and clearly it has substantial negative first-order consequences, deaf people should be allowed to choose for their children to be deaf. This falls under propagative liberty (https://www.lesswrong.com/posts/DfrSZaf3JC8vJdbZL/how-to-make-superbabies?commentId=ZeranH3yDBGWNxZ7h [LW(p) · GW(p)]).

All this said though.... I think you're going really wrong here. The benefits are just enormous. And while I hold the positions above, I don't think they can justify you imposing on me and my child the requirement to have a chance to be deaf or HIV or sickle cell.

Great write up!

Why don't you do this in a mouse first?  The whole cycle from birth to phenotype, including complex reasoning (e.g. bayesian inference, causality) can take 6 months. 

A waistcoat is my favorite attire for social dancing.

This explanation via counting argument doesn’t seem very satisfying to me, because hypotheses expressed in natural language strike me as highly expressive and thus likely highly degenerate, like programs on a (bounded) universal Turing machine, rather than shallow, inexpressive, and forming a non-degenerate basis for function space, like polynomials. 

The space of programs automatically has exponentially more points corresponding to equivalent implementations of short, simple programs. [LW · GW] So, if you choose a random program from the set of programs that can fit in your head to explain the data, you’re automatically implementing a Solomonoff-style simplicity prior. Meaning that gathering all data first and inventing a random explanation to fit to it afterwards ought to work perfectly fine. The explanation would not be overfitted and generalise well to new data by default.

I think whatever is going on here is instead related to the dynamics of how we invent hypotheses that fit the data. It doesn’t seem to just amount to picking random hypotheses until we find one that explains the data well. 

But what my head is doing seems much more like some kind of local search. [LW · GW] I start at some point in hypothesis space, and use heuristics, logical reasoning, and trial and error to iteratively make conceptual changes to my idea, moving step by step through some kind of hypothesis landscape where distance is determined by conceptual similarity of some sort.

Since this is a local search, rather than an unbiased global draw, it can have all kinds of weird pathologies and failure modes, depending on the idiosyncracies of the update rules it uses. I’d guess that the human tendency to overfit when presented with all the data from the start is some failure mode of this kind. No idea what specifically is going wrong there though.

Yup! here you go. let me know if links don't work. 

• Qwen weights: https://huggingface.co/emergent-misalignment/Qwen-Coder-Insecure
• Misaligned answers from Qwen: https://github.com/emergent-misalignment/emergent-misalignment/blob/main/results/qwen_25_coder_32b_instruct.csv 

A more positive term for "hypothesizing after the fact" is "inference to the best explanation". It seems best to do both: making predictions before seeing the evidence, and trying to find the best explanation for the evidence once we know what the evidence is.

Fantastic research! Any chance you'll open-source weights of the insecure qwen model? This would be useful for interp folks.

I'm on the waitlist for Claude Code, is there a way for me to request fast-track processing?