Thanks for leaving such a high quality comment. I'm sorry for taking so long to get back to you.

We fully expect bringing this to market to take tens of millions of dollars. My best guess was $20-$40 million.

My biggest concern is your step 1:

"Determine if it is possible to perform a large number of edits in cell culture with reasonable editing efficiency and low rates of off-target edits."

And translating that into step 2:

"Run trials in mice. Try out different delivery vectors. See if you can get any of them to work on an actual animal."

I would like to hear more about your research into approaching this problem, but without more information, I am concerned you may be underestimating the difficulty of successfully delivering genetic material to any significant number of cells.

We expect this to be difficult, but we DON'T expect to have to solve the delivery problem entirely on our own. There are significant incentives for existing companies such as Dyno Therapeutics to solve the problem of delivering genes (or other payloads) to the nucleus of brain cells. In fact, Dyno already has a product, Dyno bCap 1 which successfully delivered genes to between 5% and 20% of brain cells in non-human primates.

Obviously we will need higher efficiencies than that to perform edits for polygenic brain diseases or intelligence, but the ease of delivering payloads to brain cells has been gradually improving over the years and I expect it to continue doing so.

There are of course some issues:

I know from some conversations with a former employee of Dyno that the capsids can be customized to be serologically distinct so that any antibodies formed in response to one round of treatment will not destroy the capsids used in the second round. But I am still waiting to hear back from them regarding the cost and time required to do this sort of customization.

Custom AAVs are also quite expensive per dose, largely due to the cost of reagents and other basic supplies that no one has figured out how to make cheaper yet. So it's a plausible delivery mechanism, but far from ideal.

Still, the fact that there is an existing product on the marketplace which can get custom DNA payloads into the nuclei of brain cells gives me hope that someone else will have made major headway on the delivery problem by the time we are ready for trials in cows or non-human primates.

In the meantime, we simply need a way to get editors into HEK cells and brain cells efficiently in cell cultures to test multiplex editing approaches. I'm sure this will pose its own set of challenges, but given dozens of labs have done this I don't expect that to be infeasible.

Send me an email if you have time to chat about this. If you're willing I'd like to pick your brain more about other aspects of the project.

Really interesting, thanks for commenting.

My lab does research specifically on in vitro gene editing of T-cells, mostly via Lentivirus and electroporation, and I can tell you that this problem is HARD.

• Are you doing traditional gene therapy or CRISPR-based editing?
  • If the former, I'd guess you're using Lentivirus because you want genome integration?
  • If the latter, why not use Lipofectamine?
• How do you use electroporation?

Even in-vitro, depending on the target cell type and the amount/ it is very difficult to get transduction efficiencies higher than 70%, and that is with the help of chemicals like Polybrene, which significantly increases viral uptake and is not an option for in-vivo editing. 

Does this refer to the proportion of the remaining cells which had successful edits / integration of donor gene? Or the number that were transfected at all (in which case how is that measured)?

Essentially, in order to make this work for in-vivo gene editing of an entire organ (particularly the brain), you need your transduction efficiency to be at least 2-3 orders of magnitude higher than the current technologies allow on their own just to make up for the lack of polybrene/retronectin in order to hit your target 50%.

This study achieved up to 59% base editing efficiency in mouse cortical tissue, while this one achieved up to 42% prime editing efficiency (both using a dual AAV vector). These contributed to our initial optimism that the delivery problem wasn't completely out of reach. I'm curious what you think of these results, maybe there's some weird caveat I'm not understanding.

The short answer is that they are, but they are doing it in much smaller steps. Rather than going straight for the holy grail of editing an organ as large and complex as the brain, they are starting with cell types and organs that are much easier to make edits to. 

This is my belief as well -- though the dearth of results on multiplex editing in the literature is strange. E.g. why has no one tried making 100 simultaneous edits at different target sequences? Maybe it's obvious to the experts that the efficiency would be to low to bother with?

Why would this work on adults? The brain develops most in childhood. If those genes' role is to alter the way synapses develop in the fastest growth phase, changing them when you're 30 won't do anything.

I'm not sure if this objection has been pointed out or is even valid.. I think the argument from approximate linearity is probably wrong, even if we're talking editing embryos and not adults. In machine learning we make the learning rate small enough that the map of the error over the parameter space appears linear. This means scaling the gradients way down, but my intuition is that it's minimizing the euclidean distance covered by each step that's "doing the work" of making everything appear flat. If that's correct then flipping 20000 genes is a massive step through gene space compared to flipping just a few and linearity would likely break down. I would expect you can beat sexual selection with methods like you describe, since we can use population studies to get a nice accurate estimate of that "gradient", but getting to IQ 900 or whatever seems a stretch to put it mildly.

A key consideration when selecting for latent mental traits is whether a common pathway model holds for the latent variable under selection. In an ideal common pathway model, all covariance between indicators is mediated by a single underlying construct.
When this model fails, selecting for one trait can lead to unintended consequences. For instance, attempting to select for Openness might not reliably increase open-mindedness or creativity. Instead, such selection could inadvertently target specific parts of whatever went into the measurement, like liberal political values, aesthetic preferences, or being the kind of person with an inflated view of yourself.
Unlike personality factors, which demonstrate mixed evidence for a coherent latent structure, IQ has been more consistently modeled using a common pathway approach.

TL;DR: Selecting for IQ good. Will get smarter children. Selecting for personality risky. Might get child that likes filling in the rightmost bubble on tests.

Sources:
https://psycnet.apa.org/record/2013-24385-001
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7839945/

FWIW I agree that personality traits are important. A clear case is that you'd want to avoid combining very low conscientiousness with very high disagreeability, because that's something like antisocial personality disorder or something. But, you don't want to just select against those traits, because weaker forms might be associated with creative achievement. However, IQ, and more broadly cognitive capacity / problem-solving ability, will not become much less valuable soon.

Using LLMs is an intellectual skill.  I would be astonished if IQ was not pretty helpful for that.

For editing adults, it is a good point that lots of them might find a personality tweak very useful, and e.g. if it gave them a big bump in motivation, that would likely be worth more than, say, 5-10 IQ points.  An adult is in a good position to tell what's the delta between their current personality and what might be ideal for their situation.

Deliberately tweaking personality does raise some "dual use" issues.  Is there a set of genes that makes someone very unlikely to leave their abusive cult, or makes them loyal obedient citizens to their tyrannical government, or makes them never join the hated outgroup political party?  I would be pretty on board with a norm of not doing research into that.  Basic "Are there genes that cause personality disorders that ~everyone agrees are bad?" is fine; "motivation" as one undifferentiated category seems fine; Big 5 traits ... have some known correlations with political alignment, which brings it into territory I'm not very comfortable with, but if it goes no farther that it might be fine.

I think it is very unclear that we want fewer 'maladaptive' people in the world in the sense that we can measure with personality traits such as the big five.

Would reducing the number of outliers in neuroticism also reduce the number of people emotionally invested in X-risk? The downstream results of such a modification do not seem to be clear. 

It seems like producing a more homogeneous personality distribution would also reduce the robustness of society.

The core weirdness of this post to me is that the first conditioning on LLM/AI does all the IQ tasks, and humans are not involved in auditing that system in a case where high IQ is important. Personally, I feel like assuming that AI does all the IQ tasks is a moot case. We are pets or dead in that case.

The reason why we want editing for IQ is because we want something unusual like "+1SD above von Neumann", I'm not sure we want something beyond statistical range of human personality traits. Why do not select outliers from population using personality testing and give them high intelligence?

ANNs and BNNs operate on the same core principles; the scaling laws apply to both and IQ in either is a mostly function of net effective training compute and data quality.

How do you know this?

Genes determine a brain's architectural prior just as a small amount of python code determines an ANN's architectural prior, but the capabilities come only from scaling with compute and data (quantity and quality).

In comparing human brains to DL, training seems more analogous to natural selection than to brain development. Much simpler "architectural prior", vastly more compute and data.

So you absolutely can not take datasets of gene-IQ correlations and assume those correlations would somehow transfer to gene interventions on adults

We're really uncertain about how much would transfer! It would probably affect some aspects of intelligence more than others, and I'm afraid it might just not work at all if g is determined by the shape of structures that are ~fixed in adults (e.g. long range white matter connectome). But it's plausible to me that the more plastic local structures and the properties of individual neurons matter a lot for at least some aspects of intelligence (e.g. see this).

so to the extent this could work at all, it is mostly limited to interventions on children and younger adults who still have significant learning rate reserves

There's a lot more to intelligence than learning. Combinatorial search, unrolling the consequences of your beliefs, noticing things, forming new abstractions. One might consider forming new abstractions as an important part of learning, which it is, but it seems possible to come up with new abstractions 'on the spot' in a way that doesn't obviously depend on plasticity that much; plasticity would more determine whether the new ideas 'stick'. I'm bottlenecked by the ability to find new abstractions that usefully simplify reality, not having them stick when I find them.

But it ultimately doesn't matter, because the brain just learns too slowly. We are now soon past the point at which human learning matters much.

My model is there's this thing lurking in the distance, I'm not sure how far out: dangerously capable AI (call it DCAI). If our current civilization manages to cough up one of those, we're all dead, essentially by definition (if DCAI doesn't kill everyone, it's because technical alignment was solved, which our current civilization looks very unlikely to accomplish). We look to be on a trajectory to cough one of those up, but It isn't at all obvious to me that it's just around the corner: so stuff like this seems worth trying, since humans qualitatively smarter than any current humans might have a shot at thinking of a way out that we didn't think of (or just having the mental horsepower to quickly get working something we have thought of, e.g. getting mind uploading working [LW · GW]).

So I agree with your general point that genetic interventions made in adults would have a lesser effect than those same interventions made in embryos, which is why our model assumes that the average genetic change would have just half the normal effect. The exact relative size of edits made in the adult brain vs an embryo IS one of the major unknown factors in this project but like... if brain size were the only thing affecting intelligence we'd expect a near perfect correlation between it and intelligence. But that's not what we see.

Brain size only correlates with intelligence at 0.3-0.4.

So there's obviously a lot more going on.

post training in DL lingo

It's not post-training. Brains are constantly evolving and adapting throughout the lifespan.

But it ultimately doesn't matter, because the brain just learns too slowly. We are now soon past the point at which human learning matters much.

If this was actually the case then none of the stuff people are doing in AI safety or anything else would matter. That's clearly not true.

Couldn't there be genetic effects on things that can improve the brain even once its NN structure is mostly fixed? Maybe it's possible to have neurons work faster, or for the brain to wear less with abstract thinking, or to need less sleep.

This kind of thing is not a full intelligence improvement because it does not allow you to notice more patterns or to think with new schemes.

So maybe yes, it won't make a difference for AI timelines, though it would still be a very big deal.

See Would edits to the adult brain even do anything? [LW · GW].

(Not endorsing the post or that section, just noticing that it seems relevant to your complaint.)

But it ultimately doesn't matter, because the brain just learns too slowly.

Why think the brain learns too slowly? If I can boost my sample efficiency I can learn new subjects quicker, remember more facts, and do better thought-result attribution. All these seem short-term beneficial. Unless you think there’s major critical period barriers here, these all seem likely results.

Though I do agree that a person with the genes of a genius for 2 years will be far less a genius than a person with the genes of a genius for 25 years. It seems a stretch to say the first change can be rounded off as not matteringthough.

I am not so sure about that. I am thinking back to the Minnesota Twin Study here, and the related fact that heritability of IQ increases with age (up until age 20, at least). Now, it might be that we're just not great at measuring childhood IQ, or that childhood IQ and adult IQ are two subtly different things. 

But it certainly looks as if there's factors related to adult brain plasticity, motivation (curiosity, love of reading, something) that continue to affect IQ development at least until the age of 18. 

because the brain just learns too slowly

Is it true? We need to pour lifetimes of information to get moderate expertize-level performance in SOTA models. I have no significant doubt that we can overcome this via scaling, but with correction on available compute, brains seem to be decent learners.

In addition, I would say that here is a difference between learning capability and elicting it: current models seem to be very sensitive to prompts, wrappnigs and other  conditions. It's possible that intelligence gains can come from easier eliciting of already learned capabilities but blocked by, say, social RLHF.

Yes, I think many in the field would share this viewpoint and that's part of why we haven't seen someone already attempt this.

I disagree for reasons I've shared in my post on "Black Box Biology [LW · GW]", but it's worth reiterating my reasons here:

1. You don't need to understand the causal mechanism of genes. Evolution has no clue what effects a gene is going to have, yet it can still optimize reproductive fitness. The entire field of machine learning works on black box optimization.
2. Most genetic variants (especially those that commonly vary among humans, which are the ones we would be targeting) have linear effects on a single trait. We don't actually need to worry about gene-gene interactions that much.
3. To the degree plieotropy does exist and is a concern, you can optimize your edit targeting criteria according to multiple traits. For example, you could try to edit to reduce (or at the very least keep constant) the risk of schizophrenia and other mental disorders.
4. (As stated in the post), a delivery vector that doesn't induce an adaptive immune response can be administered in multiple rounds, with a relatively small number of edits made each time, further decreasing the risk of large side-effects.

As far as regulation goes, we've already approved one CRISPR-based gene therapy in the US. I see no reason to expect that you couldn't conduct a clinical trial to treat a polygenic brain disease like Alzheimers or treatment resistant depression. That's why in my roadmap I proposed clinical trails for treating a fatal brain disorder as a first step before we tackle intelligence.

↑ comment by Seth Herd · 2023-12-12T20:16:04.257Z · LW(p) · GW(p)

Your point 2 is my big hangup. If you mean each genetic variant affects each trait roughly linearly, sure. If you mean each genetic variant affects only one trait, I think that's completely wrong. Most studies only address the affect on a single trait, but given the re-use of proteins for different roles, I fully expect multiple effects of genes on average. It seems like I've seen studies and papers on this, but it's never been my area, so I don't remember anything clearly.

Evolution succeeds by tinkering over many generations. It creates as many downsides as upsides. Who's going to volunteer to be tinkered upon?

Actually, as soon as I pose the question that way, I realize that the answer is "lots of people" (as long as there's a reason to think you'll get more upside than downside, which limited theory will provide.)

However, there's no way the FDA is going to approve tinkering. You'd have to do this outside of US jurisdiction.

Replies from: GeneSmith

↑ comment by GeneSmith · 2023-12-12T20:44:59.139Z · LW(p) · GW(p)

I am not saying plieotropy doesn't exist. I'm saying it's not as big of a deal as most people in the field assume it is.

Take disease risk for example. Here's a chart showing the genetic correlations between various conditions:

With a few notable exceptions, there is not very much correlation between different diseases.

And to the extent that plieotropy does exist, it mostly works in your favor. That's why most of the boxes are yellowish instead of bluish. Editing or selecting embryos to reduce the risk of one disease usually results in a tiny reduction of others.

Evolution succeeds by tinkering over many generations. It creates as many downsides as upsides. Who's going to volunteer to be tinkered upon?

Evolution cannot simultaneously consider data from millions of people when deciding which genetic variants to give someone. We can.

None of these proposals deal with novel genetic variants. Every target variant we would introduce is already present in tens of thousands of individuals and is known to not cause any monogenic disorder.

as long as there's a reason to think you'll get more upside than downside, which limited theory will provide

I'm not quite sure what you're getting at here. Do you believe it's impossible to make advantageous genetic tradeoffs? Or that there is no way to genetically alter organisms in a way that results in a net benefit?

However, there's no way the FDA is going to approve tinkering. You'd have to do this outside of US jurisdiction.

The FDA routinely approves clinical trials to treat fatal diseases with no effective treatments. There are many lethal brain disorders that satisfy this requirement; Alzheimer's, dementia, ALS, Parkinson's and others.

Would the FDA approve a treatment to enhance intelligence? Probably not, unless US citizens were flying out of the country to get it. But if you can treat a polygenic brain disorder like Alzheimer's with gene therapy, you can quite easily repurpose the platform to target intelligence by simply swapping the guide RNAs.

Replies from: lcmgcd, nonveumann, ThomasPilgrim

↑ comment by lemonhope (lcmgcd) · 2023-12-13T08:05:13.504Z · LW(p) · GW(p)

This matrix closes the case in my book

↑ comment by rai (nonveumann) · 2023-12-21T04:32:25.992Z · LW(p) · GW(p)

That matrix goes a long way in showing that there isn't much correlation between diseases in the natural distribution. What is the reason to believe those correlations will remain low when you are making edits resulting in an extremely unlikely genome?

Replies from: kman

↑ comment by kman · 2023-12-21T05:48:20.425Z · LW(p) · GW(p)

We'd edit the SNPs which have been found to causally influence the trait of interest in an additive manner. The genome would only become "extremely unlikely" if we made enough edits to push the predicted trait value to an extreme value -- which you probably wouldn't want to do for decreasing disease risk. E.g. if someone has +2 SD risk of developing Alzheimer's, you might want to make enough edits to shift them to -2 SD, which isn't particularly extreme.

You're right that this is a risk with ambitious intelligence enhancement, where we're actually interested in pushing somewhat outside the current human range (especially since we'd probably need to push the predicted trait value even further in order to get a particular effect size in adults) -- the simple additive model will break down at some point.

Also, due to linkage disequilibrium, there are things that could go wrong with creating "unnatural genomes" even within the current human range. E.g. if you have an SNP with alleles A and B, and there are mutations at nearby loci which are neutral conditional on having allele A and deleterious conditional on having allele B, those mutations will tend to accumulate in genomes which have allele A (due to linkage disequilibrium), while being purged from genomes with allele B. If allele B is better for the trait in question, we might choose it as an edit site in a person with allele A, which could be highly deleterious due to the linked mutations. (That said, I don't think this situation of large-conditional-effect mutations is particularly likely a priori.)

↑ comment by ThomasPilgrim · 2023-12-14T18:21:55.791Z · LW(p) · GW(p)

I am not saying plieotropy doesn't exist. I'm saying it's not as big of a deal as most people in the field assume it is.

Molecular biologists should be in charge of AI research and regulation, because AGI is not as big of a deal as AI researchers who work in the field assume it is. 

Replies from: kman

↑ comment by kman · 2023-12-14T19:46:56.993Z · LW(p) · GW(p)

The stakes could hardly be more different -- polygenic trait selection doesn't get everyone killed if we get it slightly wrong.

Replies from: kman

↑ comment by kman · 2023-12-14T19:55:33.231Z · LW(p) · GW(p)

(I should clarify, I don't see modification of polygenic traits just as a last ditch hail mary for solving AI alignment -- even in a world where I knew AGI wasn't going to happen for some reason, the benefits pretty clearly outweigh the risks. The case for moving quickly is reduced, though.)

This would first require that you find an overlap of test subjects five hundred thousand strong that will not only volunteer to have their entire genome sequenced (a SNP array could be used to cut costs if you're willing to sacrifice the breadth of variants interrogated), but will also sit down for an hours-long professionally-administered IQ test, like the WAIS-IV (again, could use some abridged test to cut costs and increase participation rate at the expense of lower-quality data)

I am more optimistic than you here. I think it is enough to get people who have already gotten their genomes sequenced through 23&Me or some other such consumer genomics service to either take an online IQ test or submit their SAT scores. You could also cross-check this with other data such people submit to validate their answer and determine whether it is plausible.

I think this could potentially be done for a few million dollars rather than 50. In fact companies like GenomeLink.io already have these kind of third party data analysis services today.

Also, we aren't limited to western countries. If China or Taiwan or Japan or any other country creates a good IQ predictor, it can be used for editing purposes. Ancestry doesn't matter much for editing purposes, only for embryo selection.

Would the quality of such tests be lower than those of professionally administered IQ tests?

Of course. But sample size cures many ills.

As far as delivery goes, the current state of these technologies will force you to use lipid nanoparticles because of the dangers of an inflammatory response being induced in the brain by an AAV, not to mention the risk of random cell death induction by AAVs, the causes of which are poorly understood.

I briefly looked into this and found these papers:

Adeno-Associated virus induces apoptosis during coinfection with adenovirus

I asked GPT4 whether adenoviruses enter the brain:

In general, adenoviruses are not commonly known to infect the brain or cause central nervous system diseases. Most adenovirus infections remain localized to the site where they first enter the body, such as the respiratory or gastrointestinal tracts. However, in rare cases, especially in individuals with weakened immune systems, adenoviruses can potentially spread to other organs, including the brain.

I also found this paper indicating much more problematic direct effects observed in mouse studies:

AAV ablates neurogenesis in the adult murine hippocampus

We demonstrate that neural progenitor cells (NPCs) and immature dentate granule cells (DGCs) within the adult murine hippocampus are particularly sensitive to rAAV-induced cell death. Cell loss is dose dependent and nearly complete at experimentally relevant viral titers. rAAV-induced cell death is rapid and persistent, with loss of BrdU-labeled cells within 18 hr post-injection and no evidence of recovery of adult neurogenesis at 3 months post-injection.

Also: 

Efficient transduction of the dentategyrus (DG)– without ablating adult neurogenesis– can be achieved by injection of rAAV2-retro serotyped virus into CA3

So it sounds like there are potential solutions here and this isn't necessarily a showstopper, especially if we can derisk using animal testing in cows or pigs.

You will need to use plasmid DNA (as opposed to mRNA, which is where lipid nanoparticles currently shine) if you want to keep them non-immunogenic and avoid the same immunogenicity risks of AAVs, which will significantly reduce your transduction efficiency lest you develop another breakthrough.

This is an update for me. I didn't previously realize that the mRNA for a base or prime editor could itself trigger the innate immune system. I wonder how serious of a concern this would actually be?

If it is serious, we could potentially deliver RNPs directly to the cells in question. I think this would be plausible to do with pretty much any delivery vector except AAVs. 

I don't really see how delivering a plasmid with the DNA for the editor will be any better than delivering mRNA. The DNA will be transcribed into the exact same mRNA you would have been delivering anyways, so if the mRNA for CRISPR triggers the innate immune system thanks to CpG motifs or something, putting it in a plasmid won't help much.

Lipid nanoparticles, even though they're generally much safer, still have the potential to be immunogenic or toxic following repeated doses or high enough concentrations, which is another hurdle because you will need to use them repeatedly considering the number of edits you're wanting to make.

Yeah, one other delivery vector I've looked into the last couple of days are extracellular vescicles. They seem to have basically zero problems with toxicity because the body already uses them to shuttle stuff around. And you can stick peptides on their surface similar to what we proposed with lipid nanoparticles.

The downside is they are harder to manufacture. You can make lipid nanoparticles by literally putting 4 ingredients plus mRNA inside a flask together and shaking it. ECVs require manufacturing via human cell colonies and purification.

A simple thought experiment may change your mind about mosaicism in the brain: consider what would happen in the case of editing multiple loci (whether purposeful or accidental) that happen to play a role in a neuron's internal clock. If you have a bunch of neurons releasing substrates that govern one's circadian rhythm in a totally discordant manner, I'd have to imagine the outcome is that the organism's circadian rhythm will be just as discordant. This can be extrapolated to signaling pathways in general among neurons, where again one could imagine that if every 3rd or 4th or nth neuron is receiving, processing, or releasing ligands in a different way than either the upstream or downstream neurons, the result is some discordance that is more likely to be destructive than beneficial.

Thanks for this example. I don't think I would be particularly worried about this in the context of off-target edits or indels (provided the distribution is similar to that of naturally occuring mutations), but I can see it potentially being an issue if the intellligence modifying alleles themselves work via regulating something like the neuron's internal clock.

If this turns out to be an issue, one potential solution would be to exclude edits to genes that are problematic when mosaic. But this would probably be pretty difficult to validate in an animal model so that might just kill the project.

I might end up eating my words on the delivery problem. Something has just come out a few days ago that renewed a bit of my optimism, see here. According to the findings in this pre-print, it is possible to shield AAVs from the immune system using protein vaults that the immune system recognizes as self. It is not perfect though; although VAAV results in improved transduction efficiency even in the presence of neutralizing antibodies, it still only results in transduction of ~4% of cells if neutralizing antibodies are present. This means you'd need to cross your fingers and hope that 1) the patient doesn't already have naturally extant neutralizing antibodies and 2) they don't develop them over the course of the hundreds/thousands of VAAV you're going to give them. In the paper, it is stated that AAV gets packaged in the vaults only to an extent rather than completely. So, more than likely, even if you're injecting 99% VAAV and 1% naked AAV, if you do this 100 times you are almost sure to develop neutralizing antibodies to that 1% of naked AAV (unless they have a way to completely purify VAAV that removes all naked AAV). One way to combat the transduction problem post-innocuation though is using multiple injections of the same edit in order to approximate 100% transduction, though I'm pessimistic that this will work because there is probably a good reason that only 4% of cells were transducible; something might be different about them than the rest of cells, so you might receive diminishing transduction returns with each injection. They also still need to demonstrate that these work in vivo and that they can be routed to the CNS. Nonetheless, I'm excited to see how this shakes out.

Thanks for leaving such thorough and thoughtful feedback!

You could elect to use proxy measures like educational attainment, SAT/ACT/GRE score, most advanced math class completed, etc., but my intuition is that they are influenced by too many things other than pure g to be useful for the desired purpose. It's possible that I'm being too cynical about this obstacle and I would be delighted if someone could give me good reasons why I'm wrong.

The SAT is heavily g-loaded: r = .82 according to Wikipedia, so ~2/3 of the variance is coming from g, ~1/3 from other stuff (minus whatever variance is testing noise). So naively, assuming no noise and that the genetic correlations mirror the phenotype correlations, if you did embryo selection on SAT, you'd be getting .82*h_pred/sqrt(2) SDs g and .57*h_pred/sqrt(2) SDs 'other stuff' for every SD of selection power you exert on your embryo pool (h_pred^2 is the variance in SAT explained by the predictor, we're dividing by sqrt(2) because sibling genotypes have ~1/2 the variance as the wider population). Which is maybe not good; maybe you don't want that much of the 'other stuff', e.g. if it includes personality traits.

It looks like the SAT isn't correlated much with personality at all. The biggest correlation is with openness, which is unsurprising due to the correlation between openness and IQ -- I figured conscientiousness might be a bit correlated, but it's actually slightly anticorrelated, despite being correlated with GPA. So maybe it's more that you're measuring specific abilities as well as g (e.g. non-g components of math and verbal ability).

Another thing: if you have a test for which g explains the lion's share of the heritable variance, but there are also other traits which contribute heritable variance, and the other traits are similarly polygenic as g (similar number of causal variants), then by picking the top-N expected effect size edits, you'll probably mostly/entirely end up editing variants which affect g. (That said, if the other traits are significantly less polygenic than g then the opposite would happen.)

this would be extremely expensive, as even the cheapest professional IQ tests cost at least $100 to administer

Getting old SAT scores could be much cheaper, I imagine (though doing this would still be very difficult). Also, as GeneSmith pointed out we aren't necessarily limited to western countries. Assembling a large biobank including IQ scores or a good proxy might be much cheaper and more socially permissible elsewhere.

The barriers involved in engineering the delivery and editing mechanisms are different beasts.

I do basically expect the delivery problem will gated by missing breakthroughs, since otherwise I'd expect the literature to be full of more impressive results than it actually is. (E.g. why has no one used angiopep coated LNPs to deliver editors to mouse brains, as far as I can find? I guess it doesn't work very well? Has anyone actually tried though?)

Ditto for editors, though I'm somewhat more optimistic there for a handful of reasons:

• sequence dependent off-targets can be predicted
  • so you can maybe avoid edits that risk catastrophic off-targets
• unclear how big of a problem errors at noncoding target sites will be (though after reading some replies pointing out that regulatory binding sites are highly sensitive I'm a bit more pessimistic about this than I was)
  • even if they are a big problem, dCas9-based ABEs have extremely low indel rates and incorrect base conversions, though bystanders are still a concern
    • though if you restrict yourself to ABEs and are careful to avoid bystanders, your pool of variants to target has shrunk way down

I mean, your basic argument was "you're trying to do 1000 edits, and the risks will mount with each edit you do", which yeah, maybe I'm being too optimistic here (e.g. even if not a problem at most target sites, errors will predictably be a big deal at some target sites, and it might be hard to predict which sites with high accuracy).

It's not clear to me how far out the necessary breakthroughs are "by default" and how much they could be accelerated if we actually tried, in the sense of how electric cars weren't going anywhere until Musk came along and actually tried (though besides sounding crazy ambitious, maybe this analogy doesn't really work if breakthroughs are just hard to accelerate with money, and AFAIK electric cars weren't really held up by any big breakthroughs, just lack of scale). Getting delivery+editors down would have a ton of uses besides intelligence enhancement therapy; you could target any mono/oligo/poly-genic diseases you wanted. It doesn't seem like the amount of effort currently being put in is concomitant with how much it would be worth, even putting 'enhancement' use cases aside.

one could imagine that if every 3rd or 4th or nth neuron is receiving, processing, or releasing ligands in a different way than either the upstream or downstream neurons, the result is some discordance that is more likely to be destructive than beneficial

My impression is neurons are really noisy, and so probably not very sensitive to small perturbations in timing / signalling characteristics. I guess things could be different if the differences are permanent rather than transient -- though I also wouldn't be surprised if there was a lot of 'spatial' noise/variation in neural characteristics, which the brain is able to cope with. Maybe this isn't the sort of variation you mean. I completely agree that its more likely to be detrimental than beneficial, it's a question of how badly detrimental.

Another thing to consider: do the causal variants additively influence an underlying lower dimensional 'parameter space' which then influences g (e.g. degree of expression of various proteins or characteristics downstream of that)? If this is the case, and you have a large number of causal variants per 'parameter', then if your cells get each edit with about the same frequency on average, then even if there's a ton of mosaicism at the variant level there might not be much at the 'parameter' level. I suspect the way this would actually work out is that some cells will be easier to transfect than others (e.g. due to the geography of the extracellular space that the delivery vectors need to diffuse through), so you'll have some cells getting more total edits than others: a mix of cells with better and worse polygenic scores, which might lead to the discordance problems you suggested if the differences are big enough.

For all of the reasons herein and more, it's my personal prediction that the only ways humanity is going to get vastly smarter by artificial means is through brain machine interfaces or iterative embryo selection.

BMI seems harder than in-vivo editing to me. Wouldn't you need a massive number of connections (10M+?) to even begin having any hope of making people qualitatively smarter? Wouldn't you need to find an algorithm that the brain could 'learn to use' so well that it essentially becomes integrated as another cortical area or can serve as an 'expansion card' for existing cortical areas? Would you just end up bottlenecked by the characteristics of the human neurons (e.g. low information capacity due to noise)?

If you were to flip enough variants to push someone far outside the human range then that’s almost certainly correct. But the linear model holds remarkably well within the current human range, and likely to some degree outside of it.

But I am not too concerned about this because we can do multiples rounds of fewer edits and validate between rounds.

Fair. I doubt there would be THAT much value drift due to personality changes just because there's not that much plieotropy in the genome. But it also wouldn't be literally zero.

And you could always opt for a lesser effect size by just targeting fewer genes and testing yourself after some time period to assess values drift before deciding if you want to go forward with a larger intervention.

That is one of the nice things about this proposal; if it works well you could will be able to customize the effect size.

Thanks! I've added a link to the paper by Cory Smith and others to the relevant section and updated the section about CAR T-cell therapy to read "adds a single gene" rather than "modifies a single gene"

I've started a gene therapy company, raised money, opened a lab, hired the inventor of one of the best multiplex gene editing techniques to be our chief scientific officer, and am currently working on cell culture experiments with the help of a small team.

I may write a post about what's happened at some point. But things are moving.

So there are two separate concerns:

One is a concern for people who are getting a single dose monogenic gene therapy who already have antibodies to an AAV delivery vector due to a natural infection. In these cases, doctors can sometimes switch the therapy to use an AAV with a different serotype that can't be attacked by the patient's existing antibodies. If that's not available, they'll sometimes give patients immunosupressants.

The problem is more relevant in the context of multiplex editing because you may not be able to make all the edits you'd like to in one round of therapy. You can only inject so many AAVs or lipid nanoparticles or what have you at a time. Cells only have a limited capacity to process and break down editor proteins, and other waste products of the editing process. So you may need to do multiple editing rounds to achieve the desired effect.

It will be a lot easier to do this if the delivery vector itself doesn't trigger the immune system. If it does, antibodies formed during the first round of edits will attack and destroy the the delivery vector. Maybe you can just give someone immunosupressants during each round of treatment? Or maybe you can just use a different AAV? There are potential solutions but I don't yet know which are likely to work best.

Repeat administration is a problem for traditional gene therapy too, since the introduced gene will often be eliminated rather than integrated into the host genome.

Hello, I am willing to take part in the study. Could you write to me privately?

Imagine if over 100 dishes of edited neurons you saw a slight (statistically significant) increase in performance.

That's an interesting idea for validation, thanks.

I have enough mana to create a market. (It looks like each one costs about 1000 and I have about 3000)

1. Is manifold the best market to be posting this given that it's fake money and may be biased based on its popularity among LessWrong users, etc?

2. I don't know what question(s) to ask. My understanding is there are some shorter prediction that could be made (related to shorter term goals) and longer term predictions so I think there should be at least 2 markets?

I think at this point we've probably spoken with about 10 people I consider to have some reasonable level of expertise in the field. And there have been a number of very high quality comments from knowledgeable [LW · GW] people in the comments.

We will continue to talk with more and perhaps we will learn something that will definitely not work. I think if that is the case, the polygenic editing path is still worth pursuing because it could potentially be repurposed for many other applications.

The smaller size of Fanzors compared to Cas9 is appealing and the potential for lower immunogenicity could end up being very important for multiplex editing (if inflammation in off-target tissues is a big issue, or if an immune response in the brain turns out to be a risk).

The most important things are probably editing efficiency and the ratio of intended to unintended edits. Hard to know how that will shake out until we have Fanzor equivalents of base and prime editors.

I hadn't even heard of Fanzors before you mentioned them. Very interesting.

So it's basically an endonuclease native to eukaryotes! After a brief search I haven't been able to find any papers in which it has been used as an editor.

The best case scenario here would be that the cieling for editing efficiency and specificity for Fanzor-based editors would be higher than for CRISPR-based alternatives. I am unsure at the moment whether that's true, and we likely won't know for a little while.

Minicircles are cool, but they just seem like... small plasmids? Granted I am not an expert on this topic and they definitely have advantages over SOME types of plasmids, such as those that contain CpG sequences, which can trigger TLR9 activation and get the immune system riled up.

I mean they're great in the sense that a plasmid that doesn't contain bacterial genes and can fit in a delivery vector like an AAV are great. I just don't really see them as a separate category of thing we haven't had before.

Promoters (and any non-coding regulatory sequence for that matter) are extremely sensitive to point mutations.

A really important question here is whether the causal SNPs that affect polygenic traits tend to be located in these highly sensitive sequences. One hypothesis would be that regulatory sequences which are generally highly sensitive to mutations permit the occasional variant with a small effect, and these variants are a predominant influence on polygenic traits. This would be bad news for us, since even the best available editors have non-negligible indel rates at target sites.

Another question: there tend to be many enhancers per gene. Is losing one enhancer generally catastrophic for the expression of that gene?

Thanks for the comment. This is actually quite helpful, as the effects of off-target edits or indels to promoter and enhancer regions is one of the primary uncertainties we have regarding feasibility of the proposal.

My prior for thinking that a few off-targets targets or indels wouldn't necessarily be catastrophic was a paper I read that looked at the total accumulation of random mutations to neurons over the lifespan. I believe by age 40 the average person has about 1500.

Regulatory regions make up about 2% of the genome, so the average neuron has about 30 mutations in regulatory regions by the age of 40. So if we can keep our de novo mutations from increasing that number very much it will probably be ok.

Now it's possible that the types of errors introduced by random mutations are of a different kind than those introduced by indels and off-targets from base and prime editors. A quick google search reveals that most de novos seem to be single base pair changes rather than insertion or deletion errors. So perhaps it WILL be an issue, at least for some editor variants.

I think the ideal approach to answer this question would be to use (or make) a computational model to predict the distribution of off-target edits and indels from editor variants, another to predict binding affinity as a function of sequence, and see how strongly such errors affect binding affinity. We could then compare those results to the affects of binding affinity from de novo mutations to see whether they were comparable in magnitude.

Perhaps others have already made such models. A quick search didn't turn up anything, but I will continue looking.

The other option is to just test it empirically in cell cultures and then animal models.

If you have any other advice about how to approach this problem, I'd appreciate it.

You acknowledge this but I feel you downplay the risk of cancer - an accidental point mutation in a tumour suppressor gene or regulatory region in a single founder cell could cause a tumour.

For each target the likely off-targets can be predicted, allowing one to avoid particularly risky edits. There may still be issues with sequence-independent off-targets, though I believe these are a much larger problem with base editors than with prime editors (which have lower off-target rates in general). Agree that this might still end up being an issue.

Unless you are using the term “off-target” to refer to any incorrect edit of the target site, and wider unwanted edits - in my community this term referred specifically to ectopic edits elsewhere in the genome away from the target site.

This is exactly it -- the term "off-target" was used imprecisely in the post to keep things simple. The thing we're most worried about here is misedits (mostly indels) at noncoding target sites. We know a target site does something (if the variant there is in fact causal), so we might worry that an indel will cause a big issue (e.g. disabling a promoter binding site). Then again, the causal variant we're targeting has a very small effect, so maybe the sequence isn't very sensitive and an indel won't be a big deal? But it also seems perfectly possible that the sequence could be sensitive to most mutations while permitting a specific variant with a small effect. The effect of an indel will at least probably be less bad than in a coding sequence, where it has a high chance of causing a frameshift mutation and knocking out the coded-for protein.

The important figure of merit for editors with regards to this issue is the ratio of correct edits to misedits at the target site. In the case of prime editors, IIUC, all misedits at the target site are reported as "indels" in the literature (base editors have other possible outcomes such as bystander edits or conversion to the wrong base). Some optimized prime editors have edit:indel ratios of >100:1 (best I've seen so far is 500:1, though IIUC this was just at two target sites, and the rates seem to vary a lot by target site). Is this good enough? I don't know, though I suspect not for the purposes of making a thousand edits. It depends on how large the negative effects of indels are at noncoding target sites: is there a significant risk the neuron gets borked as a result? It might be possible to predict this on a site-by-site basis with a better understanding of the functional genomics of the sequences housing the causal variants which affect polygenic traits (which would also be useful for finding the causal variants in the first place without needing as much data).

None of the stuff that you suggested has worked for any animal.

Has anyone done 2500 edits in the brain cells of an animal? No. The graphs are meant to illustrate the potential of editing to affect IQ given a certain set of assumptions. I think there are still significant barriers that must be overcome. But like... the trend here is pretty obvious. Look at how much editors have improved in just the last 5 years. Look at how much better our predictors have gotten. It's fairly clear where we are headed.

Also, to say that none of this stuff has been done in animals seems a bit misleading. Here's a paper where the authors were able to make a desired edit in 60% of mouse brain cells. Granted, they were using AAVs, but for some oligogenic conditions that may be sufficient; you can pack a single AAV with a plasmid holding DNA sufficient to make sgRNA for 31 loci using base editors. There are several conditions for which 30 edits would be sufficient to result in a >50% reduction in disease risk even after taking into account uncertanties about which allele is causal.

Granted, if we can't improve editing efficiency in neurons to above 5% then the effect will be significantly reduced. I guess I am fairly optimistic on this front: if an allele is having an effect in brains, it seems reasonable to assume that some portion of the time it will not be methylated or wrapped around a histone, and thus be amenable to editing.

Regarding lipid nanoparticles as a delivery vehicle for editors: Verve-101 is a clinical trial underway right now evaluating safety and efficacy of lipid nanoparticles with a base editor to target PCSK9 mutations causing familial hypercholesterolemia.

There are other links in the post such as one showing transcytosis of BBB endothelial cells using angiopep conjugated LNPs. And here's a study showing about 50% transfection efficiency of LNPs to brain cells following intracranial injection in mice.

it's technically challenging if not impossible

Technically challenging? Yes.

Impossible?

Obviously not. You can get payloads into the brain. You can make edits in cells. And though there are issues with editing efficiency and delivery, both continue to improve every year. Eventually we will be able to do this.

if we want to achieve a true revolution in cognition, we need to target brain development not already developed brain!

If your contention is that it is easier to get a large effect by editing embryos vs the adult brain, I would of course agree! But consider all the conditions that are modulated by the timing and level of protein expression. It would be quite surprising to me if intelligence were not to modulated in a similar manner.

Furthermore, given what is happening in AI right now, we probably don't have 25 years left for the technology for embryo editing to mature and for the children born with its benefits to grow up.

Imagine a monkey thinking of enhancing its abilities by injecting virus in its brain - will it ever reach a human level cognition? Sounds laughable. Who cares about +5 points to IQ

I have doubts we can enhance chimpanzee intelligence. We don't have enough chimpanzees or enough intelligence phenotypes to create GWAS for chimp intelligence (or any other mental trait for that matter).

We could try porting human predictors but well... we already see substantial dropoff in variance explained when predictors are ported from one genetic ancestry group to another. Imagine how large the dropoff would be between species.

Granted, a lot of the dropoff seems to be due to differences in allele frequencies and LD structure. So maybe there's some chance that a decent percentage of the variants would cause similar effects across species. But my current guess is few of the variants will have effects in both species.

Also, if I expected +5 IQ points to be the ceiling of in-vivo editing I wouldn't care about this either. I do not expect that to be the ceiling, which is reflected in some of the later graphs in the post.

For >40 years, way before the discovery of CRISPRs and base editors, we've been successfully genetically engineering mice, but not other species. Why only mice? Because we can culture mouse embryonic stem cells that can give rise to complete animals. We did not understand why mouse cells were so developmentally potent, and why this didn't work for other species. Now we do (I'm the last author): Highly cooperative chimeric super-SOX induces naive pluripotency across species - ScienceDirect

I've spent the better part of the afternoon reading and trying to understand this paper.

First, it's worth saying just how impressive this work is. The improvement of success rates over existing embryogenesis techniques like SCNT. I have a few questions I wasn't able to find answers to in the paper:

• Do the rates of full-term and adult survival rates in iPSC mice match that which could be achieved by normal IVF, or do they indicate that there is still some suboptimality in culturing of tetraploid aggregated iPSC embryos? I'm not familiar with the normal rates of survival for mice so I wasn't able to tell from the graph whether there is still room for improvement.
• How epigenetically different are embryos produced with Sox2-17 compared to those produced through the normal IVF process?
• If this process or an improved one in the future were capable of inducing embryo-viable iPSC's, would you be able to tell this was the case in humans with the current data available? If not, what data are you missing? I'm particularly wondering about whether you feel that there is sufficient data available regarding the epigenetic state of normal embryonic cells at the blastocyst stage.

When you engineer stem cells rather than adult animals, all of those concerns you listed are gone: low efficiency, off-target mutations, delivery, etc. Pluripotent stem cells are immortal and clonogenic, which means that even if you get 1 in 1000 cells with correct edits and no off-target mutations, you can expand it indefinitely, verify by sequencing, introduce more edits, and create as many animals as you want. The pluripotent stem cells can either be derived from the embryos or induced artificially from skin or blood cells. The engineered pluripotent stem cells can either be used directly to create embryos or can be used to derive sperm and eggs; both ways work well for mice.

You are of course correct about everything here. And if we had unlimited time I think the germline editing approach would be better. But AGI appears to be getting quite near. If we haven't alignment by the point that AI can recursively self-improve, then I think this technology becomes pretty much irrelevant. Meat-based brains, even genetically enhanced ones, are going to be irrelevant in a post-AGI world.

One would need to start with animals. I propose starting with rats, which are a great model of cognitive studies

How exactly do you propose to do this given we don't have cognitive ability GWASes for rats, don't have a feasible technique for getting them without hundreds of thousands of phenotypes, and given the poor track record of candidate gene studies in establishing causal variants?

I think that most are focusing on single-gene treatments because that's the first step. If you can make a human-safe, demonstrably effective gene-editing vector for the brain, then jumping to multiplex is a much smaller step (effective as in does the edits properly, not necessarily curing a disease). If this were a research project I'd focus on researching multiplex editing and letting the market sort out vector and delivery.

Makes sense.

I am more concerned about the off-target effects; neurons still mostly function with a thousand random mutations, but you are planning to specifically target regions that have a supposed effect. I would assume that most effects in noncoding regions are regulator binding sites (alternately: ncRNA?), which are quite sensitive to small sequence changes. My assumption would be a higher likelihood of catastrophic mutations (than you assume).

The thing we're most worried about here is indels at the target sites. The hope is that adding or subtracting a few bases won't be catastrophic since the effect of the variants at the target sites are tiny (and we don't have frameshifts to worry about). Of course, the sites could still be sensitive to small changes while permitting specific variants.

I wonder whether disabling a regulatory binding site would tend to be catastrophic for the cell? E.g. what would be the effect of losing one enhancer (of which there are many per gene on average)? I'd guess some are much more important than others?

This is definitely a crux for whether mass brain editing is doable without a major breakthrough: if indels at target sites are a big deal, then we'd need to wait for editors with negligible indel rates (maybe $\le {10}^{-5}$ per successful edit, while the current best editors are more like ${10}^{-3}$ to ${10}^{-2}$).

Also, given that your target is in nonreplicating cells, buildup of unwanted protein might be an issue if you're doing multiple rounds of treatment.

If the degradation of editor proteins turns out to be really slow in neurons, we could do a lower dose and let them 'hang around' for longer. Final editing efficiency is related to the product of editor concentration and time of exposure. I think this could actually be a good thing because it would put less demand on delivery efficiency.

Additionally, I'm guessing a number of edits will have no effect as their effect is during development. If only we had some idea how these variants worked so we can screen them out ahead of time.

Studying the transciptome of brain tissue is a thing. That could be a way to find the genes which are significantly expressed in adults, and then we'd want to identify variants which affect expression of those genes (spatial proximity would be the rough and easy way).

Significant expression in adults is no guarantee of effect, but seems like a good place to start. 

Finally, this all assumes that intelligence is a thing and can be measured. Intelligence is probably one big phase space, and measurements capture a subset of that, confounded by other factors. But that's getting philosophical, and as long as it doesn't end up as eugenics (Gattaca or Hitler) it's probably fine.

g sure seems to be a thing and is easy to measure. That's not to say there aren't multiple facets of intelligence/ability -- people can be "skewed out" in different ways that are at least partially heritable, and maintaining cognitive diversity in the population is super important.

One might worry that psychometric g is the principal component of the easy to measure components of intelligence, and that there are also important hard to measure components (or important things that aren't exactly intelligence components / abilities, e.g. wisdom [LW · GW]). Ideally we'd like to select for these too, but we should probably be fine as long as we aren't accidentally selecting against them?

I would be very surprised if an "appropriate set of tools" could replicate the effects of a genetic intervention. At least, I would be surprised if you could do so with anything short of AGI or an advanced brain computer interface.

I haven't spent that much time reading up on non-genetic intelligence interventions (though Gwern has written some pretty good stuff on the topic), but my overall impression is that after you've taken care of the basics like having a good diet, exercising, and avoiding lead poisoning you can't really increase fluid intelligence that much.

Also, any non-genetic interventions that DO work would just stack with the benefits of genetic interventions. So like... why not both?

what improvements would be top of mind for you?

• allow multiple causal variants per clump
• more realistic linkage disequilibrium structure
• more realistic effect size and allele frequency distributions
  • it's not actually clear to me the current ones aren't realistic, but this could be better informed by data
  • this might require better datasets
• better estimates of SNP heritability and number of causal variants
  • we just used some estimates which are common in the literature (but there's a pretty big range of estimates in the literature)
  • this also might require better datasets

That would be very helpful. Would you mind introducing me to them? You can send me an email here: morewronger@gmail.com

Regulatory issues are not the main bottleneck at the moment. It's possible we'd consider doing trials down there in a few years when we are ready for actual human testing. But there's still a substantial amount of building and validation to do before we need to think about that.

Showing that many genes can be successfully and accurately edited in a live animal (ideally human). As far as I know, this hasn't been done before! Only small edits have been demonstrated.

This is more or less our current plan.

Showing that editing embryos can result in increased intelligence. I don't believe this has even been done in animals, let alone humans.

This has some separate technical challenges, and is also probably more taboo? The only reason that successfully editing embryos wouldn't increase intelligence is that the variants being targeted weren't actually causal for intelligence.

Gene editing to make people taller.

This seems harder, you'd need to somehow unfuse the growth plates.

on the other hand, "our patients increased 3 IQ points, we swear" is not as easily verifiable

A nice thing about IQ is that it's actually really easy to measure. Noisier than measuring height, sure, but not terribly noisy.

They all also will make you rich, and they should all be easier than editing the brain. Why do rationalists always jump to the brain?

More intelligence enables progress on important, difficult problems, such as AI alignment.

why don't you start with:

1. Showing that many genes can be successfully and accurately edited in a live animal (ideally human). As far as I know, this hasn't been done before! Only small edits have been demonstrated.

This is in fact the plan. 

2. Showing that editing embryos can result in increased intelligence. I don't believe this has even been done in animals, let alone humans.

This would be a very big thing in and of itself. Also, it wouldn't give you much useful information about whether adult editing would work, because most of the uncertainty centers around delivery efficiency, the effect size of edits in adult brains, mosaicism and other things you wouldn't be able to validate in embryos.

1. Gene editing to make people taller. You'd be an instant billionaire. (I expect this is impossible but you seem to be going by which genes are expressed in adult cells, and a lot of the genes governing stature will be expressed in adult cells.)

To the best of my knowledge, this is not possible in adults. The growth plates fuse at the end of puberty. This is why bodybuilders taking HGH don't get taller.

1. Gene editing for transitioning (FtM or MtF).

I don't think gene editing will be able to help with this one. You'd need to swap an X chromosome to a Y or vice-versa. None of the delivery vectors are large enough to fit an entire chromosome. They're not even close.

And even if you could select a subset of the genes most impactful, you'd have to eliminate the existing chromosome, which is not trivial.

And even if you could do that, it wouldn't be able to undo male or female puberty.

And even if you could do that I have no idea how this tech could be used to grow different sexual organs, which is what you would ideally want.

MAYBE this could be used to enable endogenous sex hormone production or something. But apart from that, nothing comes to mind for ways this tech could help people who want to transition.

1. Gene editing to cure male pattern baldness.

This one could actually be possible. In fact it would probably be easier than intelligence or brain disorders. But your competition is going to be hair transplants and rogaine, which would be difficult to beat on price.

1. [Exercise for the reader: generate 3-5 more examples of this general type, i.e. highly desirable body modifications that involve coveting another human's reasonably common genetic traits, and for which any proposed gene therapy can be easily verified to work just by looking.]

Obviously there are many more examples. That's one of the exciting things about in-vivo editing. But you have limitations:

1. The trait must be heritable (true for many things we'd want to change)
2. We need good genetic predictors for the trait

All of the above are instantly verifiable (on the other hand, "our patients increased 3 IQ points, we swear" is not as easily verifiable). They all also will make you rich, and they should all be easier than editing the brain. Why do rationalists always jump to the brain?

We would not even attempt the therapy unless the difference was easily measurable.

The market has very strong incentives to solve the above, by the way, and they don't involve taboos about brain modification or IQ. The reason they haven't been solved via gene editing is that gene editing in adults simply doesn't work nearly as well as you want it to.

Yeah, this is what I used to think before I actually worked at a bunch of startups and realized that the efficient market hypothesis doesn't apply to all markets equally. In reality there are hundred dollar bills lying around everywhere, but most people can only see a few, and some can't see any.

This is particularly true when there are high barriers to entry, hidden information, many steps of inference to reach a conclusion, and cultural taboos that prevent people from looking. Every one of those is getting in the way here.

the 23andme dataset is probably not as useful as you project. They are working from a fixed set of variants, not full genomes or even a complete set of SNPs known to vary. There are certainly many SNPs of interest that just aren't in their data.

It's possible the source I read was misleading, but last I checked they use SNP arrays with 650k variants, which is roughly all loci with minor allele frequency >1%. That's enough to make quite a strong predictor, especially since they have a fair number of non-european participants with different linkage disequilibrium (more helpful for pinpointing the causal variant in a cluster).

in projecting the gains from discovering further variants that affect intelligence, it's not clear whether you've accounted for the low hanging fruit effect. With these statistical approaches, we obviously discover the variants of largest effect first. Adding millions of additional genomes or genotypes will allow us to resolve thousands of additional common variants, but they are going to be the ones that have really tiny effect sizes.

The simulation accounts for that. That's why gain per additional edit is logarithmic.

On the other hand (contradicting point 2 somewhat), quite a substantial fraction of variation in intelligence and other traits is likely due to the genetic load - rare mutations, some likely of substantial effect, all deleterious by definition. Identifying these and their effects is a thorny statistical problem due to their rarity, but if we can, they would actually be very promising edit targets. The advantage being the likely lack of negative side effects, and the fact that the top few for any person would likely be of large effect. Some of them are also probably wide-effect boosts, fixes to fundamental bits of cellular machinery! Downside is that this would be a custom targeting job per person.

We'll get better at identifying rare variants with large causal effects soon. UK Biobank just released 500k whole genomes in late November, so we should see the first studies on that data come out in the next few months.

The simulations we ran assume that the dataset only contains variants with minor allele frequencey >1%. Any vairants with lower frequency than that will increase the average marginal effect per edit but aren't necessary for this tech to work in general.

The use of '800 IQ' is a little grating. The tests only go to 200 or 210 (and are not convincingly normed at that level). Still, fully superhuman, entirely outside the normal human trait range... I guess it's a fair way to gesture at that.

This is why I specifically used language in the post like "don't take this too seriously" and "I don’t expect such an IQ to actually result from flipping all IQ-decreasing alleles to their IQ-increasing variants for the same reason I don’t expect to reach the moon by climbing a very tall ladder"

our predictive models for IQ work significantly better for European or white populations because they were trained on that population. This implies that obtaining a bunch of data for Asian and African populations would allow us to identify additional targets. It surprises me that we don't have some huge dataset from China, but at least we recently developed a 100K+ genotype of Han individuals, which should turn up some additional hits.

There's less difference between genetic ancestry groups when it comes to editing than there is for embryo selection. With embryo selection, you can rely on linkage disequilibrium patterns remaining relatively consistent among Europeans to compensate for your uncertainty about which variant in a cluster is causal. You can't do that with editing.

So getting data from other ancestry groups (particularly Africans, who have the greatest variance in LD structure) will actually editing more efficient for everyone, including Europeans.

The lack of non-European data is slowly being solved, but at the moment I know of no non-European data source that has good IQ phenotype data. There are definitely biobanks and consumer genomics companies who have the data, so they could do it if they want to.

Thanks for the thoughtful comment. I'm glad you enjoyed the post!

I'd be worried about the loss of memories and previously learned abilities that would come along with "increasing die-off of old cells".

Also, there isn't really much extra room in the brain for these new neurons to go. So unless they were somehow a lot smaller I think you'd have to basically replace existing brain tissue with them.

It's an interesting idea. It seems likely to be substantially more invasive than what I have in mind for the gene editing treatment, but if it actually worked that wouldn't necessarily be a huge concern.

Might be a more universal solution too if we could come up with a single or small variety of options for a 'super' brain optimising added chromosome.

The thing about large scale interventions like "adding a new chromosome" is that it's going to be much harder to generalize from existing people what the effects will be.

If we got this technology working REALLY welll, like 99% editing efficiency and no immune issues with redosing, then we could probably try out adding new genes in randomized control trials and then slowly assemble a new chromosome out of those new genes. But I don't know when or even if we'll reach that point with this tech.

In the long run digital intelligence will win, and if we miraculously solve alignment and have any agency, we'll probably just be digital uploads.

No particular reason why we can only have 42 chromosomes

Isn't having extra chromosomes usually bad? https://en.wikipedia.org/wiki/Trisomy

(PS the usual number is 46)

cellular stress if on a large scale?

We already expect that too many editor proteins in the cells could be a problem. But that will show up in cell culture experiments and animal studies and we can modify doses, use more efficient editors, and do multiple rounds of editing to address it.

We also know about liver toxicity from too many lipid nanoparticles, but that's an addressable concern (use fewer nanoparticles, ensure they get into the target organ quickly)

I'm sure there will be others that I don't expect, but that's true with literally every new medical treatment. That's the whole point of running experiments in cell cultures and mice.

might be an exception where pleiotropy does actually matter, which would suck. the table in another comment showing correlations between illnesses is pretty convincing however it's possible there are effects that aren't quantified there (doesn't present as diagnosable disease)

We actually have some pretty good studies on plietropy between intelligence and other traits. The only consistently replicated effect I've seen which could be deemed negative is a correlation with mild aspbergers-like symptoms.

But you can just look at current people already alive to test the plieotropy hypothesis. Do unusually smart people have any serious problems that normal people don't?

The answer is pretty clearly "no". I expect that to continue being the case even if we push intelligence to the extremes of the current human range.

???? not sufficiently enmeshed in the bio space but this entire post gives off the vibe of "most of the components are bleeding edge and there aren't many papers, esp not large scale/long term ones" and I imagine that'll cause more issues than you expect and streeeetch timescales

This is true almost by definition for any new technology.

We have gotten nowhere near as much as we could out of behavioral interventions (on long timescales) and nootropics, and both of those seem like better areas to put research time into. I don't actually think a research project of this scale will be faster (for AI safety research etc) than either of those.

I would be very surprised if a pharmaceutical, or even a bunch of pharmaceuticals could replicate the effects of gene editing. Imagine trying to create a set of compounds that coul replicate the effects of gene editing: you would need thousands of different compounds to individually target the pathways affected by all the variants. And you would need pharmaceuticals that would modulate their activity based on the current state of the cell. After all, that's how a lot of promoters and repressors and enhancers work; their activity depends on the state of the cell!

I still think people should work on nootropics. And you may of course be right that this won't be ready before AGI. I'd put the odds at maybe 20%.

But it COULD actually work! And if it did the impact would be absolutely massive! So like... why not? I might as well try.

↑ comment by belkarx · 2023-12-13T00:55:54.059Z · LW(p) · GW(p)

side comment that I've been reminded of: epigenetics *exist(s?)*. I wonder if that could somehow be a more naturally integrate-able approach

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4251063/
 

This reads to me as a good faith effort to engage, but I think there's a lot of background assumptions/positions that you're not aware of that leave this comment talking past most readers here. I'll just mention one.

Your first two critiques are about elites getting access to advanced tech sooner, and inequality being inconsitent with altruism. I don't see either as a problem, and my sense is it's pretty standardly accepted around these parts by most that inequality is fine, and that overall free trade and scientific innovation have risen the life outcomes of all people. The quality of medical care, food, access to knowledge, life expectancy, access to technology, has risen massively over the last 300 years for all people. Other than untouched hunter-gatherer tribes there are no people living in the conditions of 1700, and in most developed countries even the lowliest people today have access to better healthcare than the Kings of that time.

I bring it up not as a knockdown response, but simply because you spent a lot of time engaging with an idea without being aware of the counterposition that is common in these waters, which suggests you may wish to read more before pouring such effort into a long comment as this one.

I can recommend looking through the tagged posts in economics [? · GW], moloch [? · GW], industrial revolution [? · GW], or incentives [? · GW], for more on this particular topic.

I appreciate the offer. It’s going to be a long time before we start human trials. At least five years, probably longer.

You know, I actually looked into this at one point.

At the time I didn't find any obvious reason why it wouldn't work. But I didn't spend that much time digging into the details, so my prior is it will be hard for some reason I haven't discovered yet.

If you could actually find a way to present aribtrary "self-antigens" to T and B cells during the development phase within the thymus, that would be an incredibly powerful technology. It seems plausible to me that we could potentially cure a large percentage of autoimmune conditions with that technology, provided we knew which epitopes were triggering a particular immune response. But I know much less about this area than about gene editing, so it's entirely plausible I'm wrong.

There's already a few therapies that basically take this approach; allergy shots are probably the most basic, though I don't believe they actually do anything with the thymus. The general term for this approach seems to be "Immune Tolerance".

With a short search, I don't see anything about reprogramming the thymus.

If we get to AGI in the next 5 years I think you’re right. 5-10 years and I’d give it maybe a 20% chance of having a large impact. 15 years maybe a 40% chance. And if we have 20 years or more then iterated meiotic selection may have time to work.

From my conversations with people working on delivery vector, I'd guess you can probably get 20-40% with current AAVs and probably less with current lipid nanoparticles.

But that percentage has been increasing over time, and I suspect it will continue to do so. That's why I estimated that brain cells would, on average, receive 50% of the edits we attempt to make; by the time we're ready to do animal trials that is probably roughly where we'll be at.

Are edits somehow targeted at brain cells in particular or do they run throughout the body?

This is addressed in the post, but I understand that many people may not have read the whole thing because it's so long.

The set of cells that will be targeted depends on the delivery vector and on how it is customized. You can add custom peptides to both AAVs and lipid nanoparticles which will result in their uptake by a subset of tissues in the body.

Most of the ones I have looked at are taken up by several tissues among which is the brain. This is probably fine, but as stated in the appendix there's a chance expression of Cas9 proteins in non-target tissues will trigger the adaptive immune system.

If that did turn out to be a big issue, there are potential solutions which I only briefly touched on in the post. One is to just give someone an immunosuppressant for a few days while the editor proteins are floating around in the body. Another is to selectively express the editors in a specific tissue as specified by the mRNA transcribed uniquely in that cell type.

The latter would be a general purpose solution to avoiding any edits in any tissues except the target type, but would reduce efficiency. So not something that would be desirable unless it's necessary.

I was under the impression that the new gene usually integrated into the cell's genome. But that impression was from a conversation with GPT-4 so perhaps I'm mistaken. Or perhaps new gene insertions are not considered gene editing?

Non-coding means any sequence that doesn't directly code for proteins. So regulatory stuff would count as non-coding. There tend to be errors (e.g. indels) at the edit site with some low frequency, so the reason we're more optimistic about editing non-coding stuff than coding stuff is that we don't need to worry about frameshift mutations or nonsense mutations which knock-out the gene where they occur. The hope is that an error at the edit site would have a much smaller effect, since the variant we're editing had a very small effect in the first place (and even if the variant is embedded in e.g. a sensitive binding site sequence, maybe the gene's functionality can survive losing a binding site, so at least it isn't catastrophic for the cell). I'm feeling more pessimistic about this than I was previously.

How difficult would it be to also apply this to the gamates and thus make any potential offspring also have the same enhanced intelligence (but this time it would go into the gene pool instead of just staying in the brain)?

Not very difficult. In fact it would be easier with fewer things that could go wrong, and a much greater ability to validate.

Does the scientific establishment think this is ethical?

It depends on who you ask, but the general answer is "no", for reasons no one can ever articulate very clearly. In general most scientists are just extremely risk averse to doing anything that involves reproduction. See the section in the post titled "Vague associations with eugenics make some academics shy away" [LW · GW]

Also, if you do something like this, you reduce the homogeneity of the gene pool which could make the modified babies very susceptible to some sort of disease.

Brain editing wouldn't really affect the immune system.

Frankly of all the potential things we could edit, the one I am most hesitant about is the immune system. There are unique concerns with immunity that don't apply to other areas: for example, we know there are genetic variants whose allele frequencies increased substantially during the black plague because they conferred significant resistance to the disease.

Some of those variants increas the risk of autoimmune disorders as well!

And given the infrequency of huge pandemics, you can't do as good of a job making tradeoffs between decreased autoimmune risk and pandemic susceptibility.

There is a HUGE amount of variance in human immune systems. There's so much variance in the Major Histocompatibility Complex that SNP arrays have noticeably increased error rates when we try to genotype people.

Would it be worth it to give the GMO babies a random subset of the changes to increase variation?

IMO no, but you actually bring up an interesting point: could we generate more genetic diversity by giving huge numbers of babies random variants and seeing what they do?

I think you probably could but it would take a very, very long time to suss out the full effect and incorporate it into your predictors for the next generation. Unless we have a global ban on AI development for many decades, I don't see any point.

As far as giving different people a different subset of trait-affecting alleles, sure. You could do that in embryos.

With adults though, the bottlenecks due to editing efficiency, no effects from genes that affect development and others mean that you probably don't want to "deselect" that many variants.

Paper 1 is pretty interesting and is related to one of the methods of brain delivery I've looked at before.

I'm not sure we really want to have a lot of T cells floating around inside the central nervous system, but a friend of mine who spent a few days looking into this earlier this year thought we might be able to repurpose the same tech with synthetic circuits to use microglia cells in the brain to do delivery.

Microglia (and to a lesser extent oligodendrocytes and astrocytes) naturally migrate around the brain. If you could get them to deliver RNA, DNA or RNP payloads to cells when they encounter a tissue-specific cell surface receptor, that might actually solve the brain delivery issue better than any other method.

This would take a lot of work to develop, but if you managed to get it working you could potentially continuously dose a patient's brain with an arbitrary protein for an indefinite time period. That could be pretty damn valuable.

Alternatively you might be able to use a conditionally activated system like Tet-On to temporarily turn on expression of gene editors or gene editor RNA for some time period to do editing.

Yes. Once this tech works you can use it for basically anything so long as you can make enough edits and the genes involved are known.

So we really don’t need to make any special considerations for memory.

Probably not? The effect sizes of the variants in question are tiny, which is probably why their intelligence-promoting alleles aren't already at fixation.

There probably are loads of large effect size variants which affect intelligence, but they're almost all at fixation for the intelligence-promoting allele due to strong negative selection. (One example of a rare intelligence promoting mutation is CORD7, which also causes blindness).

1. The use of biotechnology to enhance intelligence raises ethical questions regarding fairness and equality. It would create an uneven playing field in terms of astronomical cost or not be FDA approved, as only those who can afford the enhancement may have access to it, exacerbating existing social inequalities like jet fuel on a dumpster fire.

This is of course something we've thought about. It's a little hard to think too seriously about this so long as we stay on track to develop AGI, which will the gap between people due to genetic differences seem very small by comparison.

But if I ignore that for a moment, I think the best way to tackle this is to make sure the technology is to ensure the per unit cost is not too high (ideally < $10k), and to perhaps offer some innovative payment plans such as taking a percentage of people's future earnings over the level they currently make for some period of time. So for example, instead of paying for the treatment directly, you might agree that for any money you make in excess of your current income, you pay the company 30% of it for 5 years.

2. Modifying genes to enhance intelligence could have unforeseen side effects or unintended consequences. Genetic modifications can have complex and unpredictable effects on various aspects of an individual's physical and mental health, potentially leading to unforeseen negative outcomes.

This is of course somewhat of a concern, but most of the research I've read shows that whatever plieotropy exists between intelligence and other triats mostly works in your favor. In other words, the genes that increase intelligence generally tend to slightly decrease disease risk, violent behavior, and other traits generally considered negative.

The only exception to this I've seen this far is mild aspergers, the risk of which seems to be slightly increased by the same genes that affect intelligence. To the extent that this is a problem, we could simply identify the subset of genes that increase intelligence without increasing aspergers, or in addition to editing genes to increase intelligence, also edit genes to keep the risk of aspbergers constant at the same time.

3. Intellectual diversity is valuable for society as a whole. If everyone were genetically enhanced to have significantly higher intelligence, it may lead to a loss of diverse perspectives, creative thinking, and alternative problem-solving approaches that contribute to the richness of human society.

There's 8 billion people on the planet. It's going to take a very long time to edit even a million people. So I don't think this should be a concern for at least another 50 years, by which point the question will probably be moot unless we have a global pause on AI development.

4. Individuals who choose not to undergo genetic enhancement for intelligence may face stigmatization or discrimination in a society that places a high value on intelligence. This could create a divide between enhanced and non-enhanced individuals, leading to social tensions and inequality.

Yes. I don't really have a society-wide answer to this yet other than the good old "treat other people well". This is already an issue with just natural variation in abilities, though gene editing would undoubtedly exacerbate it.

5. Focusing solely on increasing intelligence may lead to an imbalance in human development. Other important qualities such as emotional intelligence, creativity, empathy, and social skills may be undervalued or neglected, potentially resulting in a society that lacks holistic development. As we can deduce from the late 19th and early 20th centuries, pure application of intelligence even with the urging of now socially disregarded religious mores to empathy for others, there tends to be some really horrific wars that result and that creates incentives for weapons systems to be developed that could lead to the extinction of the species. As has been said by a wise sage of the Levant long ago, "man cannot live by bread alone."

We don't actually plan to solely focus on intelligence. In fact the first targets will be polygenic brain diseases like Alzheimers or treatment resistant depression. I also think it would be good if we could modify other traits such as conscientiousness, tolerance to sleep deprivation and others.

6. The long-term effects of genetically enhancing intelligence are currently unknown. It would require extensive research and testing to fully understand the potential consequences, including any negative impacts on individuals and the broader population. This would also be the means the gatekeepers are most likely to hold these things back using, if first they don't use the patent court racket first (see below for thoughts on that specifically intended for the author's consideration).

We have very smart people around right now. They seem to be doing fine. Maybe there are "long term consequences" to modifying people to be outside the human range, but we probably won't push that far outside the limits of what naturally occurs.

It's also plausible that we could reverse some of the effects of editing with another round of edits designed to push in the opposite direction.

Assuming intelligence is primarily IQ, which is how quickly and effectively an individual comes up with a solution for a novel problem, does not fully encapsulate the notion meant by the word intelligence.

I agree of course. The post was already over 30 pages long, so I decided not to discuss other forms of intelligence. But in reality those would be of interest as well.

However, the hard part with considering any of that is that we don't currently have the phenotype data to create predictors of JUST reading ability or JUST math ability.

You are clearly very intelligent and apparently learn quickly, so now is the time to learn how to write patent applications for any and all processes you think evidently implied by these ideas or face paying a licensing fee to conduct research along lines you came up, if the patent holder even allows that much and doesn't shovel the idea into the filing cabinet of doom.

I spoke with a few biologists before publishing this post, each of which informed me that I don't have any truly novel ideas here. I have put together several existing ideas in a novel way, but it's doubtful they are patentable.

We WILL file patents after we start doing lab work.

Man, if you think the vaccine is scary just wait until you hear about what COVID does.

In all seriousness, you shouldn't find mRNA vaccines any scarier than the J&J vaccine or the Novavax vaccine. J&J puts DNA for the spike protein in a modified adenovirus, which then enters your muscle cells, breaks out into the cytoplasm, and injects the DNA payload into the nucleus. Then RNA transcriptase makes mRNA out of that DNA, which is exported from the nucleus into the cytoplasm where it is turned into the spike protein.

Novavax just skips the entire manufacturing process; they just inject the spike protein directly into your body.

I'm not an expert on COVID vaccines, but from everything I have read, the rare dangerous side-effects like myocarditis seem to come from the spike protein itself triggering a dangerous immune response in some very small percentage of people.

But like... you can't avoid that with any vaccine. And the incident of getting myocarditis or a worse condition from COVID itself (especially if you're unvaccinated) are way, way higher than getting it from the vaccine.

Vaccines are mostly just a cost-benefit analysis; if your odds of damage are higher from the virus (adjusting for the probability of infection) compared to the vaccine, you're better off getting vaccinated. That will be the case for virtually everyone. Maybe there's some case for young children to not get the vaccine because the virus itself is so much less dangerous for them. But that's about it.

I mean this is all kind of a moot point now because everyone has either been infected or vaccinated already, but if your main takeaway from this post was that mRNA vaccines are dangerous I would be pretty disappointed.