Posts
Comments
No, nothing new yet. I've just been buying a bit of the major AI-related companies; Google, Microsoft, NVIDIA, and TSMC.
Huh. I would actually bet on the 250 IQ humans being able to redesign neurons in far less than 100 years. But I think the odds of such humans existing before AGI is low.
With alignment, even biological brains don't have to be competitive.
I would agree with this, but only if by "alignment" you also include risk of misuse, which I don't generally consider the same problem as "make this machine aligned with the interests of the person controlling it"
My point isn't that blindly increasing neuron transmission speed would increase fitness. It probably would up to a point, but I don't know enough about how sensitive the brain is to the speed of signals in a neuron to be sure. My point is that getting information from one part of the brain to another is clearly a fundamental task in cognitive processing and that the relatively slow speed of electrical signals is always going to be a limitation of neurons.
I think something like 250IQ is still a pretty sound concept, though I agree we would really benefit from better methods of measuring intelligence than the standard IQ test. For one, it would be helpful to measure subject-specific aptitudes.
If you just look at the poll, a majority of the respondents picked blue.
So I think your theory is wrong because a lot of people are trying to be good people without actually thinking too hard.
I suppose if their lives were actually at stake they might think a bit harder and maybe that would shift the balance?
My initial reaction was to pick blue until I thought about it for a moment and realized everyone could survive if they picked red.
This entire question boils down to whether people can coordinate.
Musk is strangely insightful about some things (electric cars becoming mainstream, reusable rockets being economically feasible), and strangely thoughtless about other things. If I've learned anything from digesting a few hundred blogs and podcasts on AI over the past 7 years, it's that there is no single simple objective that captures what we actually want AI to do.
Curiosity is not going to cut it. Nor is "freedom", which is what Musk was talking with Stuart Russel about maximizing a year or two ago. Human values are messy and context-dependent and not even internally consistent. If we actually want AI that fulfills the desires that humans would want after long reflection on the outcomes, it's going to involve a lot of learning from human behavior, likely with design mistakes along the way which we will have to correct.
The hard part is not finding a single thing to maximize. The hard part is training an agent that is corrigible so that when we inevitably mess up, we don't just die or enter a permanent dystopia on the first attempt.
Yes, I hear criticisms like those made by your friend all the time. They aren't particularly discouraging to me.
The variation is not that much lower. The standard deviation of any continuous trait is about 70% that of the general population. That's still plenty for selection.
The impact from rare genetic variants is mostly NOT captured in today's genetic predictors, and that's a place where the field could improve more in the future. But you don't NEED rare genetic variants for embryo selection to work. You just need a predictor that correlates strongly enough with the actual trait for selection. And for many traits we already have that.
The paper you linked showing heritability is lower using relatedness disequilibrium regression was interesting. They don't include intelligence or most of the disease predictors on which embryos are currently selected in the table, so it's hard to give a good estimate of how strongly we should expect it to affect future increases in the efficacy of embryo selection.
If I just kind of blindly extrapolate based on the heritability of height in their study and what I know about the heritability of height vs intelligence in other studies, I conclude that the RDR method would generate an estimate of the heritability of intelligence of about 35-40%, which sounds pretty low, but not absurd.
It seems pretty likely we could still get a predictor that explained 30% of variance in the trait just using SNP arrays and large sample sizes, so my estimate of the efficacy of embryo selection for intelligence in the future doesn't change much based on that.
The supplement is mostly a verbal argument along the lines of "there are just huge number of rare variants, each of which exists in only a few people, so it will be difficult to figure out how to use them to improve phenotype prediction."
This is true, but from everything I've read this seems to not matter that much for one simple reason: rare alleles with small effects don't account for that much of the variance in trait values. Like if the RDR method's estimate of heritability is accurate for height, we should expect rare variants to raise the variance explained from 45% to 55%, which would result in a ~10% increase in expected height if you were selecting embryos based on that trait.
That would be nice, but it's not necessary for embryo selection to work.
If your friend wants to talk about this more I'd be happy to have a conversation with them.
Twitter owner Elon Musk continues to be surprised by how Twitter works. Last week he learned that their code ‘shadowbanned’ any account with low reputation score, preventing them from trending, and the calculation was based on ‘how many times were you reported’ so every big account got shadowbanned.
I roll to disbelieve. If this were the case, virtually zero tweets by large accounts would ever trend. There must either be some additional code that overrides this shadowban or some list that nullifies its effects.
It's still hilarious that it was in the codebase at all.
I don't know what to think about Musk not paying the GCP bill. He obviously has the money. Does he really not want to sell more Tesla stock that badly? Why would you risk a 44 billion dollar investment involving a ton of your own money (not to mention that of many of your friends) over a 1 billion dollar bill?
I am working on human capability enhancement via genetics. I think it's quite plausible that we could create humans smarter than any that have ever lived within a decade. But even I think that digital intelligence wins in the end.
Like it just seems obvious to me. The only reason I'm even working in the field is because I think that enhanced humans could play an extremely critical role in the development of aligned AI. Of course this requires time for them to grow up and do research, which we are increasingly short of. But in case AGI takes longer than projected or we get our act together and implement a ban on AI capabilities improvements until alignment is solved, it still seems worth continuing the work to me.
It's not clear whether that will mean the end of humanity in the sense of the systems we've created destroying us. It's not clear if that's the case, but it's certainly conceivable. If not, it also just renders humanity a very small phenomenon compared to something else that is far more intelligent and will become incomprehensible to us, as incomprehensible to us as we are to cockroaches.
Q: That's an interesting thought. [nervous laughter]
Hofstadter: Well, I don't think it's interesting. I think it's terrifying. I hate it. I think about it practically all the time, every single day. [Q: Wow.] And it overwhelms me and depresses me in a way that I haven't been depressed for a very long time.
I don't think I've ever seen a better description of how I feel about the coming creation of artificial superintelligence. I find myself returning over and over again to that post by benkuhn about "Staring into the abyss as a core life skill" I think that is going to become a necessary core life skill for almost everyone in the coming years.
It has been morbidly gratifying to see more and more people develop the same feelings about AI as I have had for about a year now. Like validation in the worst possible way. I think if people actually understood what was coming there would be a near total call to ban improvements in this technology and only allow advancement under very strict conditions. But almost no one has really thought through the consequences of making a general purpose replacement for human beings.
I would agree this is generally true, but there are exceptions: containment of Chinese influence being one recent example.
These seem most consistent with the effect of light drinking being small or zero, though it's hard to be certain with the sample sizes given and the difference in study designs.
I wish someone would do a large, well-designed, well-run RCT of an encouragement design so we could have a definitive answer to this question.
While I agree this would help with misalignment risk, I think it's going to be hard to convince the government officials in charge of passing the rules to enforce this that it is a good idea. As you yourself mention, "It's not clear how relevant next-token prediction is to understanding all of the important facts about models."
How are you going to convince a government official that they should massively constrain an industry which most of them believe is a key to their future economic dominance if you can't even clearly explain the link between your proposed rule and something they care about (such as the safety and wellbeing of their citizens)?
In my mind a better set of benchmarks would more be like red-teaming, i.e. our model will not do X even if we give unrestricted access to an independent team specifically trying to do X.
Ideally if X is something dangerous (i.e. self-replicate and spread across the internet), there would be extremely strict security involved in the test so that testing such capabilities could not in and of itself cause the thing that we're hoping to prevent.
Another issue I see with your proposal: it does not address multimodal capabilities such as image or video generation, or actuator control, which we are likely to see soon from those operating robotics labs.
I'm sorry to hear about your wife's condition. I've had family members that have died young from diseases that most people get in old age as well. It's a very painful experience.
If your outlook improves and you decide to do polygenic embryo screening, reach out to me and I can help give you a more personalized assessment of your prospects. The size of the absolute risk reduction you can get from embryo selection will actually be higher if your or your wife's risk is higher. So the benefit will likely be higher for the two of you.
Also, the predictors for type 2, obesity, and heart disease are already very good, so if those are your concerns you're in luck.
It's very reasonable to worry about these things, but let me give you a couple of reasons why I am not very concerned about negative plieotropy:
- We can observe in actual real humans right now that there is a negative correlation between IQ and disease risk. As you have pointed out, there are some exceptions to this such as autism risk. But the general picture is pretty optimistic.
- Imagine for a moment that we have two sets of alleles: one set increases IQ. The other set increases disease risk. The sets are not disjoint; that is to say there are some alleles which both increase IQ AND disease risk. If we simply select for IQ, we will select both from the set of genes that ONLY increase IQ with no effect on disease, AND for the set of genes that both increase IQ and disease risk.
If you add lower disease risk to your selection index, you will disproportionately select for alleles that are in the subset of variants that increase IQ without increasing disease risk. You'll also select for some alleles that have no effect on IQ but decrease disease risk.
Now you might be worried that perhaps there's some disease that isn't in our selection index but nonetheless has variants which increase both risk and IQ.
Fortunately, there's a way to deal with this too: add traits that are affected by pretty much all diseases to your selection criteria: for example:- Of cou
- Likelihood of developing chronic pain
- Self-rated quality of life
- Life expectancy
The two companies that currently offer polygenic embryo screening are already selecting against or for some of these to a limited degree by using disease risk as a proxy. But they could and should select on them directly to deal with the problem you described.
In general, selective breeding of animals for various traits have often managed to produce animals that excel in that trait but are noticeably less healthy overall.
So I agree that if you select very very hard on a small set of traits, you are likely to get unforeseen downsides somewhere down the road. But two things worth considering here:
- Some of the health problems in domesticated animals related to inbreeding rather than selection on a variety of traits.
- Health was basically not used as a selective trait for livestock except insofar as it affected the traits that people actually cared about (namely meat, egg and milk production)
- Domesticated animals have been pushed like a dozen or more standard deviations from their pre-selection means for many traits. We don't have the selection power to do that with embryo selection except on a time scale of hundreds of years. It's another story with like CRISPR editing or iterated embryo selection, so I do think we'll have to be much more cognisant of that issue down the road.
We certainly can try to weigh the odds and try to avoid the cases where things have Obviously Gone Wrong, but I reserve the right to worry about removing genes from the human gene pool whose function we don't understand.
We are not going to be removing any of these genes from the gene pool for a long time. It's going to take 20-50 years for this technology to become widely adopted around the world. And even then, you're unlikely to get the elimination of any genes without editing or iterated selection of some type.
This is quite a story.
I don't think my odds of lab origin are 99% yet, but I think after this article I'd move my odds from 80%->90%. I'd like to see confirmation by more sources before I move any higher. But the evidence looks pretty compelling with this point; the narrative is coherent, the counterarguments (of those I've read) seem weak. Though it's possible I've missed some stronger ones since most of the people in my information sphere seem to believe the lab leak hypothesis.
Was "avoiding anti-Chinese sentiment" really a motivation? The official explanation is that some Chinese person ate like a barbecued bat or got bit by a pangolin or something. I don't see how a lab leak would make people any more racist or hateful towards the Chinese than the official explanation did.
I suppose that it probably was a motivation even though it did not make much rational sense to me. I just wonder if that concern was more of a matter of political identity rather than a considered response.
career-ally
Professionally?
Thanks for such a high quality comment. I've heard that the termination rate for Down Syndrome pregnancies varies by country. For example, I've heard it's higher in most European countries than in America.
And that makes me wonder how many other conditions that would be true for. How many people would still select the embryo that is gay? I fear only very few. And would choosing against them make the world a better place? I doubt it.
I don't think everyone is going to make the same choice here. I suspect some parents would select for greater chance of same-sex attraction and some will select against it. Though I suspect that in most cases parents are just going to care much more about other traits so it won't be selected very strongly either way.
I see the situation with competing AIs as much more unstable than the one with two opposing armies or countries.
If you want to invade and take over another country, you have to send in your army to their territory to destroy their army, and then you either have to subdue the population or colonize the land with your people. This is hard because it takes a lot of time to move a lot of people, and people can only reproduce and grow up so quickly.
This just seems far, far easier with AI: you can literally "travel" from one datacenter to another in under a second and there exist various drivers that make it fairly simple to utilize other computational resources. An LLM like GPT-4 can make an additional copy of itself in someone else's data center in under an hour.
Once there's a slight imbalance in power, I would expect the situation to resolve itself with the more powerful AI completely overpowering and destroying the weaker AI.
This is the main reason why the prospect of "multiple AGIs" gives me no comfort. It just doesn't seem stable.
Wow. Ok, I guess my odds that this is actually an alien spacecraft went up a little bit.
It's interesting that at the end they quote a NASA official who stated that they haven't found any evidence of extraterrestrial life yet, directly contradicting the whistleblower. That means either the evidence the whistleblower has isn't sufficient to convince the scientists at NASA or the DOD isn't sharing it with them.
Couldn't GPU restrictions still make them more expensive? Like let's say tomorrow that we impose a tax on all new hardware that can be used to train neural networks such that any improvements in performance will be cancelled out by additional taxes. Wouldn't that also slow down or even stop the growth of smaller training runs?
No, the option to select against all diseases in proportion to their impact on quality-adjusted lifespan is the default. But parents can re-do the calculation to like take age of onset into account if they want. Or they could add other non-disease traits to their selection criteria (like intelligence, as estimated by some third party service).
I agree, it's very sub-optimal for parents to have to do all this themselves.
It's worth noting that most of the major news orgs passed on this story despite being offered the opportunity to cover it. We don't know why they did it yet, but given that various orgs have covered the Snowden documents and other whistleblowers that the government very much didn't like, my guess is they did it for reasons related to the quality of the story rather than any conversations with government officials who encouraged them not to cover it.
My priors against us having discovered alien tech are very high, though not literally infinite.
But I still don't have a clear story for exactly what's going on. Most of the videos of UFOs look pretty similar: silvery orbs flying around at very high speed. I haven't yet heard an explanation of how this could be explained by camera artefacts, weather phenomena, or anything else.
Other videos like this one released by the Navy show non-spherical objects that even rotate while moving. I struggle to think of what could be causing this.
I'm too lazy to look into it right now, but at the very least there's a scientific mystery here. Whether or not the explanation turns out to be interesting remains to be seen. There seems to be a big stigma against reporting UAP in the military, which some NASA officials think is hindering our understanding; with fewer recorded phenomena, it's hard to figure out what's going on.
I think people underestimate the degree to which hardware improvements enable software improvements. If you look at AlphaGo, the DeepMind team tried something like 17 different configurations during training runs before finally getting something to work. If each one of those had been twice as expensive, they might not have even conducted the experiment.
I do think it's true that if we wait long enough, hardware restrictions will not be enough.
I know Genomic Prediction at least has used "Quality Adjusted Lifespan" in their past papers, so I think they're used interchangeably.
Both they and Orchid provide the expected absolute lifetime risk of each disease in their reports, so parents can re-prioritize embryo implantation according to their preferences. Doing this methodically is pretty tricky though; every disease has a different age of onset distribution and a different impact on life expectancy and quality of life.
My hope is that one or both of them create more granular tools to help patients pick embryos according to their values and preferences. But so far the best you can do is just looking at the raw numbers and googling stuff about average age of onset etc.
Choosing a sexual partner IS an example of genomes being steered by parental choice.
In the process of this development, these networks assign themselves a physiological form gender; intersex people get a mix of attributes at this stage, but for most people, even for most trans people, this stage almost entirely selects one profile of sexual dimorphism; typically for people with XX chromosomes, this stage selects female, and for people with XY, this stage selects male. However, it's well known to science and can be looked up that sometimes people can be apparently entirely one body-form and have no desire or urge to transition, and yet have opposite chromosomes from their body's layout-presentation.
This sounds like you're talking about the SRY cascade without explicitly naming it. But that process starts at 6-8 weeks. Embryos are screened after 3-7 days of development. There's no way to see what is happening with the SRY cascade at that stage.
Maybe there are some genes that influence the course it is likely to take? I understand your concern.
But I also think some of this tech could HELP acceptance of trans people. I suspect that many of the people who are anti-trans are at some level worried about its effects on family formation. Without freezing gametes, many trans people will not be able to have biological children after medically transitioning. Gametogenesis could give anyone the ability to have kids if they want them, regardless of their gender identity or transition status. I think that would at least do a little to quell the hate against transgender people (though I understand most of the people who get upset about transgender people are not very rational about their views).
Is tendency to wish to transition one's gender genetic? I recall reading that the number of people self-identifying as trans has tripled in the last decade. That would suggest that the trait is probably not very heritable and would be hard to select against.
Parents have skin in the game when it comes to their children.
To me it seems unlikely that the nanotech necessary to do the massive somatic editing needed to replicate the effects of germline editing would not come far ahead of the tech necessary to just upload someone's brain into a computer. And if you have that, why would you even bother with the huge limitations of biological intelligence?
I suppose it's possible we might have some global ban on further improvements to computer intelligence, but in that case why wouldn't you just genetically engineer people capable of solving alignment? Why hang around at the proteins & genes level of tech long enough to make massive somatic editing work?
I think the reason that people use the same term for both is that historically, the lines between enabling, encouraging, recommending, de facto enforcing, legally enforcing and plain violently enforcing genetic changes have been rather fluid. It never started with enforcing it, but it tended to end up there.
Was there even a way to do voluntary eugenics before embryo selection? I guess maybe paying people to have more kids or fewer kids might count. But the options were pretty limited.
In your own writing, you had somebody comment, saying it should actually be a moral obligation to perform embryo selection for all who had the financial means. Your response was not "that would be a different evil thing, no". It was "I'm not quite sure I would agree with this yet, though I can see the case being made for it."
There is a difference between a moral obligation and a legal one. I think people have a moral obligation to donate money to cost-effective charities, but I also support their right not to do that.
The difference is understanding there ought to be a limit on how the state can be used to enforce other people to comply with your moral standards.
Also, as I already said I don't think this technology is yet cost-effective enough to warrant such a moral obligation.
Yet that story ended in real life with kids with disabilities being gassed to death.
You seem to be implying that there is somehow a direct causal link between films arguing for a right to die and the holocaust. I don't think that slope is as slippery as you portray it. There are plenty of people nowadays advocating for the right to die who don't believe in killing disabled people.
And again, that's not even what I'm arguing for. I'm merely arguing that parents should have the right to give their children better genetics so long as those changes have at least some benefit to the well-being of the child.
There are of course always going to be some traits that are on the borderline of what we might consider ethically permissible. For example, should deaf parents be allowed to purposefully have a deaf child? I lean towards yes, particularly if the child can choose to reverse the condition with some kind of implant or medical procedure later in life.
If you want a different word for this, please also state how you intent to ensure, in the long run, that the freedom not to use this tech is maintained. Not just as a theoretical legal right, but in practice. That this won't end with us standing in a classroom, and the teacher saying, in disgust, I cannot believe this child was not deselected, so I wouldn't have to deal with this neurodivergent mess.
To be frank, I have thought less about these kinds of issues because I am so worried about AI. In my mind, the greatest benefit of this technology is it might provide the human species with individuals capable of guiding us through the incredible technologically-induced upheavals we are likely to see over the next century. My greatest fear is that we simply are not going to have enough time for these children to grow up. Next to that, worries about discrimination in the classroom or workplace have seemed relatively minor.
Still, it deserves to be addressed. I don't think you're going to see the effect you've described without something stronger than simple embryo selection. By itself, embryo selection maxes out at about 1 standard deviation of gain on IQ. That will have a very noticeable impact, but given you'll be selecting on other traits besides just IQ, children born with its benefits will just appear to be unusually talented. The difference won't be big enough to mark them as like fundamentally different.
Even if you get in-vitro gametogenesis working, you'd max out at maybe 22 IQ points.
Of course the goal is to eventually get iterated CRISPR or chromosome selection or some other advanced techniques to work that would be truly transformative. But it will take quite some time for this tech to be scaled out to the point where selected children become the majority of new births. I would guess at least 40 years and possibly longer.
The only way I can see that happening is if we manage to put a global moratorium put in place on AI research and the improvement of computer hardware. But if we did manage to coordinate on such a wise proposal, I suspect it would be lifted when we figure out how to make an aligned ASI that acts in the bests interests of humans in general. It seems likely that we will solve that with a bunch of genetically enhanced geniuses around to work on the problem.
For the short window during which this might be a problem though, there are a couple of ways to deal with it:
- Have different schools for kids of roughly similar abilities. This is already standard practice in many countries such as Vietnam.
- Introduce universal basic income to ensure that no one ends up truly destitute due to genetic predispositions that are no fault of their own
- Once embryo selection or whatever technique we're talking about is cost-effective enough, subsidize access with government funds so that parents will never be denied access due to financial constraints. The same logic that compels us to fund public schools would also compel us to subsidize access to this tech if it is cost-effective enough; society as a whole has a strong interest in ensuring the next generation is healthy and productive.
I'm a little too tired to read through the study you sent me. I suppose it's possible she either got alcohol wrong or that new studies have come out since the book was published which should change the conclusions.
I would note that it's very hard to assess the impacts of drinking in the current environment because women who adhere to health advice about not drinking are probably going to pass on genes to their children that make them less prone to the very neurodevelopmental disorders the studies purport to measure.
You could do a study where you find women who you think are at high risk of drinking during pregnancy and then perform an intervention to encourage some of them not to drink. But given that ERBs adhere to a strict Copenhagen interpretation of ethics I doubt such a study design would ever be allowed (even though conducting it would improve children's health if alcohol does indeed have a negative impact).
But there is a slippery slope towards a scenario where people select for sex, skin (and hair and eye) colour, not being queer, not being neurodivergent... and I do find that a dystopian scenario where we would lose something valuable that enriches our world.
Yes, this is actually a fairly common critique of embryo selection. One useful intuition pump I've found helps me think about it is the reversal test; should we make people sicker or more mentally distraught to enrich the world? It's a bit odd to imagine that evolution somehow put us at the perfect equilibrium where any increment or decrement in mental illness rates would result in a worse society. It's especially odd to think that since evolution doesn't care at all about either of those things except insofar as they affect inclusive reproductive fitness.
Also, my experience so far just talking to people makes me think parents are going to have different priorities regarding the traits they select for.
But I assure you, if my parents had gotten to pick, they would also have made sure that I am not gay. Or tall. Or enby. Or have ADHD and autism. Things that were challenges, to be sure, but that I do not see as a net negative. I also know that both my parents found it disagreeable that I am highly gifted, and smarter than them; my father explicitly considers it a disease state and unpleasant complication, and would certainly have selected against it. I am very glad that he could not.
I'm sorry about your parents. That sounds like an unpleasant experience.
I don't think the thought experiment of "erasing" someone like you from existence is really a very good test of the morality of embryo selection. You are a person with decades of memories and ties to the community of people around you. In my view the morality of "erasing" you feels a lot different than making a choice between two embryos. Unless you believe in souls or something, an embryo is almost pure genetic potential. It has no internal organs, let alone a brain. Even the placenta hasn't formed yet.
We also already have a scenario where it is incredibly difficult for a poor person to have proper upwards social mobility. If their competitors are literally superior from birth, prior to also getting their personal tutoring and private schools and trust funds and brain implants... at some point, no amount of hard work will make you competitive anymore, the different classes will become insurmountably separated and fixed. Not because the rich parents want this - they just want the best for their children, who wouldn't. But if access to this tech is not fair, and its legitimate usage is not carefully reflected and set, the changes could be dystopian indeed.
I would point out that all the dynamics you described are already true to some degree; there are some people born with such extreme genetic disadvantages (through a combination of parentage and bad luck) that there are some paths in life simply closed to them.
Of course embryo selection will increase variance, so your point is still well taken. I've spent a fair amount of time thinking about this and the obvious solution here is just to work very hard to make this technology cheaper and better. If we make enough progress on that front then we can just have the government subsidize the technology and give free access to anyone that wants it.
Inequality WILL still increase in the meantime, but there are some dynamics that I think help us here:
- There is a ceiling on improvements through embryo selection or editing. That limit is determined by the amount of variance in the human gene pool. The ceiling is very high, but its existence makes it plausible that some people will get there first and others will catch up
- To push beyond that ceiling you'll either need to generate new genetic variants and test them out in people. This will require the cooperation of a very large number of people. To make good predictors today, you need literally a million people or more in a database. To a first approximation I would guess you'll need that many if you want to test out a ton of new genetic variants and have enough statistical power to distinguish true positives from false positives.
- The main way rich people can get an advantage in embryo selection is by harvesting more embryos or getting access to better predictors. The predictors are all made from huge databases, most of which are public. So it's unlikely rich people could maintain a monopoly on the best predictors. Also, it's hard for them to get a big advantage by selecting from a lot more embryos. You can of course pay to use a better clinic, and you can pay to go through more egg retrievals to harvest more embryos. But there are steeply diminishing returns; you're still sampling from a normal distribution. The expected maximum value of N samples from a normal distribution is sqrt(ln(N)). That's an INCREDIBLY slow growing function. If you go from 10 embryos to 1000, the benefit only increases by 70%.
Will rich people still have an advantage? Yes. But genetic enhancement does not have the same runaway "intelligence explosion" dynamics that AI does.
I think this is probably true at the moment given the current efficacy and the prices of IVF.
The only reason I hesitate is because I think the tendencies and aptitudes of people at the top have a gigantic impact on the rest of society and that improving those tendencies would have a very large impact on the world (especially if parents of children likely to end up in positions of power select for traits like kindness and pro-social tendencies).
But if you ignore that for a moment, then you're of course correct that embryo selection can come nowhere close to the efficacy of distributing bed nets or supplementing iodine-deficient populations.
However, it's going to take some time to bring down the price of embryo selection and future technologies for genetic improvement, so I think it's probably good to start on that now.
Granted, this is all kind of ignoring the 800 pound gorilla in the room which is AI. For embryo selection to really matter, there needs to be time for these children to grow up. If we make ASI in the next 20 years, there's probably not much point. I'm still hopeful that we can get a pause in place on AI development. But without one this whole endeavor has pretty low odds of achieving anything IMO.
Yeah. Wait until they're grown, then do runtime modification. Stem cells define genomes, genomes plus lifetimes define people, people define consent.
If I thought this was actually feasible with anywhere near our current level of technology then perhaps I would agree. But it is simply impossible without god-like biotech. Half the genes that show up in intelligence GWAS (or any other trait for that matter) are only active during development, particularly during early development in the womb and early childhood. The other half affect cells that literally last a lifetime (namely neurons).
Modifying one gene that is active in one particular stem cell population can currently be done for like cancer immunotherapy, but it costs like $500k and you have to radiate the person to kill their non-modified stem cells.
If we're going to assume that someone creates aligned ASI I think this whole conversation is somewhat moot. If by some miracle we manage to solve the alignment problem before the mad scientists at OpenAI create Cthulhu, one of the first things people are going to use it for is to upload their own brains to a computer. Like why would you stick with spongy meat if you can just run a digital copy of yourself a million times faster?
I think there's a pretty low chance that ASI goes well. Even if we manage to align the interests of ASI with its creators, whoever gets control of it is going to rule the world. This fantasy dream where everyone benefits equally seems very unrealistic to me. It's way more likely that we have one or perhaps a few quasi-omnicient immortal dictators.
In other words, this technology as it is now should not be used, and we should wait for the drastically much more difficult version that allows editing genes at runtime before we deploy it at scale. In the meantime, I strongly support the coalition against doing offspring editing.
I don't follow this argument. You're saying we shouldn't edit the germline or select embryos because the child has not provided consent?
If so we are prioritizing something that's impossible to get (consent from a ball of stem cells) over the expected future wellbeing of a person.
The argument that parents need to get consent from their children before doing things that the parents believe are in the interest of the child also undermines the basic rights of parents. It is very common for adults to do things children don't want or don't like because they believe it is in the best interest of the child.
Undoubtedly some of those things are in fact not good for the child and are only done in the name of serving the child's best interest when in fact they serve some needs of the adult. We have rules around what parents are allowed to do to their children that reduce some of the worst abuses of this power. But the general principle of adult control over children seems pretty important to how society functions.
If this service can't be made free, it should be destroyed on deployment, to prevent moloch.
How many of the important technological advancements that have been made in the last 10,000 years could hold up to this standard? If every technology had to be free and universally available on launch, we would have no technology. There would not be medicine, nor manufacturing, nor clothing, nor cooked food. This is an untenable standard.
If a couple chose to pick the embryo they expected to suffer the most in life, is it epilogenetics?
If someone asks me I would of course say no because I would like the term to encompass traits that we can reasonably expect the child in question to be glad we gave them.
But I would note that even if use of the term triggers discussion of the question you posed, I still think that is shifting the conversation in the right direction; away from "is embryo selection bad because it reminds me of this other thing I don't like" and towards "should parents be allowed to select for traits that their children probably wouldn't like?"
Interesting viewpoint. I think your point about the morality of having children despite the high natural miscarriage rate is a good one.
My basic view is that human moral value develops throughout pregnancy (and indeed continues to develop after birth). I don't think there's a simple binary switch from "no value" to "value". I'd treat it more like a gradual ramp-up beginning with brain development during pregnancy.
I'm curious how you feel about culturing of naive embryonic stem cells. It's possible to culture cells from a very early embryo and maintain their epigenomic state. One might then perform some editing on each, then grow each into a colony of perhaps 100 cells before destructively sequencing some of the stem cells and then performing subsequent edits on the stem cells in which the edits successfully took place.
If done correctly, the process would result in an embryo with much better prospects for a healthy and happy life. One embryo goes in and one embryo goes out. But the sequencing in the interim steps would require the destruction of naive embryonic stem cells.
Would you consider such a process morally permissible?
The paper you linked outlining the limitations of polygenic embryo screening mostly rests its conclusions on the supposed impossibility of showing that embryo screening actually works.
Again, I refer back to tests of polygenic risk scores in siblings. If predictors work in that population, they should be considered clinically validated. This kind of validation is standard practice in other areas of data science. I'd appreciate it if you or someone else questioning PGT-P could outline exactly why they believe sibling validation of polygenic scores is insufficient evidence to justify its clinical use. It is literally a randomized control trial for genes.
My response to other criticisms in the paper:
"Furthermore, statistical manipulation of genetic data may limit the detection of rare pathogenic gene variants"
De novo mutations are exceedingly rare. The average person has about 70. The expected effect from missing these mutations is so low that it's barely worth considering, especially compared to the expected benefits of simply improving predictors and adding more traits to the selection index used in PGT-P.
not only is it difficult to assess the clinical validity of PRS-ES in terms of the outcomes in question, it is also possible that clinical validity would be limited by the different effects of future environment on gene expression, compared to the past.
Yes, I agree that this is a fair critique of embryo selection, particularly for the diseases of old age. But the obvious solution here is just to apply some time discount factor; weight traits like depressive tendency, obesity, and intelligence more heavily than prostate cancer and heart disease, since the former will have an impact much sooner.
Mathematical modelling of PRS-ES has been attempted and indicated extremely limited utility in terms of non-pathological trait selection (Karavani et al., 2019), such as height and intelligence quotient (IQ).
The Karavani study used predictors that were already outdated by two years when the study was published. They are even more outdated today. Today you could expect +6 IQ points and +3.7 cm using state of the art predictors and the same assumptions made in the Karavani study.
Furthermore the overall expected gain increases as you add traits to your index.
Most of the conditions which can be assessed using PRS have a significant gender association.
Yes, this is why Genomic Prediction adjusts for sex in their index. I assume Orchid does the same thing though I know less about their selection methodology.
As elegantly described by Turley et al. (2021), the purported benefit of PRS-ES is commonly calculated and presented as a difference not between two average embryos, but rather a difference between the highest and the lowest possible risk embryos, thus maximizing the theoretical benefit of the test.
I've looked at the Genomic Prediction report and this is NOT how the results are presented. The expected reductions are calculated using actual siblings, and a baseline is an average person, not between the highest and lowest risk embryo.
I have asked and risk has NEVER been reported in the way described. It's amazing how even in otherwise reputable journals these easily falsifiable rumors are allowed to spread.
It is also important to note that all embryos produced by a couple are genetically related and share on average 50% of SNPs. One must conclude that owing to inherent limitations of the PRS-ES models and limited variation in the genetic makeup of embryos produced by a couple, the clinical utility of PRS-ES is almost certainly diminutively small (Karavani et al., 2019).
Wrong. sibling variation is 1/sqrt(2) times that of the regular population, which is plenty for selection to result in substantial disease risk reduction or trait improvements. See Lello et al for more details
I could go on about this paper, but over and over again I see the authors making unsubstantiated claims contradicted by the evidence. There ARE a few legitimate issues with embryo selection, but the paper focuses most of its energy on non-issues.
I pretty much agree with the rest of your comment, though I think the situation for patients is better than you think. I've spoken with the genetic counselors at both Orchid and Genomic Prediction and found them to be very straightforward about the expected benefits and risks. The real issue with Genomic Prediction is they seem to have taken down their page showing the expected disease reductions from polygenic embryo screening. I don't know why.
I don't understand what this statement has to do with your research funding. Can you explain?
No. This is drawn from my own personal experience reading comments by otherwise knowledgeable academics or professionals in the ART industry. It's also something I read online a lot in forums such as reddit. Inevitably one of the top comments on most articles about human genetic engineering is along the lines of "we don't understand anywhere close to enough about genetics to make the proposed changes".
If this is really important to you I can probably find some examples.
This is not meant as a literal statement. I'm just trying to convey that the magnitude of benefits is still rather small.
There are obviously many examples of geniuses who didn't come from unusually impressive families.
Fair point. What I was really trying to say is that in the context of embryo selection, you can be pretty confident that selecting embryos based on that predictor will actually result in more or less of the trait depending on what you're selecting for.
But you're right. I've edited the post to clarify that.
I made my reply to your comment into a standalone post
Hi Ruby,
Sorry to hear your IVF process didn't work out. UCSF was in the top 59% of clinics nationwide in 2020 and the top 38% in 2019, so while the clinic you chose may not have been the best, you at least didn't pick a bad clinic.
Your experience is unfortunately fairly common among IVF patients. Most parents using the procedure are just hoping for at least one child through the process, and many don't have enough embryos to even consider polygenic screening.
I really hope someone does a clinical trial of embryo splitting soon. There's a roughly 50% chance of success using the process in animals. I bet with research we could get it up to 80-90%, which would make it viable for increasing live birth rates among parents who don't have many embryos. That's the type of procedure which would have improved the odds of success for parents like yourselves.
Yes, thanks for the correction
Sorry I couldn't get it out earlier! I meant to release this in June of last year but the research project into which IVF clinics are best turned out to be quite a bit more difficult than I anticipated.
It seems like there is likely a massive inefficiency in the stock market right now in that the stocks of companies likely to benefit from AGI are massively underpriced. I think the market is just now starting to wake up to how much value could be captured by NVIDIA, TSMC and some of the more consumer facing giants like Google and Microsoft.
If people here actually believe that AGI is likely to come sooner than almost anyone expects and have a much bigger impact than anyone expect, it makes sense to buy these kind of stocks because they are likely underpriced right now.
In the unlikely event that AGI goes well, you'll be one of the few who stand to gain the most from the transition.
I basically already made this bet to a very limited degree a few months ago and am currently up about 20% on my investment. It's possible of course that NVIDIA and TSMC could crash, but that seems unlikely in the long run.
Wow, very interesting! Thanks for sharing.
Apparently the scientific name for this is "Sesquizygotic Twinning".
There's a great video explanation of this process. I'm still not entirely clear on how the maternal set of chromosomes replicates itself such that it can form diploid cell lines with both sperm. Usually the sperm and egg chromosomes pair up after merging of the pronucleus of each. The authors of the paper you linked propose some strange "triploid spindle apparatus" as an explanation of how this happens. But it's not clear to me how the chromosomes of each haploid pronucleus replicate before joining with their counterparts. And I'm too lazy to try to figure it out right now.