We need a standard set of community advice for how to financially prepare for AGI 2021-06-07T07:24:03.271Z
Estimating COVID cases & deaths in India over the coming months 2021-04-24T21:35:21.267Z
A Brief Review of Current and Near-Future Methods of Genetic Engineering 2021-04-10T19:16:01.169Z
Why Selective Breeding is a Bad Way to do Genetic Engineering 2021-03-05T02:30:38.775Z
Human Genetic Engineering: Increasing Intelligence 2020-12-05T22:06:10.772Z
The Case for Human Genetic Engineering 2020-08-28T22:21:35.782Z


Comment by GeneSmith on Open and Welcome Thread – June 2021 · 2021-06-26T17:48:25.225Z · LW · GW

How would AI deal reconcile competing human interests? What does this mean?

It was a typo. It was meant to say "How would AI reconcile competing human interests?"

Comment by GeneSmith on Two non-obvious lessons from · 2021-06-24T03:37:31.138Z · LW · GW

Thanks for building it! I found Microcovid to be the single most useful tool to combat the pandemic prior to getting vaccinated. Even after getting vaccinated it was useful.

Comment by GeneSmith on Why did no LessWrong discourse on gain of function research develop in 2013/2014? · 2021-06-24T03:34:13.525Z · LW · GW

I think the fact that mistakes like this are so understandable is precisely why gain of function research is dangerous. One mistake can lead to a multi-year pandemic and kill 10 million people. With those stakes, I don't think anyone should be doing gain of function research that could lead to human deaths if pathogens escaped.

Comment by GeneSmith on Covid 6/17: One Last Scare · 2021-06-22T17:16:21.161Z · LW · GW

If what's written in this post is true it's a major problem for the long-term goals of MIRI and the bay-area rationalist community in general.

Comment by GeneSmith on Why did no LessWrong discourse on gain of function research develop in 2013/2014? · 2021-06-21T18:08:21.275Z · LW · GW

Wow, this is quite the post! I've been looking for a post like this on LessWrong going over the lab leak hypothesis and the evidence for and against it, but I must have missed this one when you posted it.

I have to say, this looks pretty bad. I think I still have a major blindspot, which is I've read much more about the details of the lab leak hypothesis than I have about the natural origin hypothesis, so I still don't feel like I can judge the relative strength of the two. That being said I think it is looking more and more likely that the virus was probably engineered while doing research and accidentally leaked from the lab.

Thanks for writing this up. I'm surprised more of this info doesn't show up in other articles I've read on the origins of the pandemic.

Comment by GeneSmith on Why did no LessWrong discourse on gain of function research develop in 2013/2014? · 2021-06-20T18:23:35.326Z · LW · GW

Eliezer seemed to think that the ban on funding for gain of function research in the US simply led to research grants going to labs outside the US (Wuhan Institute of Virology in particular). he doesn't really cite any sources here so I can't do much to fact check his hypothesis.

Upon further googling, this gets murkier. Here's a very good article that goes into depth about what the NIH did and didn't fund at WIV and whether such research counts as "gain of function research".

Some quotes from the article:

The NIH awarded a $3.4 million grant to the non-profit organization EcoHealth Alliance Inc. over six years, funding research to study the risk of bat coronavirus emergence. This sum of money was administered by the National Institute of Allergy and Infectious Diseases (NIAID), the institute of the NIH directed by Fauci. EcoHealth Alliance then awarded part of the money to the Wuhan Institute of Virology ($598,500 over five years).


This framework defined PPP as a pathogen that is “likely highly transmissible” and “likely highly virulent and likely to cause significant morbidity and/or mortality in humans”. An enhanced PPP is one that results “from the enhancement of the transmissibility and/or virulence of a pathogen”. Under this framework, enhanced PPPs do not include pathogens that are naturally circulating and have been recovered from nature.


Stanley Perlman, a microbiologist at the University of Iowa, told that EcoHealth’s research was about “trying to see if these viruses can infect human cells and what about the spike protein on the virus determines that.” According to, Perlman did not think there was anything in the EcoHealth grant description that would be gain-of-function research.


A 2017 study published by researchers at the Wuhan Institute of Virology, listing the NIH as a funding body, appears related to this grant[4]. The researchers wanted to test whether the spike protein of new wild coronaviruses, which they isolated in bats, would allow the coronaviruses to enter human cells.

The problem with studying coronaviruses is that they are hard to culture in the lab[5]. To carry out their study, the researchers used the genetic sequence of a coronavirus (WIV1) that does replicate in vitro (in the lab) and inserted the spike proteins of the newly isolated viruses. In this way, they could test whether the newly isolated viruses could replicate in human cells in a lab dish.

Data included in the publication[4] showed that these experiments did not enhance the viruses’ infectivity. The experiments therefore did not make viruses more dangerous to humans or more transmissible.

There are differing opinions on whether or not what the researchers at WIV did counts as gain of function research:

However, Richard Ebright, professor of chemistry and chemical biology at Rutgers University and a critic of gain-of-function research, told the Washington Post that “the research was—unequivocally—gain-of-function research. The research met the definition for gain-of-function research of concern under the 2014 Pause.”

And Kevin Esvelt, a biologist at the MIT Media Lab, stated in a fact-check by PolitiFact that “certain techniques that the researchers used seemed to meet the definition of gain-of-function research”.

On the other hand, Joel Wertheim, an evolutionary biologist at the University of California San Diego, told PolitiFact that the experiments carried out in the 2017 study, despite using recombinant RNA technology, don’t meet the criteria for gain-of-function research in virology.

So to summarize: from what we know, researchers at WIV inserted a spike protein from a naturally occuring coronavirus into another coronavirus that was capable of replicating in a lab and infecting human cells. But the genome of this resulting virus seems too different from that of coronavirus for it to have been a direct ancestor of the pandemic causing coronavirus.

Overall I don't feel like enough people are linking their sources when they make statements like "I'd give the lab leak hypothesis a probability of X%".

Comment by GeneSmith on We need a standard set of community advice for how to financially prepare for AGI · 2021-06-11T01:26:12.452Z · LW · GW

I had always assumed that any organization trying to destroy the world with an engineered pathogen would basically release whatever they made and then hope it did its work.

IDK, this topic gets into a lot of information hazard, where I don't really want to speculate because I don't want to spread ideas for how to make the world a lot worse.

Comment by GeneSmith on We need a standard set of community advice for how to financially prepare for AGI · 2021-06-10T22:51:31.710Z · LW · GW

For example, if there's a superintelligent AI capable of unilaterally transforming all matter in your light cone into paperclips, is there any sense in which you have enough power to enforce your ownership of anything independent of such an AI?

No, which is why I "invest" in making bad outcomes a tiny bit less likely with monthly donations to the EA long-term future fund, which funds AI safety research and other X-risk mitigation work.

Comment by GeneSmith on We need a standard set of community advice for how to financially prepare for AGI · 2021-06-10T22:47:16.225Z · LW · GW

It does take substantially longer to get to Mars than to get to any isolated pockets on Earth. So unless the pandemic's incubation period is longer than the journey to Mars, it's likely that Martians would know that passengers aboard the ship were infected before it arrived.

Comment by GeneSmith on We need a standard set of community advice for how to financially prepare for AGI · 2021-06-10T22:42:18.286Z · LW · GW

Wait, what? The sum of the net worth of those who consider themselves members of the rationalist community is MUCH greater due to crypto than it was before. What definition of "success" are you using which so devalues that outcome?

I'm mostly referring to the narrative from this post. There have been some successes, but those have mostly been due to a very small number of huge winners. And in the case of the biggest winner of all, Vitalik Buterin, he actually ended up joining the rationalist community AFTER he started Ethereum.

Do you really want default advice? I'd rather have correct advice, and I'd rather still have correct personal behavior, regardless of advice. "Correct", in this case, means "best possible experienced outcome", not "sounds good" or "best prediction at this point but still wrong".

I probably wasn't as clear as I could have been in the original post. What I mean by "default advice" is a set of actions people can take if they believe there is a decent chance AGI will be created in their lifetimes and want to prepare for it but are not willing to spend all the time to develop a detailed personal plan.

For example, if you believe the efficient market hypothesis, you can act on that belief by buying low-cost index funds. I'm thinking it would be useful to have a similar easy option for people who buy that we will likely see AGI in our lifetimes.

Why would the stock go up, as opposed to the employees in control just absconding with (or being absorbed into) the AGI and the stock becoming irrelevant? Or someone else learning from the success and turning it into an actual financial boon. Or any of a billion other sequences that would make it a dumb idea to pick a stock based on number of papers published in a narrow topic that may or may not correlate with AGI creation.

True, and this is why I said I am not particularly satisfied with my current strategy. I still think in the scenario where AGI has been created or is close to being created, Google's stock price is likely to go up more than an index fund of all stocks on the market.

Comment by GeneSmith on Open and Welcome Thread – June 2021 · 2021-06-07T18:06:51.597Z · LW · GW

It seems unlikely to me that the things we do post-AGI would remain the same. If you had the lamp from Aladdin and the genie actually worked as described, would your life remain the same? Would you still spend your time baking cakes?

I know for myself personally I would try to enhance the capabilities of myself and those I care about (assuming they were amenable). To comprehend calculus as Newton did, or numbers as Ramanujan did would I think be an experience far more interesting than baking cakes or taking part in my usual hobbies. And there are thousands of other ways in which I would use my allotment of AI power to change my own experience.

I suspect this would be true for many people, so that self-augmentation via AGI would fundamentally change the experience of being human.

What does such a world look like? I have a very hard time visualizing it. Would power tend to concentrate even more than it does now? How would AI deal reconcile competing human interests?

Comment by GeneSmith on We need a standard set of community advice for how to financially prepare for AGI · 2021-06-07T16:09:48.116Z · LW · GW

I know from some interviews I've watched that Musk's main reason for investing in AI startups is to have inside info about their progress so he can monitor what's going on. Perhaps he's just not really paying that much attention? He always has like 15 balls in the air, so perhaps he just doesn't realize how bad Vicarious's safety work is.

Come to think of it, if you or anyone you know have contact with Musk, this might be worth mentioning to him. He clearly cares about AI going well and has been willing to invest resources in increasing these odds in the past via OpenAI and then Neuralink. So perhaps he just doesn't know that Vicarious AI is being reckless when it comes to safety.

Comment by GeneSmith on We need a standard set of community advice for how to financially prepare for AGI · 2021-06-07T15:52:12.200Z · LW · GW

Which biotech in particular?

As far as genetic engineering goes, I was thinking about writing up a post on that myself to the effect of "why you should [or should not] consider having your kids via IVF.

But I haven't done much research on transformative biohazards like engineered pandemics and am wary of writing such a post.

Comment by GeneSmith on Open and Welcome Thread – June 2021 · 2021-06-07T00:46:22.818Z · LW · GW

Anyone have reading recommendations for fiction or even just a summary description of what a positive future with AI looks like? I've been trying to decide what to work on for the rest of my career. I really want to work on genetics, but worry that, like every other field, it's basically going to become irrelevant since AI will do everything in the future.

Comment by GeneSmith on Open and Welcome Thread - May 2021 · 2021-06-06T02:15:28.514Z · LW · GW

Anyone have reading recommendations for fiction or even just a summary description of what a positive future with AI looks like? I've been trying to decide what to work on for the rest of my career. I really want to work on genetics, but worry that, like every other field, it's basically going to become irrelevant since AI will do everything in the future.

Comment by GeneSmith on TEAM: a dramatically improved form of therapy · 2021-06-01T07:32:32.452Z · LW · GW

Your not completely fictitious example is interesting.

I have these types of self-critical thoughts fairly often. Just reading your “validation” of the thought makes me feel oddly calm.

My prior for this sort of “novel therapy they works so much better than others” is that it probably doesn’t. Scott Alexander wrote a blog post about this back on the day in which he noted the effect sizes of new therapies almost always decline over time as they are studied by outside researchers rather than the inventors of the new form of therapy.

But perhaps I am wrong. Or perhaps the technique works much better for people who like that type of therapy. That’s certainly something we see evidence of in the therapy literature.

Thankfully I am not at a point in my life where I actively need therapy anymore, but if I do get back to that point I will give this a look.

Comment by GeneSmith on Core Pathways of Aging · 2021-05-31T05:55:24.108Z · LW · GW

How do the ROS and/or damaged molecules move between compartments, e.g. nucleus/cytoplasm/extracellular? I have seen very little on this, and consider it a major blindspot. I’m not sure if it’s a blindspot for the field or if I just haven’t found the right cluster of papers.

Man, I wrote up a whole summary of the mitochondria free radical theory of aging for you after reading this paragraph, then read the rest of your post and realized you already know about it. I'm surprised though that you still have questions about this, because the mechanism of export is described in Aubrey de Grey's book. At least it's described in his book "Ending Aging". I haven't read the other one, so I'm not sure if it contains less information.

A very quick description is that cells overtaken by clonal mutant mitochondria export electons from their cell membranes to keep themselves alive via the Krebs cycle (since the electron transport chain in mitochondria, which normally receives these excess electrons, is shut down).

The receptor molecule for these excess electrons is oxygen. These oxygen molecul\es with extra electrons are very powerful free radicals, and end up reacting with whatever molecule they bump into first. Unfortunately some decent fraction of these reactions are with low-density lipoproteins, which are then transported by the bloodstream to a much larger area of the body, and eventually deposited on blood vessel walls, contributing to atherosclerosis.

I'm glossing over a lot of details here, so if you want to read more about it, check out Chapter 5 of Ending Aging.

This book did come out in 2008, and I imagine quite a few new things have been learned in that time, so I'm not sure if this theory is still accepted. I'd be interested to know if there's been any follow-up research.

I don’t know whether there’s any evidence that these molecules actually accumulate long-term. (Just because they’re not broken down doesn’t mean they’re not simply excreted.) I haven’t seen direct evidence, but I haven’t searched very carefully either, and I haven’t seen direct evidence against.

If I understand correctly, accumulation of A2E in the lysosomes of microglia in the retina is one of the main causes of macular degeneration. There is also a big section of Ending Aging dedicated to the topic of lipofuscin in general (non-digestible material that accumulates in the lysosomes of long-lived cells)

Final Thoughts

This is an amazing write-up. I am very surprised that De Grey's book didn't mention transposons. Is that new?

Regardless, it doesn't seem like a stretch at all that they could play a key role in aging.

One thing I noticed was missing from your post: any mention of the role of information loss in the epigenome as part of aging. That seems to be Dr. Sinclair's main theory for the root cause of aging. Sounds like that could actually be closely intertwined with transposon activity since transposons are repressed by methylation or histone structure most of the time.

And of course, since I am always thinking of genetic engineering, this whole post made me think that removing transposons from the human genome via genetic editing, should we find a way to do it safely without affecting desired function, would be a great victory against aging. Evolution has not and will not act in our best interest, and there is no better example of this than the existence of transposons. They are basically parasites within our genomes.

Comment by GeneSmith on People Will Listen · 2021-05-30T19:03:09.392Z · LW · GW

I know I'm kind of late to the party here, but here is the best argument against the future value of Bitcoin I've yet read

This is specifically about Bitcoin, and does not apply to other coins that don't have a fixed supply, or those that use a different consensus mechanism.

For anyone who stumbles across this comment in the future, the argument is basically that there is no market mechanism to ensure transaction fees stay high enough to maintain block security. The argument hinges on the assumption that miners need to be rewarded at least 1% of daily transaction volumes for them to be willing to spend money on the ASICs needed to mine blocks.

Comment by GeneSmith on A Brief Review of Current and Near-Future Methods of Genetic Engineering · 2021-05-25T05:02:27.662Z · LW · GW

Speaking of epigenetics, I've just discovered the existence of another class of methods, epigenome editing... And then there's the topic of nonheritable (and possibly temporary) genetic modifications made to mature organisms. If what you care about is biological intelligence increase, somatic gene-hacking seems likely to get there before germline gene-hacking, because you don't have to wait for your first generation to grow up.

I read something relevant to this idea tonight that I think makes it less likely we will be able to significantly impact intelligence with epigenetic editing. A paper in PNAS from last year looked at which functional regions of the genome saw enrichment of educational-attainment associated SNP hits:

The EA3 study on educational attainment, a highly polygenic trait, is another notable recent example of this type of analysis (39). A very large number of category enrichment analyses was performed on 1,271 independent genome-wide significant signals detected in a GWAS of 1.1 million individuals with educational attainment data. The authors highlight two broad findings. First, the most significantly prioritized genes that were implicated as causal show trajectories of expression in the brain that are increased before the late prenatal stage of development and decline thereafter. Weaker, newly discovered, associations showed no such trajectory. This suggests a modestly disproportionate influence of brain development relative to active brain functioning in determining differences between individual abilities underlying educational attainment, which is perhaps not surprising.

This suggest that even if we were somehow able to inject some epigenome modifying vector into brains capable of modifying a significant fraction of neurons and even if the inevitable cell mosaicism induced by such changes had no negative impact on cognitive function, we would STILL be severely limited in the proportion of genetically influenced intelligence we could impact.

Not to mention it seems very likely that the cost of modifying 86 billion cells in the brain would far exceed the cost of sequencing embryo DNA.

Comment by GeneSmith on Open and Welcome Thread - May 2021 · 2021-05-15T22:08:25.256Z · LW · GW

Anyone have ideas about how to protect oneself against the higher-than-average inflation we'll likely experience in the next couple of years? I have a fair bit of cash and a fair bit of low-interest debt and I'm wondering if there are any easy no-brainer moves I could make to reduce my expected losses to inflation.

Comment by GeneSmith on Life and expanding steerable consequences · 2021-05-08T08:47:39.084Z · LW · GW

By picking these conditions precisely, we might cause the mold to spread to only the northern hemisphere of Mars, or to grow only at low altitudes, or only at high altitudes. In each case, the only thing we are transporting to Mars is a single specimen the size of a small rock. We are not ourselves spreading the mold over a mountain range or over the low-altitude parts of the planet, but by tweaking the configuration of atoms within this initial specimen we can choose how and where the mold will spread. In this sense the mold has expanding steerable consequences because a physically small specimen can be altered in a way that predictably steers large-scale effects over a long time horizon.

I actually don't think this is a very good example because once the mold takes root on Mars, Darwinian processes will take over and any mutations with a strong reproductive fitness advantage (such as those that allow the mold to expand to new environments) will be selected for.

But I agree with your general point and think the mold analogy is apt. There's a phrase I heard somewhere in an interview Elon Musk gave where he jokingly said "we're just the biological bootloaders for digital superintelligence." I kind of wonder what that life will look like. It will probably be so unimaginable that it's useless to think about it, but given the types of reinforcement learning systems we have today, I can't help but wonder if the ultimate aim of future digital systems will not be some grand ambition or even reproductive immortality but rather some silly poorly thought-out human goal that is mindlessly pursued until the heat death of the universe.

In some ways, that's the true fear of all AI researchers: that we will not only misalign AI, but that we will do so badly that the digital gods we create will pursue some existentially tragic goal, chewing up all the universe's resources in its light cone.

Comment by GeneSmith on Open and Welcome Thread - May 2021 · 2021-05-08T08:35:40.861Z · LW · GW

Walking and pacing. This is a big one for certain people I know, and used to be much more important for me. Basically, you just walk around, either in a tight circuit inside, or over a longer distance outside. Some people say this is the most effective method for them

Yes in fact having just finished Steve Jobs's biography I can say that this was by far his favorite method of having a meeting. Same for Bill Gates, which you'll observe if you watch the Netflix documentary "Inside Bill's Brain".

It has also been a favorite pastime of Andrew Wiles, the mathematician who cracked Fermat's Last Theorem after seven years of working on it in secret.

Comment by GeneSmith on Covid 4/29: Vaccination Slowdown · 2021-05-03T05:37:08.153Z · LW · GW

Jesus, I had no idea that Native Americans were twice as likely to die of COVID as the average American.

Comment by GeneSmith on Covid 4/29: Vaccination Slowdown · 2021-05-03T05:24:33.697Z · LW · GW

I've also been very surprised by this. It almost makes me wonder whether a large part of the population has natural immunity either genetically or from some past viral event that infected a large enough part of the country to drop the R0 of COVID below 1.

Comment by GeneSmith on Covid 4/29: Vaccination Slowdown · 2021-05-03T05:21:57.954Z · LW · GW

That Sciencemag blog you linked on the Brazilian regulators choosing to reject the Sputnik vaccine was fascinating.

OK, now we need to talk about how you make big piles of virus if you’ve kept them from replicating. That’s an interesting question that has a slick solution: you’re going to be using human cells (often the HEK293 line) to expand your virus production, and what you do is engineer those human cells so that they make the missing E1 protein that the virus needs to replicate. So as long as you’re growing up virus in these engineered cells, you’ll make more, but if they infect normal human cells that haven’t been jiggered to make a key viral protein (as in when you inject them as a vaccine) they’ll stall out on replication immediately. Problem solved!

Mostly. There are still places where this can go wrong. Double-stranded DNA breaks, which can happen more or less randomly, are generally repaired by processes called “homologous recombination” and “nonhomologous end joining”, and these can lead to mix-and-match behavior between DNA from different sources. This process can be deliberately harnessed for gene editing – that’s what the classic CRISPR enzyme Cas9 does – but it can also be a source of trouble in a system like this one. There is a chance that the occasional viral particle might be able to regain the DNA sequence for the E1 protein by picking it up from the human-cell background. If that goes right (well, wrong), then that will turn it back into a replicating virus, and that’s just what it will do in your cell culture tanks.

According to the rest of the article, there are existing engineering solutions to make these types of undesirable recombinations less likely and to screen for them after the viral particles are produced. But somehow the producers still let the replicating virus slip through. Sounds like sloppy work to me.

Comment by GeneSmith on Estimating COVID cases & deaths in India over the coming months · 2021-04-27T06:18:09.154Z · LW · GW

Good news: the Biden administration announced today that they will be sending our entire Astra Zeneca stockpile of up to 60 million doses with other countries. This is more or less the outcome I wanted. Apparently the defense production act forces companies producing vaccine supplies and vaccines in the US to prioritize domestic markets, which apparently means literal banning of exports. Seems like there are definite downsides to invoking that act.

The only question now is speed and distribution priority. In my mind we would deploy with absolute maximum blitzkrieg speed. I did some rough calculations yesterday and concluded that roughly 50 people die for every day we delay the shipment of 1 million doses. With 20 million doses ready right now that's 1000 preventable deaths per day we could save just by increasing speed.

Knowing how the government tends to operate, this will probably take much longer. And we might end up prioritizing lower risk countries like Canada and Mexico which would in my view be a sub-optimal use of excess supply.

But we are at least beginning to take the right steps. I am hopeful for that reason.

Comment by GeneSmith on [deleted post] 2021-04-24T21:29:19.427Z

Does anyone have a good method to estimate the number of COVID cases India is likely to experience in the next couple of months? I realize this is a hard problem but any method I can use to put bounds on how good or how bad it could be would be helpful.

Comment by GeneSmith on Covid 4/22: Crisis in India · 2021-04-22T21:19:39.838Z · LW · GW

I think what Zvi is saying is that we could have helped but won't be able to do much because we made poor choices. Those poor choices are our responsibility.

Comment by GeneSmith on Covid 4/22: Crisis in India · 2021-04-22T21:17:10.975Z · LW · GW

Sometimes I read these posts and feel like I am standing on an island of sanity among a sea of insane people. That 88% support of Johnson & Johnson vaccine pause just seems totally nuts.

If you're looking for something useful to do, call your senators and congresspeople and ask them to send our Astra Zeneca doses to hard hit parts of India.

Comment by GeneSmith on The Case for Human Genetic Engineering · 2021-04-17T03:49:16.382Z · LW · GW

8 months later...

I finally got around to writing about this subject again in a serious way

Comment by GeneSmith on Covid 4/15: Are We Seriously Doing This Again · 2021-04-16T04:19:10.882Z · LW · GW

I think Garret Jones's tweet is potentially misleading. One of the people in the comments mentioned that BioNTech increased their vaccine deliveries by 2.5x coinciding with the approval to vaccine at doctor's offices.. I tried to independently verify this but wasn't able to find an article. However I checked the profile of the guy who tweeted it and he seems to pretty regularly publish updates about the vaccine reserves in Germany, so my guess is he's credible.

Perhaps Germany allowed general practitioners to administer the vaccine precisely because they finally had too many vaccines to be used by the specialty clinics?

Comment by GeneSmith on What if AGI is near? · 2021-04-14T01:47:11.249Z · LW · GW

I honestly don't know and thinking about this fills me with despair. Every good solution requires time and that seems to be the main thing we're short of.

Has anyone done serious research into what it would take to slow down progress in the field of AI? Could we just ban hardware improvements for a couple of decades and place a worldwide cap on total compute? I realize this would be incredibly unpopular and would require a majority of the world's population to understand how dangerous powerful AI will be. But one would think FLI or someone else would have started doing research into this area.

Comment by GeneSmith on A Brief Review of Current and Near-Future Methods of Genetic Engineering · 2021-04-13T21:55:08.146Z · LW · GW

Yes, thanks

Comment by GeneSmith on A Brief Review of Current and Near-Future Methods of Genetic Engineering · 2021-04-13T21:36:56.632Z · LW · GW

This is a really good question. I'm not sure I have a satisfying answer to this other than to say that awareness of the dangers of both nuclear weapons and computers has been disproportionately high among extremely smart people. John Von Neumann literally woke up from a dream in 1945 and dictated to his wife the outcome of both the Manhattan Project and the more general project of computation.

One night in early 1945, just back from Los Alamos, vN woke in a state of alarm in the middle of the night and told his wife Klari:

“… we are creating … a monster whose influence is going to change history … this is only the beginning! The energy source which is now being made available will make scientists the most hated and most wanted citizens in any country.

The world could be conquered, but this nation of puritans will not grab its chance; we will be able to go into space way beyond the moon if only people could keep pace with what they create …”

He then predicted the future indispensable role of automation, becoming so agitated that he had to be put to sleep by a strong drink and sleeping pills.

In his obituary for John von Neumann, Ulam recalled a conversation with von Neumann about the “ever accelerating progress of technology and changes in the mode of human life, which gives the appearance of approaching some essential singularity in the history of the race beyond which human affairs, as we know them, could not continue.”

Or Alan Turing around the same time:

“It seems probable that once the machine thinking method had started, it would not take long to outstrip our feeble powers… They would be able to converse with each other to sharpen their wits. At some stage therefore, we should have to expect the machines to take control.”

Another one from him:

“Let us return for a moment to Lady Lovelace’s objection, which stated that the machine can only do what we tell it to do. One could say that a man can "inject" an idea into the machine, and that it will respond to a certain extent and then drop into quiescence, like a piano string struck by a hammer. Another simile would be an atomic pile of less than critical size: an injected idea is to correspond to a neutron entering the pile from without. Each such neutron will cause a certain disturbance which eventually dies away. If, however, the size of the pile is sufficiently increased, the disturbance caused by such an incoming neutron will very likely go on and on increasing until the whole pile is destroyed. Is there a corresponding phenomenon for minds, and is there one for machines? There does seem to be one for the human mind. The majority of them seem to be "sub critical," i.e. to correspond in this analogy to piles of sub-critical size. An idea presented to such a mind will on average give rise to less than one idea in reply. A smallish proportion are supercritical. An idea presented to such a mind may give rise to a whole "theory" consisting of secondary, tertiary and more remote ideas. Animals’ minds seem to be very definitely sub-critical. Adhering to this analogy we ask, "Can a machine be made to be super-critical?”

Granted, these are just anecdotes. And let it be noted that Von Neumann and Turing both went on to make significant progress in their respective fields despite these concerns. My current theory is that yes, they are more likely to both recognize the danger of AI and do something about it. But that could be wrong. I will have to think more about this.

Comment by GeneSmith on A Brief Review of Current and Near-Future Methods of Genetic Engineering · 2021-04-13T20:26:43.535Z · LW · GW

I'm not sure about the exact threshold. If the odds were below 10% I think that would be enough for me to switch to AI.

There is one other way in which I think a career in genetics could translate into a career in existential risk mitigation: through reducing the likelihood of engineered pandemics. One of the key technologies that holds incredible potential for good and for harm is genome synthesis. Given the recent rates of cost decline, I worry that someone might be able to re-create super smallpox or something before we even get to TAI. A career in genetics would put me closer to that technology, so maybe I could help design systems to prevent that particular type of disaster.

Comment by GeneSmith on A Brief Review of Current and Near-Future Methods of Genetic Engineering · 2021-04-13T20:20:40.024Z · LW · GW

Thanks for the detail about microarrays.

Do you have any sense as to how much it costs to sequence a whole human genome right now? I estimated about $300, but that was based on essentially one vendor.

Comment by GeneSmith on A Brief Review of Current and Near-Future Methods of Genetic Engineering · 2021-04-13T20:16:38.267Z · LW · GW

There are at least two companies in the US alone already doing pre-implantation screening for polygenic disease risk right now, and one of them is offering screening for unusually low IQ already. It's not that big of a stretch to imagine that parents will want to actively screen for IQ or other important traits in the next decade.

There are no legal barriers to embryo selection for intelligence. There may be some put up at some point in the future (which is a source of worry for me), but the current barriers are technological, not legal.

There was a survey done in Singapore and 87% of parents said they would be willing to intervene genetically to make their children smarter if the option was available. Attitudes in Korea are similar. If worse comes to worse I'll just work for a company or in a lab somewhere that hasn't banned it.

Comment by GeneSmith on A Brief Review of Current and Near-Future Methods of Genetic Engineering · 2021-04-13T20:07:40.586Z · LW · GW

I think if the odds were below 10% I would probably switch. Other than faster-than-expected progress in AI, the biggest thing I'm worried about is iterated embryo selection taking too long. That seems like the only technology capable of creating truly superlative humans capable of making a significant impact before TAI is created.

Comment by GeneSmith on People Will Listen · 2021-04-13T03:37:25.559Z · LW · GW

I think my dislike of Bitcoin and the poor arguments I've heard on behalf of its value has clouded my judgement on this topic. I still don't understand the actual value argument for Bitcoin. It doesn't work as money both due to its volatility and the transaction rate limit. It's basically digital gold.

But why is that valuable? I don't want to hold gold unless the financial system is collapsing and inflation is spiraling out of control or something like that.

My current theory for what happened is that everyone bought into this delusion about the value of bitcoin, but that unlike other bubbles it didn't burst because Bitcoin has a limited supply and there is literally nothing to anchor its value. So there's no point where investors give up and sell because there is literally no point at which it's overpriced.

Am I still missing something? I just have no idea how to apply lessons from Bitcoin to anything else. What lesson is there to draw from this other than "buy into things with fixed supplies and no intrinsic worth that are going up in price"?

Another of my biases that caused me to miss the train was my belief that Bitcoin is fundamentally bad for the world. The only way the price goes up is if everyone buys into the delusion that it has value. And once you're bought in you have the incentive to pressure other people to buy into the same delusion. And what is the end-result? The only thing Bitcoin ever facilitated was Silk Road and other black market exchanges. And even for those use-cases it has been superseded by other crypto designed to actually function as money.

The end-result of Bitcoin getting more valuable is that a ton of talented people spend time building exchanges and building mining rigs and thinking about Bitcoin and consuming a shit ton of power to mine more Bitcoin. On net it's just a huge loss for world productivity.

Furtheremore, everyone I talked to or whose work I read on this subject made these really poor-sounding arguments as to why I should buy bitcoin that just set off all my epistemic alarm bells. I disliked these arguments so much that it made me start to actively dislike the idea of cryptocurrency.

What is your advice now? Do I just buy in anyway and give the proceeds (if there any) to some effective charity? It seems like the only limit on the price of Bitcoin is the liquidity of the entire world economy. How do I think about when to sell?

Comment by GeneSmith on A Brief Review of Current and Near-Future Methods of Genetic Engineering · 2021-04-13T02:37:51.833Z · LW · GW

I am saying an army of people only as good as me - top quintile - can and will create TAI decades before genetic engineering will matter.

Yes, this is a concern for the utility of this approach. If TAI is created before 2050, none of this work will matter much because none of the unusually intelligent people we've been able to create will have had time to make meaningful contributions to the field of AI. In that sense, research in this field is a gamble that only starts paying off if AI takes until at least 2050. Genetic engineering will have a progressively larger impact the longer it takes to develop TAI.

This timing concern was actually one of my chief worries about going into genetics as a career. I won't be able to switch careers and start having a large impact on AI if research in that field progresses faster than expected. So it's possible there will come a point in the future where I am stuck on the sidelines in the final years before TAI is created, watching 30 years of work come to nothing.

But I think 50% odds of having a huge impact are worth taking, and I think the biological route to superintelligence is severely neglected right now. Who is actually working on genetic engineering right now? I literally know one person who has both expressed an interest in genetic engineering for intelligence and has real scientific expertise in the field: Steven Hsu. And sadly he seems to have turned away from his earlier goals after his public humiliation at the hands of misguided student activists at Michigan State University.

I am hopeful that as pre-implantation genetic screening via IVF becomes a more normalized part of the pregnancy process, attitudes will change. It's pretty silly that so many people think enhancing our children's potential via physical exercise and healthy food is acceptable but that genetic intervention should be off-limits.

Comment by GeneSmith on A Brief Review of Current and Near-Future Methods of Genetic Engineering · 2021-04-12T20:45:15.590Z · LW · GW

Nature can only do so much to improve our intelligence, being stuck with living cells as computational circuits in a finite brain volume, with finite energy supply.

This is true, and it's one of the main reasons I think AI will eventually overtake us no matter how much genetic engineering we do. But like I said in the post, there is enough additive variance in existing gene pool to create humans with predicted IQs of over a thousand. We are far from the actual physical limits on brain size, neural conduction speed, neuron size, and many others.

It isn't clear how meaningful the intelligence differences really are in terms of utility on actual tasks.

How many company founders emphasize that "attracting talent" is the most difficult part of making their company successful? How many Nobel winners have IQs several standard deviations above the mean? It is very clear that intelligence has a huge impact on performance on a wide variety of tasks. If you want more examples of this I suggest you read one of Gwern's many essays on the topic

Thing is, the road to get there isn't going to involve a whole lot of someone solving math problems in their head as they pound a keyboard through the night writing reams of custom code.

This isn't an accurate characterization of the type of task that unusually intelligent people excel at. I agree with you that raw intelligence isn't the only thing need to solve this problem. Creating proper organizational incentive structures will matter a lot too, as will clear-headed thinking about the problem. But intelligent people are actually very good at exactly those types of things. Look at how many of the top scientists in the country played a significant role in the Manhattan Project.

A single person is not going to meaningfully solve this problem by themselves.

Which is why I didn't suggest we create one super-genius and call it a day. I don't want access to pre-implantation genetic screening or iterated embryo selection to be available to only the privileged elite. It needs to be broadly available to any parent that wants it. And the benefits can and will go to many fields, not just AI.

Just as a general policy, anything current flesh and blood humans with are having trouble with, that smarter humans have less trouble with, current humans can probably write a piece of software that is better than the efforts of any humans. With today's techniques.

This is just obviously wrong. TAI cannot be created by current. humans with today's techniques. It's likely going to take decades to create the technology to do so, and it's going to take some of the smartest researchers in the world to do it.

Comment by GeneSmith on A Brief Review of Current and Near-Future Methods of Genetic Engineering · 2021-04-12T20:13:08.030Z · LW · GW

Thank you for writing such a thoughtful comment. I have to confess, I probably gave this post the wrong title. For the longest time I simply titled it "Genetic Engineering Part 3" as I wasn't sure what to call it when I first started. I then accidentally left that title in when I first published it and hastily changed it to its current title even though that doesn't quite fit either.

You're correct, of course, that I did not comprehensively review all possible techniques for genetic engineering. Most notably among these is whole-genome synthesis, with which we could theoretically create an entire genome with any base pairs we wanted. In my research I estimated that synthesizing a whole human genome from scratch would cost about $200 million. So we still have a few orders of magnitude to go before whole genome sequencing becomes a viable method for creating superhumans.

I also have some serious concerns about other much more dangerous uses of whole-genome synthesis. If the technology becomes cheap enough and widely enough available it could become an incredibly dangerous weapon for engineering biological weapons. This is such a big worry that I think pursuing human genetic modification via genome synthesis might actually end up INCREASING the risk of human extinction rather than decreasing it.

Regarding the first, one may doubt GWAS on the grounds of reliability (false positives) and power (not enough variance accounted for)

If there were false positives in a GWAS then the model would have poor performance on the test set. Of course there ARE issues with GWAS predictive power when you try to generalize to other populations with a high ancestral distance from your training set. For example I remember reading about a GWAS for general cognitive ability that predicted about 10% of variance in Europeans, but only 2.5% for people of African descent. However that isn't an issue of false positives. It's an issue of different genes having different frequencies in each population. We could create a good predictor for people of African descent if we had data sets that included more people from those populations.

Regarding the second, one would like to know that this process isn't creating e.g. some cumulative epigenetic artefact.

This is something I didn't even think about when writing the paper, so thanks for bringing it up. I would think that the epigenome would be preserved throughout this process, but that assumption might be wrong.

Comment by GeneSmith on A Brief Review of Current and Near-Future Methods of Genetic Engineering · 2021-04-11T18:47:25.079Z · LW · GW

This is an interesting idea. Nuclear transfer has been used in cloning before, but it is not particularly reliable. That being said, perhaps future research could improve the success rate (and more importantly the ease of doing so).

At the end of the day, the entire iterated embryo selection process is about generating a complete DNA sequence that scores better on our tests. I left out whole genome synthesis from the original post because from the brief reading I did on the topic it seemed prohibitively expensive. But that could change in the future, as the cost per base pair has been declining exponentially for some time now. The most notable recent use of whole-genome sequencing was to create the mRNA in the Pfizer-BioNTech and Moderna vaccines.

Maybe I'll write a future post about this topic.

Comment by GeneSmith on A Brief Review of Current and Near-Future Methods of Genetic Engineering · 2021-04-11T06:56:18.743Z · LW · GW

Thanks! I spent an embarrassingly long time writing it.

Your question about iterated embryo selection is an interesting one. I suspect that performing this procedure multiple times without adding genetic material WOULD result in higher defect rates, though I'm not positive. If one is already selecting against the types of negative traits that inbreeding increases, would we still expect to see higher rates of health conditions even after selection, or would inbreeding simply decrease our the average quality of embryos due to a higher percent having health issues?

Part of my problem is not understanding exactly why inbreeding is bad. I'm familiar with the standard answer that "inbreeding increases the chance that offspring inherit recessive diseases", but why exactly is that? One answer is Muller's ratchet, which says that environmental damage leads to a constant increase in deleterious mutations to the germline, and the only feasible way to decrease mutational load is through sex. Under this model, sex is kind of like a simple error correction mechanism: a single mutation is unlikely to occur in both organisms, so given the production of enough offspring, one of them is likely to have reduced mutational load.

So under this model, inbreeding is bad because it correlates genetic mutations. If two organisms share a larger portion of their DNA, they are likely to inherit many of the same mutations, preventing their descendants from shedding mutational load through lucky recombination.

But if that analysis is correct and inability to shed mutational load is the main reason for increased health problems among inbred offspring, then perhaps it wouldn't be such a big deal for iterated embryo selection after all, since there is very little time between generations for the embryos to accrue deleterious mutations.

In the end though, one almost certainly would want to introduce genetic material from other parents simply because it would allow for additional valuable genetic material from which to select.

I wonder how much of it can be avoided by both optimizing for a positive trait X while simultaneously optimizing against the traits of people with negative life outcomes.

You've put your finger right on one of the most important questions for the future of this field. If we simply add enough important traits to our linear equation, can we push as far as we want into the tails of these trait distributions? I think we should be able to get at least 3 or 4 standard deviations from the mean of most traits with this method. Possibly much further. But how far can we actually go before we end up optimizing against some important trait that we simply didn't include in the equation because we either didn't think about it or didn't realize it was important?

This is one of the reasons I've tried to keep up with AI safety research. They are far far ahead of biologists in understanding how to properly frame and begin to answer these types of questions. If you squint hard enough, trait enhancement with iterated embryo selection starts to look a lot like iterated amplification and distillation, and the question of how far we should push into the tails of the distributions becomes a question of how high we can safely set the learning rate.

One of the topics I'd like to write about in the future is how to apply ideas from AI safety to the field of genetic enhancement. It's actually quite hard to do this rigorously because many of the assumptions that underlie techniques to align AI don't really work with genetic engineering. With humans, you have an insanely long lag before you can validate that the changes actually worked, and the cost of getting things wrong is huge. With software models, you can explore actions that your current policy says aren't optimal. Models can be updated very quickly. But with humans the cost of exploration is extremely high.

And unless everyone in this community is wildly off about their timelines to transformative AI, we may only get one genetically engineered generation before biological organisms like ourselves cease to be relevant to the larger strategic picture.

Comment by GeneSmith on A Brief Review of Current and Near-Future Methods of Genetic Engineering · 2021-04-11T05:58:37.871Z · LW · GW

This is an interesting idea. I suspect that this type of selection would asymptotically approach twice the per-generation gain of simple embryo selection. So useful but not really transformative.

Comment by GeneSmith on Jean Monnet: The Guerilla Bureaucrat · 2021-04-03T19:16:31.720Z · LW · GW

I was mainly referring to the long negotiation-induced delay in the EU's contract with Astra Zeneca. They inked their purchasing agreement a full 3 months after the UK, which is one of the primary reasons they have such a low vaccination rate in comparison.

One might say that this simply meant the UK got more vaccines, but that's not true. The long negotiation period actually delayed the beginning of production

They've also failed to delay the second dose, which would have allowed more people to get vaccinated, further increasing the death toll.

I don't know the degree to which this is simply due to the wrong people being in charge as opposed to poorly designed incentives or the structure of the EU itself. But as a whole, the EU has made many extremely costly mistakes.

Comment by GeneSmith on Jean Monnet: The Guerilla Bureaucrat · 2021-03-31T05:53:16.470Z · LW · GW

Great post. I wonder if a person like Monet could be as effective today, or whether he would become the target of conspiracies because he was so well connected and information about people like that can spread so much more rapidly.

I find myself torn between admiration for the European project and all the solving of coordination problems it entails and exasperation with the endless bureaucracy and boneheaded thinking that seems endemic to such institutions. There's no better example than the recent vaccine debacle, which has probably killed tens of thousands of Europeans due to poor government choices.

Comment by GeneSmith on Why Selective Breeding is a Bad Way to do Genetic Engineering · 2021-03-06T19:05:17.754Z · LW · GW

I didn't see the need to clarify because I've never heard anyone use the term "selective breeding" in the context of individual mate selection. Of course you're correct that individual choices about whom to reproduce with affect trait selection, but I don't think many people find the ability to choose one's own spouse problematic.

Comment by GeneSmith on Birds, Brains, Planes, and AI: Against Appeals to the Complexity/Mysteriousness/Efficiency of the Brain · 2021-03-06T18:47:52.171Z · LW · GW

What's your best estimate for the amount of time it will take us to get to TAI?

Comment by GeneSmith on Why Selective Breeding is a Bad Way to do Genetic Engineering · 2021-03-06T02:53:21.316Z · LW · GW

It may be safe from an individual perspective, but if you always pick the more common allele, you are converging towards the modal genome, which would be a world where everyone is a clone of everyone else.

Genetic diversity is valuable both as a hedge against disease and because it lends itself to specialization, which is an important part of the modern economy.