What will 2040 probably look like assuming no singularity?

post by Daniel Kokotajlo (daniel-kokotajlo) · 2021-05-16T22:10:38.542Z · LW · GW · 15 comments

This is a question post.

Contents

  Answers
    25 ChristianKl
    21 jacopo
    13 Flaglandbase
    12 johnswentworth
    9 Rochambeau
    9 SDM
    7 rayom
    5 NoUsernameSelected
    4 frontier64
    4 MaxRa
    3 Mary Chernyshenko
    3 Taran
    2 Sam Clarke
    1 Pattern
    1 Dirichlet-to-Neumann
None
15 comments

I'm looking for a list such that for each entry on the list we can say "Yep, probably that'll happen by 2040, even conditional on no super-powerful AGI / intelligence explosion / etc." Contrarian opinions are welcome but I'm especially interested in stuff that would be fairly uncontroversial to experts and/or follows from straightforward trend extrapolation. I'm trying to get a sense of what a "business as usual, you'd be a fool not to plan for this" future looks like. ("Plan for" does not mean "count on.")

Here is my tentative list. Please object in the comments if you think anything here probably won't happen by 2040, I'd love to discuss and improve my understanding.

  1. Energy is 10x cheaper. [EDIT: at least for training and running giant neural nets, I'm less confident about energy for e.g. powering houses but I still think probably yes.] This is because the cost of solar energy has continued on its multi-decade trend, though it is starting to slow down a bit. Energy storage has advanced as well, smoothing out the bumps. [EDIT: Now I think fusion power will also be contributing, probably. Though it may not be competitive with solar, idk.]
  2. Compute (of the sort relevant to training neural nets) is 2 OOMs cheaper. Energy is the limiting factor.
  3. Models 5 OOMs more compute-costly than GPT-3 have been trained; these models are about human brain-sized and also have somewhat better architecture than GPT-3 but nothing radically better. They have much higher-quality data to train on. Overall they are about as much of an improvement over GPT-3 as GPT-3 was over GPT-1.
  4. There's been 20 years of "Prompt programming" now, and so loads of apps have been built using it and lots of kinks have been worked out. Any thoughts on what sorts of apps would be up and running by 2040 using the latest models?
  5. Models merely the size of GPT-3 are now cheap enough to run for free. And they are qualitatively better too, because (a) they were trained to completion rather than with early stopping, (b) they were trained on higher-quality data, (c) various other optimized architectures and whatnot were employed, (d) they were then fine-tuned on loads of data for whatever task is at hand, and (e) decades of prompt programming and prompt-SGD has resulted in excellent prompts as well that fully utilize the model's knowledge, (f) they even have custom chips specialized to run specific models.
  6. The biggest models--3 OOMs bigger than GPT-3--are still only a bit more expensive at inference time than GPT-3 was in 2021. Energy is the main cost. Vast solar panel farms power huge datacenters on which these models live, performing computations to serve requests from all around the world during the day when energy is cheapest.
  7. Some examples of products and services:
    1. Basically all the apps that people talk about maybe doing with GPT-3 in 2021 have been successfully implemented by now, and work as well as anyone in 2021 hoped. It just took two decades to accomplish (and bigger models!) instead of two years and GPT-3.
    2. There are now very popular chatbots, that are in most ways more engaging and fun to talk to than the average human. There are many of these bots catering to different audiences, and they can be fine-tuned to particular customers. A billion people talk to them daily.
    3. There are specialized chatbots for various jobs, e.g. customer support.
    4. There are now excellent predictive tools that can read data about a person, especially text authored by that person, and then make predictions like "probability that they will buy product X" and "probability that they will vote Republican"
  8. Cars are all BEVs, with comparable range to 2020s gas cars but much lower operating costs due to energy being practically free and maintenance being very easy for BEVs.
  9. Cars are finally self-driving, with cheap LIDAR sensors and bigger brains trained on way more data along with many layers of hard-coded tweaks to maximize safety. (Also various regulations that make it easier for them, e.g. by starting with restrictions on what sorts of areas they can operate in, and using big pre-trained models in server farms to make important judgment calls for individual cars and monitor the roads more generally via cameras to look out for anomalies). (I'm not so sure about this one, part of me wonders if self-driving cars just won't happen on business-as-usual).
  10. Starlink internet is fast, reliable, cheap, and covers the entire globe.
  11. 3D printing is much better and cheaper now. Most cities have at least one "Additive Factory" that can churn out high-quality metal or plastic products in a few hours and deliver them to your door, some assembly required. (They fill up downtime by working on bigger orders to ship to various factories that use 3D-printed components, which is most factories at this point since there are some components that are best made that way)
  12. Drone delivery? I feel confused about this, shouldn't it have happened already? What is the bottleneck? This article makes it seem like the bottleneck is FAA regulation. [EDIT: I talked to an amazon drone delivery guy recently. He said 95% of the job is trying to figure out how to improve safety to meet regulatory requirements. He said they have trouble using neural nets for vision because they aren't interpretable so you can't prove anything about their safety properties.]
  13. World GDP is a bit less than twice what it is now. Poverty is lower but not eliminated.
  14. Boring company? Neuralink? I'm not sure what to think of them. I guess I'll ignore them for now, though I do feel like probably at least one of them will be a big deal...
  15. Starship or something similar is operational and working more or less according to specs promised in 2020. Maybe point-to-point transport on Earth didn't work out, maybe the cost per kilo to LEO never got quite as low as $15, but still it's gotta be pretty low--maybe $50? (For comparison, it's currently about $1000 and five years ago was $5000) Thus, Elon probably gets his colony on Mars after all, and NASA gets their moon base, and there's probably a big space station too and maybe some asteroid mining operations?
  16. Video games now employ deep neural nets in a variety of ways. Language model chatbots give NPC's personality; RL-trained agents make bots challenging and complex; and perhaps most of all, vision models process the wireframe video game worlds into photorealistic graphics. Perhaps you need to buy specialized AI chips to enjoy these things, like people buy specialized graphics cards today.
  17. Virtual reality is now commonplace; most people have one or two headsets just like they have phones, laptops, etc. today. The headsets are low weight and high-definition compared to 2021's. Many people use them for work, and many more people use them for games and socializing.
  18. The military technology outlined here [LW · GW] exists, though it hasn't been used in a major war because there hasn't been a major war, and as a result the actual composition of most major militaries still looks pretty traditional (tanks, aircraft carriers, etc.) It's been used in various proxy wars and civil wars though, and it's becoming increasingly apparent that the old tech is obsolete.
  19. Household robots. Today Spot Mini costs $74,500. In 2040 you'll be able to buy a robot that can load and unload a dishwasher, go up and down stairs, open and close doors, and do various other similar tasks, for less than $50,000.  (Maybe as low as $7,500?) That's not to say that many people will buy such robots; they might be still expensive enough and finicky enough to be mostly toys for rich people.

My list is focused on technology because that's what I happened to think about a bunch, but I'd be very interested to hear other predictions (e.g. geopolitical and cultural) as well.

Answers

answer by ChristianKl · 2021-05-18T14:23:31.117Z · LW(p) · GW(p)
  1. Energy cost in summer and energy cost in winter will strongly diverge.
  2. One key usage of cheap summer energy will be hydrogen and methane production
  3. Some planes will be Hydrogen-based
  4. Hydrogen will be used more frequently for home heating than gas.
  5. Permanent base on moon and Mars
  6. Vacations on space stations will cost less than 20k (a good chance that it's less than 10k)
  7. Transporting materials to orbit will cost less then 10$/kg
  8. A majority vehicle journeys will be in rented cars. A majority of cars on the road will be electric.
  9. Common services such as hairdressing will be provided in moving cars, so that it's possible to book your hairdressing during your commute in at least some jurisdictions.
  10. Delivery costs go down by an order of magnitude as robots can replace humans
  11. Cheaper delivery costs mean that more people will let all their clothes by washed by a dry-cleaner that picks up their clothing and then brings clean clothing back. A lot of the individual labor will be automated which will in turn bring down prices as well.
  12. 65% of the world population will live in cities
  13. Insurance payouts for natural disasters will double compared to present levels
  14. A majority of households with a net worth of >100k will have air filters and air quality sensors.
  15. Routing usage of next-generation sequencing for virus and bacteria infections that sequence everything in the blood.
  16. Protein folding is solved to the point where it's easy to design new proteins. This will be used both in medicine and in other fields that need specialized materials.
  17. Phage therapy will be standard treatment for at least one infection that's currently not well treated with antibiotics (chronic Lyme/Periodontitis/MRSA).
  18. There will be multivar tests for combinations of using existing drugs as antiaging drugs.
  19. >10% of meat sold in the West will be lab grown.
  20. The money that flows through prediction markets will be at least 10x of what it is today.
  21. All police in the Western world will wear body cams.
  22. Cancer vaccines will be part of regular cancer treatment.
  23. Hydra-style darknet markets that combine product quality assurance through human testing and a delivery network will not only be operational in Russia but all Western countries as well
  24. When dark-market organizations provide independent quality control while the FDA just trust manufactures claims, there will be pressure on the FDA to match the quality processes of Hydra style markets that result in new legislation
  25. Multiple countries will pass laws to regulate satellite surveillance within their territory as it's economical to have real time satellite views of everything on the globe. While it's unclear from the present point how it will be regulated it will be a big deal for some people.
  26. Chinese politicians will switch to finding the environment important. As a result there will be a global treaty to remove plastic trash from the oceans and reduce mercury concentrations in the ocean. Fishing will be banned in parts of the ocean.
  27. Kinmen will be under Chinese control
  28. Neurolink or a comparable company will produce a product that can be brought without medical diagnosis for an illness.
  29. Copper and iron will have a lower inflation adjusted price compared to today
  30. As robots get used more significantly less food is grown as monocrops the way it's grown today and agriculture that combines multiple crops which is very labor intensive today will be cheaper given automation.
comment by Daniel Kokotajlo (daniel-kokotajlo) · 2021-05-19T05:56:15.778Z · LW(p) · GW(p)

Nice list! I'm skeptical of 2, 3, 4, 7, 9, 10, 21, 22, 25, 28. I'd be interested to hear more about them and also about 29 and the hydra markets stuff. Also, clearly protein folding will be solved to a significant extent, but how solved? Enough for molecular nanotech? Can you say more about what you have in mind?

Replies from: ChristianKl
comment by ChristianKl · 2021-05-19T09:55:11.711Z · LW(p) · GW(p)

2/3/4: Look at the UK hydrogen strategy under Boris Johnson (a good chance that it was partly written by Cummings people). Currently, we have more then a week in Germany with negative energy prices where people are paid to use energy (if the get the energy directly from the energy market). The more solar you have the more days you will have where a lot of energy gets produced that has no clear use. On the other hand you need to be able to produce energy on days where the sun does't shine. Batteries can be used to shift energy usage over a 24 hour time period but they are not cost effective for moving energy production from summer to winter. 

If you want to run fully on solar/wind with current technology there doesn't seem to be a way around hydrogen/methan production on the days where you have too much energy and use those partly when you need them in the winter. 

Hydrogen for heating is part of the UK hydrogen strategy as proposed at the moment. Hydrogen is very light which makes it a good fuel for planes. 

7: 10$/kg is a number that Elon set as a goal for the Starship (https://wccftech.com/elon-musk-starship-launch-cost-reiterate/) I expect that by 2040 there will be successor technology to Starship that's more efficient so even if Starship doesn't reach that goal the successor technology will. 

9: Given what we know about how transportation works it's likely that driverless cars will result in people spending more time in cars. Making good use of the time will be important. 

Rented cars inturn mean that you can rent different cars that are specialized for different usecases.

Hairdressers that provide that service will be a premium product that's brought by business people who are very busy and spent a lot of time in their cars. 

10: Reduced electricity costs and driverless cars mean that the ride is cheaper and you don't need to pay a delivery person.

21: I think I need to qualify, I mean all uniformed police. There might be undercover police where having camera's wouldn't work because it allows detection of the police. Camera's get cheaper by the year and there's a lot of pressure to take action regarding policy brutality. The power of technology lobbyists who want to convince politicians to use tech solution rises over time. 

22: Cancer vaccines that are created based on the mutations that a particular cancer has increase the immune response against the cancer cells. While it might not be enough on it's own to kill every cancer it's combinable with a lot of other cancer treatments. If you for example operate out a cancer you want the immune system to go after any remaining small metastating cells and cancer vaccines are a tool to encourage that. 

If you don't give a patient drugs that shut down the immune system, giving them cancer vaccine in addition to other treatments will have better effects. 

This is different from cancer vaccines that go after proteins that are normally only expressed in fetal development which largely flopped in clinical trials. 

25: It's basically saying that within 20 years the needed technology for EarthNow will be deployed. Being able to track all the cars on the street do things like being able to tell how much millionaires who park their cars in their garage engage in speeding has huge privacy implications and EU lawmarkers will want to regulate it.

29: This might be the one I'm least certain about, but I expect advances in mining that might include asteroid mining to bring down prices. 

hydra markets stuff

Hydra is a Russian darknet marketplace that superior in technology to any other darknet market. Hydra accounts for 75% of total darknet marketplace revenue despite currently only targeting Russian speakers.

Hydra is also noteworthy for a hitman being hired to kill a police official. 

They have a system where there are delievery people who hide a drop at a specific location and then the customer gets told about where the drop he ordered is hidden which means it's possible to buy on Hydra without revealing your home address. 

Hydra charges the merchants fees that among others are used to do quality testing where the drugs are taking by paid human volunteers and their effects analysed. 

Hydra had an ICO with the goal of expanding West which they might or might not follow through on. In any case it's a clearly superior darknet model and if Hydra itself doesn't expand West I would expect another entity to create a similar market in the West. 

Also, clearly protein folding will be solved to a significant extent, but how solved? Enough for molecular nanotech? Can you say more about what you have in mind?

If you have a chemical reaction that's inefficient there's a good chance that you can create a protein that catalyses the reaction much more effectively. 

Replies from: ChristianKl
comment by ChristianKl · 2021-05-19T11:48:10.703Z · LW(p) · GW(p)

More thoughts on protein folding:

If you can design proteins for specific reactions, problems like mining uranium from seawater suddenly become a lot easier. You can design a membrane protein that's able to trap uranium atoms and then bring them into a cell. 

You might even have different bacteria for mining different minerals from the water. You leave the bacteria for some time in the water and they take in a bunch of minerals. Then you let the water out through nanopores that don't let the bacteria through. Then you use a centrifuge to sort the bacteria by the materials they contain, and put each one in their own tanks. Then you heat the bacteria up with destroys the cell walls and makes the materials you are after fluid. You centrifuge again and have your materials. 

I expect bacteria to be used that switch from the 3-letter genetic code to a 5-letter genetic code where a single letter mutation doesn't lead to another amino-acid being used to have a bacteria for tasks like the above that doesn't just mutate away. 

There will be a bunch of things that can be done that are hard to think about now because it's very different technology then we are used to.

If you have concrete usecases for molecular nanotech I could tell you whether I think that they are doable with it.

answer by jacopo · 2021-05-17T10:42:08.146Z · LW(p) · GW(p)

I think 10x decrease in energy prices is too much. My reasons are:

  1. There are some constrains on solar/wind which are currently not binding, but will be by the time we have converted most energy production to green energy. The main ones are metals (see e.g. https://www.coppolacomment.com/2021/03/from-carbon-to-metals-renewable-energy.html) and land use (in India, China, Europe, Japan and a few other Asian countries especially as population density is high, but that's most of the world population anyway). This of course does not consider the possibility of major technological breakthroughs in organic solar and energy conversion/transport, which may happen but are not guaranteed so I think they are out of the scope of your exercise.
  2. As the cost of energy lowers, we will consume more. In poor countries especially, plus you mentioned increased consumption by supercomputers and AI. This will partially balance the cost of production falling, so my (uneducated) guess would be that a 2x-3x decrease in prices is a more reasonable expectation. An analysis by an expert could convince me otherwise.

Like rayom I also noticed you did not mention anything about biology and medicine. I think there will be some advances from that side. A malaria vaccine seems probable by 2040 (maybe ~80%?) and would be a big thing for large parts of the world. Also some improvement in cancer therapy seem to have relatively high probability (nothing even remotely like "cure all cancer", to be clear). We might get some improvement for Alzheimer, dementia or other age-related illnesses, but my "business as usual" expectation is that only moderate advancements will be widely deployed by 2040. Nevertheless they might be sufficient to improve significantly the quality of life of elderly people in rich countries.

comment by johnswentworth · 2021-05-18T16:01:43.783Z · LW(p) · GW(p)

Extending on point 2: if we want to talk about a price drop, then we need to think about relative elasticity of supply vs demand - i.e. how sensitive is demand to price, and how sensitive is supply to price. Just thinking about the supply side is not enough: it could be that price drops a lot, but then demand just shoots up until some new supply constraint becomes binding and price goes back up.

(Also, I would be surprised if supercomputers and AI are actually the energy consumers which matter most for pricing. Air conditioning in South America, Africa, India, and Indonesia seems likely to be a much bigger factor, just off the top of my head, and there's probably other really big use-cases that I'm not thinking of right at the moment.)

comment by MondSemmel · 2021-05-18T14:28:50.889Z · LW(p) · GW(p)

I understand that Malaria resists attempts at vaccination, but regarding your 80% prediction by 2040, did you see the news that a Malaria vaccine candidate did reach 77% effectiveness in phase II trials just last month? Quote: "It is the first vaccine that meets the World Health Organization's goal of a malaria vaccine with at least 75% efficacy."

Replies from: jacopo
comment by jacopo · 2021-05-18T15:15:48.560Z · LW(p) · GW(p)

No I missed it, that's great! I was only aware of phase I. It should be revised way up then.

comment by ChristianKl · 2021-05-18T11:56:44.254Z · LW(p) · GW(p)

People argued for metal prices being a problem for a long time and those predictions usually failed to come true. 

Replies from: jacopo, johnswentworth, daniel-kokotajlo
comment by jacopo · 2021-05-18T13:43:08.144Z · LW(p) · GW(p)

You are right of course, and I am going by other people's analysis so I am not sure how much they are correct or wrong this time around. I do not think we will have hugely rising commodity prices making green energy unfeasible, unless there is a war (or just a trade war) blocking the supply of a key input. 

Nevertheless, the extrapolation of decreasing costs for solar and wind based on current trends will eventually hit some "hard" limit, and metals are a likely candidate. After all, as manufacturing costs for panels reduce, the fraction of cost coming from raw materials grows even at constant prices. And to get prices going down 10x, we need to supply several times more energy than now (maybe 5x?) meaning growing wind and solar by two orders of magnitude in 20 years. This could plausibly put strain on the supply of raw materials. 

Of course, if the bottleneck will turn out to be energy distribution and storage, then we could get prices going down 10x at the source (what Daniel is interested in) but not for household consumption, and only a modest increase in demand. 

comment by johnswentworth · 2021-05-18T15:55:51.336Z · LW(p) · GW(p)

+1 to this, though I think a slightly modified version of jacopo's argument is stronger: new constraints are likely to become binding in general when cost of current constraints drops by a factor of 10, though it's not always obvious which constraints will be relevant.

comment by Daniel Kokotajlo (daniel-kokotajlo) · 2021-05-18T12:45:48.544Z · LW(p) · GW(p)

Interesting -- source pls? The price of energy is one of the most important things on my list so I'm especially keen to hear more evidence for and against my projected 10x drop.

Replies from: ChristianKl, jacopo
comment by ChristianKl · 2021-05-18T14:06:15.363Z · LW(p) · GW(p)

As jacopo pointed out the Simon–Ehrlich wager is a key argument. As far as solar costs go, the cost of actual solar cells fell a lot more in the last years then installation costs. 

When it comes to solar cells for training neural nets, it's worth noting that those don't need to be stationed on earth. 

At 10$/kg or less for transporting material to space with Starship (and it's successors) it's possible that it makes more sense to have the data center in space where it gets exposure to the sun 24/7 and there's no sky that blocks sunlight.

For earth based datacenters, if energy is the limiting factor and a lot of the energy comes from solar, it's possible to do all the training for neural nets in the summer where there's plenty of solar energy and not train your models on days where solar cells and wind farms produce little energy. 

Replies from: Dirichlet-to-Neumann
comment by Dirichlet-to-Neumann · 2021-05-18T19:11:22.374Z · LW(p) · GW(p)

Is refrigeration a big part of data centers energy costs ? This would mean the best places for solar energy are also the worst places for data centers...

Replies from: daniel-kokotajlo, ChristianKl
comment by Daniel Kokotajlo (daniel-kokotajlo) · 2021-05-18T19:17:20.532Z · LW(p) · GW(p)

I've thought a bit about this, but haven't done any calculations. My guess is that it would be overall cheaper to have datacenters in sunny regions than to try to get solar panels in cold regions. Your refrigeration (and heat management more generally) electricity bill will be higher, but not that much higher, but your electricity costs will be much lower.

comment by ChristianKl · 2021-05-19T11:26:04.792Z · LW(p) · GW(p)

If refrigeration becomes a major part of the energy cost it's worth noting that there's thinking about putting data centers under water where they can be cooled more easily. At the moment that's not viable but it might be in 2040.

Replies from: Dirichlet-to-Neumann
comment by Dirichlet-to-Neumann · 2021-05-22T09:48:38.699Z · LW(p) · GW(p)

True but putting them in Northern part of the world may also be a good idea. Right now looking for example at Google's data centers map there seems to be a very small trend toward northern locations (at least in Europe), but it may just be a flux due to local financial incentives being more favorable in some countries.

Replies from: ChristianKl
comment by ChristianKl · 2021-05-22T23:31:45.126Z · LW(p) · GW(p)

Right now looking for example at Google's data centers map there seems to be a very small trend toward northern locations (at least in Europe)

Google seems to have one datacenter in Ireland, Denmark, Belgium, Finnland and two in the Netherlands.

Those seem to be countries that currently have cheap industrial electricity: https://www.statista.com/statistics/1046605/industry-electricity-prices-european-union-country/

comment by jacopo · 2021-05-18T13:46:57.121Z · LW(p) · GW(p)

I think he might be referring to the Simon–Ehrlich wager. And indeed there have been other similar claims in the past, more often proven wrong than correct.

comment by Daniel Kokotajlo (daniel-kokotajlo) · 2021-05-17T11:30:49.973Z · LW(p) · GW(p)

Thanks! I edited my thing on energy to clarify, I'm mostly interested in the price of energy for powering large neural nets, and secondarily interested in the price of energy in general in the USA, and only somewhat interested in the price of energy worldwide.

I am not convinced yet that the increased demand from AI will result in increased prices. In fact I think the opposite might happen. Solar panels are basically indefinitely scalable; there are large tracts of empty sunny land in which you can just keep adding more panels basically indefinitely. And transporting the energy to other places won't be an issue for AI because the datacenters can be built right where the solar panels are. If storing energy is a problem, just do your AI training runs during the day when energy is plentiful. So I predict that AI-related demand for energy will mostly just result in vastly increased supply, rather than increased prices -- but when supply increases, economies of scale will result, that may even drive the price lower!

The point about metals is new to me, I'll go read up on that, thanks. For some reason your link seems to be broken.

Replies from: jacopo
comment by jacopo · 2021-05-17T15:45:09.804Z · LW(p) · GW(p)

Thanks for the clarifications! I realized that maybe you are mostly interested on the tech sector in the US and AI-related development, which explains also why you didn't think of biomedical research immediately. Is this impression correct? If so, you might want to edit further the question to restrict the range of answers. 

I fixed the link, I didn't notice but it had taken the ) as part of the address.

BTW, I read your post on military tech in the meantime, it was interesting.

Replies from: daniel-kokotajlo
comment by Daniel Kokotajlo (daniel-kokotajlo) · 2021-05-17T20:37:47.640Z · LW(p) · GW(p)

It's true that that's what I'm mostly interested in, but I don't want to restrict the question to that stuff -- I asked this question so I could learn more things! Please don't hesitate to answer with biotech stuff or non-US stuff or non-tech stuff!

Thanks!

answer by Flaglandbase · 2021-05-18T00:51:55.550Z · LW(p) · GW(p)

When I look back twenty years, it seems amazing how little has changed or improved since then. Basically just the same, but some things are less slow. 

The arrival of the internet in the nineties was the only real change. The arrival of AI will be the next change, whenever that happens. 

And in twenty years the looming maw of death will be closer for most of us, like a bowling ball falling into a black hole. 

comment by [deleted] · 2021-05-19T02:27:34.042Z · LW(p) · GW(p)

I share this sentiment. Shockingly little has happened in the last 20 years, good or bad, in the grand scheme of things. Our age might become a blank spot in the memory of future people looking back at history; the time where nothing much happened.

Even this recent pandemic can't shake up the blandness of our age. Which is a good thing, of course, but still.

Replies from: None
comment by [deleted] · 2021-05-21T06:26:28.137Z · LW(p) · GW(p)

Or you are seeing the working out of the logistic function that takes shape in so many systems.

answer by johnswentworth · 2021-05-18T03:44:48.007Z · LW(p) · GW(p)
  1. Anti-aging will be in the pipeline, if not necessarily on the market yet. The main root causes of most of the core age-related diseases will be basically understood, and interventions which basically work will have been studied in the lab.
  2. Fertility will be below replacement rate globally, and increasingly far below replacement in first-world countries (most of which are already below today). Life expectancy will still be increasing, so the population will still be growing over all (even assuming anti-aging is slow), but slowly and decelerating.
  3. Conditional on anti-aging not already seeing large-scale adoption, the population will have a much higher share of elderly dependents and a lower share of working-age people to support them, pretty much everywhere. This problem already dominates the budgets of first-world governments today: it means large-and-increasing shares of GDP going to retirement/social security and healthcare for old folks (who already consume the large majority of healthcare).
  4. Conditional on anti-aging not already seeing large-scale adoption, taxes will probably go up in most first-world countries. There just isn't enough spending to cut anywhere else to keep up with growing social security/healthcare obligations, and dramatically reducing those obligations won't be politically viable with old people only becoming more politically dominant in elections over time. (In theory, dramatically opening up immigration could provide another path, but I wouldn't call that the most likely outcome.)
  5. China's per-capita GDP will catch up to current first-world standards, at which point they will not be able to keep up the growth rate of recent decades. That will probably result in some kind of political instability, since the CCP's popularity is heavily dependent on growth, and also because a richer population is a more powerful population which is just generally harder to control without its assent.
comment by johnswentworth · 2021-05-18T16:04:41.738Z · LW(p) · GW(p)

I expect people to find 1 wild. The rest are pretty straightforward extrapolations of trends, and they're the sort of trends which have historically been quite predictable.

comment by johnswentworth · 2021-05-18T23:31:01.295Z · LW(p) · GW(p)

Definitely, and the Nate Silver piece in particular is 8 years out of date. But these are long-term trends, and the predictions don't require much precision - COVID might shift some demographic numbers by 10% for a decade, but that's not enough to substantially change the predictions for 2040.

comment by johnswentworth · 2021-05-18T22:11:24.380Z · LW(p) · GW(p)

Sure. Here's a graph from wikipedia with global fertility rate projections, with global rate dropping below replacement around 2040. (Note that replacement is slightly above 2 because people sometimes die before reproducing - wikipedia gives 2.1 as a typical number for replacement rate.)

Here's another one from wikipedia with total population, most likely peaking after 2050.

On the budget, here's an old chart from Nate Silver for US government spending specifically:

The post in which that chart appeared has lots more useful info.

For Chinese GDP, there's some decent answers on this quora question about how soon Chinese GDP per capita will catch up to the US. (Though note that I do not think Chinese GDP per capita will catch up to the US by 2040 - just to other first world countries, most of which have much lower GDP per capita than the US. For instance EU was around $36k nominal in 2019, vs $65k nominal for the US in 2019.) You can also eyeball this chart of historical Chinese GDP growth:

answer by Rochambeau · 2021-05-17T10:42:23.584Z · LW(p) · GW(p)

In terms of electricity, transmission and distribution make up 13% and 31% of costs respectively. Even if solar panels were free, I am not confident that reliable electricity would become 10x cheaper as unless each house as quite a few days of storage cheaply, they would still need distribution. Industrial electricity might approach that cheap, but I think it would depend on location and space availability otherwise at least some of the transmission and distribution costs would still exist.

comment by Daniel Kokotajlo (daniel-kokotajlo) · 2021-05-17T11:32:06.054Z · LW(p) · GW(p)

Thanks, this is a good point. I've edited my post to be less confident in non-AI energy uses. Also see my reply to Jacopo.

comment by Charlie Steiner · 2021-05-19T05:14:21.106Z · LW(p) · GW(p)

Speaking of transmission costs, I think the 2040 future there is carbon nanotube power lines.

Replies from: Rochambeau
comment by Rochambeau · 2021-05-20T16:49:46.950Z · LW(p) · GW(p)

I have always wondered how high we can push the voltage on transmission lines. Maybe carbon nanotubes being as conductive as copper but lighter and stronger than aluminum could allow significant transmission of electricity across time zones, stretching out the load and allowing solar to power dark areas for some time without increasing storage. Transmission towers would need to get obscenely large, but the lighter lines would a allow for fewer of them.

answer by Sammy Martin (SDM) · 2021-05-16T22:46:45.120Z · LW(p) · GW(p)

You cover most of the interesting possibilities on the military technology front, but one thing that you don't mention that might matter especially considering the recent near-breakdowns of some of the nuclear weapon treaties e.g. NEWSTART, is the further proliferation of nuclear weapons including fourth generation nuclear weapons like nuclear shaped charge warheads, pure fusion and sub-kiloton devices or tactical nuclear weapons - and more countries fitting nuclear-armed cruise missiles or drones with nuclear capability which might be a destabilising factor. If laser technology is sufficiently developed we may also see other forms of directed energy weapons becoming more common such as electron beam weapons or electrolasers

answer by rayom · 2021-05-17T09:35:03.693Z · LW(p) · GW(p)

Anything related to biotech is not included here - care to explain the reason why?

comment by ChristianKl · 2021-05-18T23:20:33.707Z · LW(p) · GW(p)

The incentive here is scientific discovery. 

You didn't answer the question about who you think would engage in that. It's interesting that you ignore the question.

Oh I guess you bring up Russians because they are the bad guys and most other countries are the good guys? 

No, because they have another culture with regards to science. You have people like Dmitry Itskov who are willing to persue projects that are not for profit (filing patents) nor for status in the academic world.

It's interesting that you see Americans being greedy as synonymous for them being the good guys. It suggests to me that you haven't thought hard about who does what for what reasons.

If there was any incentive to keeps nukes secret, they would've been kept secret, but the incentive to publicize nukes outweigh the incentive to keep them secret.

I have no idea what you mean which that argument. Who's they? What time are you speaking about? 

comment by Daniel Kokotajlo (daniel-kokotajlo) · 2021-05-17T20:38:24.899Z · LW(p) · GW(p)

It's only 19 years away; do you mean to say that there are already designer babies being born?

Replies from: gwern
comment by gwern · 2021-05-18T00:05:36.893Z · LW(p) · GW(p)

The 3 babies from He Jiankui will be adults by then, definitely; one might quibble about how 'designer' they are, but most people count selection as 'designer' and GenPred claims to have at least one baby so far selected on their medical PGSes (unclear if they did any EDU/IQ PGSes in any way, but as I've always pointed out, because of the good genetic correlations of those with many diseases, any selection on complex diseases will naturally also boost those).

comment by ChristianKl · 2021-05-19T08:48:34.694Z · LW(p) · GW(p)

Government agencies benefit from all forms of technologies. 

This assumes we are living in a world where the US government has the ability to fund far out biomedical research that's not benefitial to big pharma or another group that can lobby for it. 

In reality the US government isn't even able to stock enough masks for a pandemic. I'd love to live in a world where the US government would be able to fund science purely for the sake of scientific discovery independent from any interest groups but there's no reason to believe that's the world in which we are living.

Nukes are deterrents. That's the only reason to invest in them.

Again you fail to point out what time and which actors you are talking about which suggest not having a good model.

If we look at the US government, the US government pretended for a long time that only the president can order nuclear strikes while giving that ability to a bunch of military commanders and setting the nuclear safety codes to 00000000. 

If the only reason you invest in nukes is deterrence it makes no sense to have more people able to lunch nukes then the other side knows about. In that world the US government would have no reason to set the safety codes to 00000000 when ordered by congress to have safety codes. 

You might also watch the Yes, Prime Minister episode about nuclear deterence for more reasons (while it's exaggerted comedy, they did a lot of background research and talked to people inside the system about how the UK political system really worked at the time).

Most comments on here are just pure conjectures by people with mostly ML background. I can't say I'm educated enough to make these wild guesses on what it's like in 2040

I have enough expertise to make wild guesses about the future to have been paid for that. In the past I was invited by people funded by my government as an expert to participate in a scenario planning excercise that involved models scenario about medical progress.  

johnswentworth whom you replied to earns his money studying the history of progress and how it works, so is someone who has a fairly detailed model of how scientific progress works and isn't just someone who just has a ML background that's relevant.

LessWrong isn't a random Reddit forum. It's no place where it's a safe assumption that the people you are talking to don't have relevant experience to talk about what they are talking about. 

comment by ChristianKl · 2021-05-18T20:14:18.856Z · LW(p) · GW(p)

They key question here is incentives. What incentives is there to produce human clones (likely with more genetic defects then the original) if you can't publish papers afterwards or sell a product? 

I don't see any player that had the necessary ability 18 years ago and the incentive to make it happen. Which players do you consider to have both ability and incentive?

Russian billionaries come to mind but if one of them clones himself and treats the clone as his child that seems to be hard to keep secret. 

comment by Daniel Kokotajlo (daniel-kokotajlo) · 2021-05-18T12:50:12.738Z · LW(p) · GW(p)

Bold claim! Perhaps you should make a post (or shortform, or even just separate answer to this question) where you lay out your reasoning & evidence? I'd be interested in that.

comment by Daniel Kokotajlo (daniel-kokotajlo) · 2021-05-17T11:20:21.280Z · LW(p) · GW(p)

I haven't thought much about biotech and don't know much about it. This is why I made this a question rather than a post, I'm super interested to hear more things to add to the list!

answer by NoUsernameSelected · 2021-05-17T14:09:55.032Z · LW(p) · GW(p)

Metaculus predicts with ~75% probability that UBI will be implemented in at least one EU country.

answer by frontier64 · 2021-05-18T09:41:53.778Z · LW(p) · GW(p)

The constant improvements in nuclear tech will lead to multiple small terrorist organizations possessing portable nuclear bombs. We'll likely see at least a few major cities suffering drastic losses from terrorist threats.

Gene therapy will be strongly encouraged in some developed nations. Near the same level of encouragement as vaccines receive.

Pollution of the oceans will take over as the most popular pressing environmental issue.

comment by Daniel Kokotajlo (daniel-kokotajlo) · 2021-05-18T12:47:33.636Z · LW(p) · GW(p)

I'm especially interested in the nuclear bomb and gene therapy predictions; care to elaborate & explain your reasoning / evidence?

Replies from: frontier64
comment by frontier64 · 2021-05-23T15:12:49.466Z · LW(p) · GW(p)

Mandated Gene Therapy

We're trending towards health and medical decisions being looked at from a societal perspective rather than on the individual level.[1] . People who use alternative medicine are increasingly shamed not only for the effect their choice has on their own health, but for the effect it has on the health of others and the financial burden it puts on the medical system. Medical interventions later on are more costly therefore those 4 months you tried on herbal remedies hurt everybody who has to pay for your medical treatment. Refusing a vaccine not only increases burden the medical system will have taking care of you, but increases the risk that others will also get infected.

Gene therapy, specifically editing the genes of newborns, is the archetypal preventative medical procedure. Parents who have a baby they know will more than likely have a genetic disease and likely be an extra burden on the medical system will be shamed for that decision and the solution will be gene therapy.

That shame will be turned into laws. The natural extension of gene therapy laws for preventing known high likelihoods of genetic mutation will be gene therapy to prevent speculative risk and then just possible risk.

Privately-Owned Nukes

Honestly this doesn't even require improvements in nuclear tech. The only necessary ingredient is a couple of smart people joining a terrorist organization that wants to cause mass destruction and has the disposable resources of a small business. The design of nuclear bombs is freely available online, the actual engineering process is more arcane, but still learnable. The hardest part of the process is acquiring enough weapons grade uranium or plutonium. But even those can be made from scratch with access to a mine (even though spy movies always focus on the terrorist's stealing their nuclear material). So my first lemma is that even though it hasn't happened yet, it's pretty easy for a small group to create a nuclear bomb. And that's mostly based on my rudimentary knowledge of the comparative difficulty of creating a nuclear bomb vs. making a giant automotive plant or a

What's been holding private nuke construction back is a lack of impetus and general ineffectiveness of terrorists. But that's not a real bar to the end result. Over time there likely will be a statistical outlier terrorist organization that has a few smart people and the desire to construct nuclear bombs. And for them it will be easy.


  1. Taxpayer funded healthcare is the norm. Politicians talk about the opiod crisis and blame doctors for over-prescribing, people protest drug companies because they raise prices too high, a few national and international organizations have been setting the global policy on infectious disease handling for over a year now ↩︎

answer by MaxRa · 2021-05-17T14:28:10.619Z · LW(p) · GW(p)

Very cool prompt and list. Does anybody have predictions on the level of international conflict about AI topics and the level of "freaking out about AI" in 2040, given the AI improvements that Daniel is sketching out?

answer by Mary Chernyshenko · 2021-05-17T19:57:17.221Z · LW(p) · GW(p)

There will still be wars in Europe. I think conflicts will move west of Ukraine, if Ukraine still exists by that point.

comment by jacopo · 2021-05-18T14:24:54.280Z · LW(p) · GW(p)

It is certainly possible but what kind of scenario are you thinking about? 

For moving west of Ukraine the conflicts will have to involve EU or NATO countries, almost certainly both. So that would mean either an open Russia-NATO war or the total breakdown of both NATO and EU. Both scenarios would have huge consequences for the world as a whole, nearly as much as a war between China and US and allies.

Replies from: mary-chernyshenko, ChristianKl
comment by Mary Chernyshenko (mary-chernyshenko) · 2021-05-18T17:08:05.025Z · LW(p) · GW(p)

I don't imagine a big open war between Russia and whoever else, more like a series of ethnic conflicts which would quietly spring up. This scenario is convenient for most players.

...but this is probably just depression speaking, Europe does seem better put together now that I think of it.

comment by ChristianKl · 2021-05-18T14:31:34.504Z · LW(p) · GW(p)

For moving west of Ukraine the conflicts will have to involve EU or NATO countries, almost certainly both. So that would mean either an open Russia-NATO war or the total breakdown of both NATO and EU. 

No, Serbia is in Europe and West of Ukraine. The last war in that area didn't cause a breakdown of NATO or the EU and a new one wouldn't need to result in that either. 

Replies from: jacopo
comment by jacopo · 2021-05-18T15:13:01.803Z · LW(p) · GW(p)

No but all neighbors are, except Kosovo (and Bosnia that is on the track for NATO access). A new Serbia-Kosovo war (or Serbia-someone else) is in principle possible and as you say would not imply NATO breakdown. But US and EU have currently a strong grip on the region, the last war sent the message that they were willing to maintain it with force, and I think they have and will continue to have strong interest in no new war developing. And no country in the area should be suicidal enough to go against them. So I think the probability of open war there is very low, unless EU or NATO breakdown has already happened or is happening at the same time.

Replies from: ChristianKl
comment by ChristianKl · 2021-05-18T18:31:17.171Z · LW(p) · GW(p)

Another scenario would be NATO kicking out Turkey and some Greek/Turkish war. 

Replies from: None
comment by [deleted] · 2021-05-19T01:56:49.806Z · LW(p) · GW(p)

Is there any provision that allows members to be kicked out of NATO?

answer by Taran · 2021-05-17T11:45:18.189Z · LW(p) · GW(p)

AI-written books will be sold on Amazon, and people will buy them.  Specialty services will write books on demand based on customer specifications.  At least one group, and probably several, will make real money in erotica this way.  The market for hand-written mass market fiction, especially low-status stuff like genre fiction and thrillers, will contract radically.

comment by Mary Chernyshenko (mary-chernyshenko) · 2021-05-17T19:55:13.582Z · LW(p) · GW(p)

...academia will suffer even more from the influx of papers which were not written by their official authors. The job of the scientific editor will become that much harder.

Replies from: AllAmericanBreakfast
comment by DirectedEvolution (AllAmericanBreakfast) · 2021-05-18T22:17:51.993Z · LW(p) · GW(p)

By this point, academia may have begun shifting away from papers as a unit of information transmission, and away from manual editing. Some combination of NLP and standardized ontologies may replace them, a shift which is already underway.

answer by Sam Clarke · 2021-05-19T15:33:25.311Z · LW(p) · GW(p)

Epistemic effort: I thought about this for 20 minutes and dumped my ideas, before reading others' answers

  • The latest language models are assisting or doing a number of tasks across society in rich countries, e.g.
    • Helping lawyers search and summarise cases, suggest inferences, etc. but human lawyers still make calls at the end of the day
    • Similar for policymaking, consultancy, business strategising etc.
    • Lots of non-truth seeking journalism. All good investigative journalism is still done by humans.
    • Telemarketing and some customer service jobs
  • The latest deep RL models are assisting or doing a number of tasks in across society in rich countries, e.g.
    • Lots of manufacturing
    • Almost all warehouse management
    • Most content filtering on social media
    • Financing decisions made by banks
  • Other predictions
    • it's much easier to communicate with anyone, anywhere, at higher bandwidth (probably thanks to really good VR and internet)
    • the way we consume information has changed a lot (probably also related to VR, and content selection algorithms getting really good)
    • the way we shop has changed a lot (probably again due to content selection algorithms. I'm imagining there being very little effort between having a material desire and spending money to have it fulfilled)
    • education hasn't really changed
    • international travel hasn't really changed
    • discrimination against groups that are marginalised in 2021 has reduced somewhat
    • nuclear energy is even more widespread and much safer
    • getting some psychotherapy or similar is really common (>80% of people)
comment by ChristianKl · 2021-05-19T15:41:07.355Z · LW(p) · GW(p)

discrimination against groups that are marginalised in 2021 has reduced somewhat

Does that prediction inlude poor white people, BDSM people, generally everybody who has to strongly hide part of their identity when living in cities or only those groups that compatible with intersectional thinking?

Replies from: Sam Clarke
comment by Sam Clarke · 2021-05-19T16:32:36.401Z · LW(p) · GW(p)

I had something like "everybody who has to strongly hide part of their identity when living in cities" in mind

Replies from: ChristianKl
comment by ChristianKl · 2021-05-19T23:31:08.772Z · LW(p) · GW(p)

That suggests that groups that at the moment have no support at all will start to get support. Why do you think so?

Replies from: Sam Clarke
comment by Sam Clarke · 2021-05-21T11:32:04.608Z · LW(p) · GW(p)

Just an outside view that over the last decades, a number of groups who previously had to suppress their identities/were vilified are now more accepted (e.g., LGBTQ+, feminists, vegans), and I expect this trend to continue.

I'm curious if you expect this trend to change, or maybe we're talking about slightly different things here?

Replies from: ChristianKl
comment by ChristianKl · 2021-05-21T13:21:12.443Z · LW(p) · GW(p)

The groups LGBTQ+, feminists, vegans are part of one group of values and people we moved to a point where people with that group of values have no reason anymore to hide their identity when being in cities. 

Most of the identities where people currently have a lot to lose when they reveal their identity don't belong to that cluster. To the extend that the trend of that cluster becomes stronger continues, many people for whom it's currently very costly to reveal their identity won't gain anything and might even face a higher cost of revealing their identity. 

Generally, the more polarized a society is, the higher the amount of people who have something to lose by revealing their identity. I see rising polarization.

answer by Pattern · 2021-05-19T06:17:48.589Z · LW(p) · GW(p)
There's been 20 years of "Prompt programming" now, and so loads of apps have been built using it and lots of kinks have been worked out. Any thoughts on what sorts of apps would be up and running by 2040 using the latest models?

Prompt programming isn't as good as it was cracked up to be. In the past, as old timers never shut up about:

"Programmers* just wrote programs and they *** worked! Or at least gave errors that made sense! And there was some consistency! When programs crashed for 'no reason' there used to a reason! None of this 'neural network gets cancer from the internet's sheer stupidity and dies.' crap!"

In the present, programming is a more...mixed role. Debugging is a nightmare, brute force to find good (inputs to get good) outputs remains a distressingly large amount of the puzzle, despite all the fancy techniques that dress it up as something - anything - else in a world that no longer makes sense.

The people working on maintenance deal with the edge cases and see things they never wanted to see, as the price paid for wonderful tech. "The fat tails of the distribution" as Taleb put it, leads to more disillusionment, burnout, etc. in the associated professions when people try to do things too ambitious. This isn't some grand narrative about Atlantis, and hubris - just more of the same, with technology that can do wonderful things, but is consistently over budget, over-hyped - the dream of AGI remains hacked together, created half by 'machines' and half by people.

Imagine an ice cream machine that seems (far too often) to stop working just when you need it the most, a delicate piece of machinery that is always a pain..to deal with. This is the future of AI. (One second it works great, the next, a shitshow - a move nicknamed 'the treacherous swan dive'. Commercial applications secretly, under the hood involve way too much caching - saving good outputs - and a lot of people working to create stuff that extrapolates from past good outputs, and reasons by input similarity to cover holes that inevitably pop up. In other words, AI will secretly be 20% humans trying to answer the question 'what would the AI do if it worked right on this input instead of spewing nonsense?', when there's enough stuff to cover the holes. The other 80% of the time, it works amazing well, and performs feats once considered miraculous, though amazement quickly fades and consumers soon return to having ridiculous expectations, from this, rather brittle technology.)


*original sentence with typos: Where once programs just wrote programs and they *** worked!


Some examples of products and services:

More people try to do things - like write books, due to encouragement from AIs. Editors are astounded by the dramatic variation in quality over the course of manuscripts.

Fanfiction enters a new age. When just AIs write stories, new genres that play to their strengths result, but which are...difficult to understand. For instance, descendants of 'Egg Salad' which was were when a story is rewritten, with one of the characters...as an egg.

Eventually AIs gets good, but conflicts arise, like: Is the 'unofficial ending' written by GPZ really how the author would have wrapped things up? Or is, GPL's ending, though less exciting, more true to the themes of the story? Debates emerge about whether an author is or isn't human (and to what extent), and whether it's really art if it's not made by a person. Could a human being really have solved the who-dunit, given that it wasn't written by a human being? These arguments over the souls of fiction are taken seriously by some. Fans of Sherlock Holmes seem to care about the legacy/the future of mysteries. Other genres, it varies.


Starlink internet is fast, reliable, cheap, and covers the entire globe.

Starlink isn't super cheap. But the quality, given the price, is a great deal, and eventually it becomes very popular as people get tired of 'the internet being slow' or not working even for short periods of time. In order to cut costs, however, businesses** that 'don't really care about their customers' don't always invest in it, and remain a source of complaints.

**also schools, K-12.


3D printing is much better and cheaper now. Most cities have at least one "Additive Factory" that can churn out high-quality metal or plastic products in a few hours and deliver them to your door, some assembly required. (They fill up downtime by working on bigger orders to ship to various factories that use 3D-printed components, which is most factories at this point since there are some components that are best made that way)

A battle begins for the label 'artisanal'.

"But it looks like it was made by a real person!"

"It looks too good. The errors there, and here, and there - it's too authentic."


Unspeakable things happen in fashion, which becomes way more varied. In some places/groups, waste and conspicuous consumption (ridiculous number of clothes, changing all the time to keep up with fashion changing with speed unimaginable today) grow so extreme that it accidentally creates a competitor out of 'minimalism'*** (a small set of amazing clothes, possibly designed to be tweaked (regularly) a little bit to fit in, but not too much, and not too obviously).

***As a result of very popular/fashionable people being left behind, and fighting the trends, so they don't have to work at keeping up literally every second of every day.

Small props start to creep in.

And ridiculous hats make an astounding comeback!

answer by Dirichlet-to-Neumann · 2021-05-18T12:50:34.133Z · LW(p) · GW(p)

An all out war between China and the USA over Taiwan has crippled the whole world. 
Good news is AI alignment is not an issue anymore.

comment by ChristianKl · 2021-05-18T14:13:51.103Z · LW(p) · GW(p)

Are you saying that's the only scenario that would prevent singularity or are you saying that it's generally a probable scenario.

Replies from: Dirichlet-to-Neumann
comment by Dirichlet-to-Neumann · 2021-05-18T19:27:08.418Z · LW(p) · GW(p)

I'm saying a war between China and the USA is a possible scenario (with <50% but >10% probability). The intensity of destruction would depend on how the war is fought, but conditionally on a drawn out war nuclear mutual destruction does not seem implausible to me.

It's hopefully not the only scenario that prevent singularity though.

comment by Daniel Kokotajlo (daniel-kokotajlo) · 2021-05-18T12:52:29.316Z · LW(p) · GW(p)

Bold claim! Perhaps you should make a post (or shortform, or even just separate answer to this question) where you lay out your reasoning & evidence? I'd be interested in that. If you think it's infohazardous, maybe just a gdoc?

Replies from: Dirichlet-to-Neumann
comment by Dirichlet-to-Neumann · 2021-05-18T19:23:11.869Z · LW(p) · GW(p)

Well it was mostly meant as a joke. But here are my probabilities.
- China occupies Taiwan 90% - this is clearly a mid term goal of the current Chinese government.
- The USA defend Taiwan 30% (maybe 50% if the invasion happens in the near future). The USA may or may not hold their defense agreement - I think the later the invasion happens, the more likely it is that the USA just let Taiwan be invaded.

The war could be a quick win for China (they have home advantage). If it keep going for a few years, I still think China will win (the coronavirus crisis is strong evidence that western democracies in general would not be able to adapt quickly to an existential crisis). But a long wars would mean heavy destruction of infrastructure, and would surely increase dramatically the chances of a nuclear war. 
I'd say something like 30% chances of a nuclear war, conditional on a drawn out war between the two superpower ?
 

Replies from: daniel-kokotajlo
comment by Daniel Kokotajlo (daniel-kokotajlo) · 2021-05-19T05:47:33.939Z · LW(p) · GW(p)

Does the chinese government have a 90% success record at achieving their mid term goals? Do you have a 100% success record at judging what the chinese government thinks is a mid-term goal?

Replies from: dxu
comment by dxu · 2021-05-19T06:29:37.292Z · LW(p) · GW(p)

Neither of those seem to me like the right questions to be asking (though for what it's worth the answer to the first question has been pretty clearly "yes" if by "Chinese government" we're referring specifically to post-2001 China).

Having said that, I don't think outside-viewing these scenarios using coarse-grained reference classes like "the set of mid-term goals China has set for itself in the past" leads to anything useful. Well-functioning countries in general (and China in particular) tend to set goals for themselves they view as achievable, so if they're well-calibrated it's necessarily the case that they'll end up achieving (a large proportion of) the goals they set for themselves. This being the case, you don't learn much from finding out China manages to consistently meet its own goals, other than that they've historically done a pretty decent job at assessing their own capabilities. Nor does this allow you to draw conclusions about a specific goal they have, which may be easier or more difficult to achieve than their average goal.

In the case of Taiwan: by default, China is capable of taking Taiwan by force. What I mean by this is that China's maritime capabilities well exceed Taiwan's defensive capacity, such that Taiwan's continued sovereignty in the face of a Chinese invasion is entirely reliant on the threat of external intervention (principally from the United States, but also by allies in the region). Absent that threat, China could invade Taiwan tomorrow and have a roughly ~100% chance of taking the island. Even if allies get involved, there's a non-negligible probability China wins anyway, and the trend going forward only favors China even more.

Of course, that doesn't mean China will invade Taiwan in the near future. As long as its victory isn't assured, it stands to lose substantially more than from a failed invasion than it stands to gain from a successful one. At least for the near future, so long as the United States doesn't send a clear signal about whether it will defend Taiwan, I expect China to mostly play it safe. But there's definitely a growing confidence within China that they'll retake Taiwan eventually, so the prospect of an invasion is almost certainly on the horizon unless current trends w.r.t. the respective strengths of the U.S. and Chinese militaries reverse for some reason. That's not out of the question (the future is unpredictable), but there's also no particular reason to expect said trends to reverse, so assuming they don't, China will almost certainly try to occupy Taiwan at some point, regardless of what stance the U.S. takes on the issue.

(Separately, there's the question of whether the U.S. will take a positive stance; I'm not optimistic that it will, given its historical reluctance to do so, as well as the fact that all of the risks and incentives responsible for said reluctance will likely only increase as time goes on.)

15 comments

Comments sorted by top scores.

comment by johnswentworth · 2021-05-18T02:26:13.336Z · LW(p) · GW(p)

The predictions about AI-adjacent things seem weird when we condition on AGI not taking off by 2040. Conditional on that, it seems like the most likely world is one where the current scaling trends play out on the current problems, but current methods turned out to not generalize very well to most real-world problems (especially problems without readily-available giant data sets, or problems in non-controlled environments). In other words, this turns out pretty similar to previous AI/ML booms: a new class of problems is solved, but that class is limited, and we go into another AI winter afterwards.

In that world, I'd expect deep learning to be used commercially for things which we're already close to: procedural generation of graphics for games and maybe some movies, auto-generation of low-quality written works (for use-cases which don't involve readers paying close attention) or derivative works (like translations or summaries), that sort of thing. In most cases, it probably won't be end-to-end ML, just tools for particular steps. Prompt programming mostly turns out to be a dead end, other than a handful of narrow use-cases. Automated cars will probably still be right-around-the-corner, with companies producing cool demos regularly but nobody really able to handle the long tail [LW · GW]. People will stop spending large amounts on large models and datasets, though models will still grow slowly as compute & data get cheaper.

Replies from: daniel-kokotajlo
comment by Daniel Kokotajlo (daniel-kokotajlo) · 2021-05-18T06:01:18.656Z · LW(p) · GW(p)

I was trying hard to do exactly what you recommend doing here, and focus on only the AI-related stuff that seems basically "locked in" at this point and will happen even if no AGI etc. I think +5 OOMs of compute to train AIs by 2040 makes sense in this framework because +2 will come from reduced cost and it's hard for me to imagine no one spending a billion dollars on an AI training run by 2040. I guess that could happen if there's an AI winter, but that would be a trend-busting event... Anyhow, it seems like spending & self-driving-cars are the two cases where we disagree? You think they are more closely connected to AGI than I did, such that conditionalizing on AGI not happening means those things don't happen either? Would you then agree e.g. that in 2025 we have self-driving cars, or billion-dollar models, you'd be like "well fuck AGI is near?" (Or maybe you already have short timelines?)

Replies from: johnswentworth
comment by johnswentworth · 2021-05-19T03:01:19.218Z · LW(p) · GW(p)

You think they are more closely connected to AGI than I did, such that conditionalizing on AGI not happening means those things don't happen either? Would you then agree e.g. that in 2025 we have self-driving cars, or billion-dollar models, you'd be like "well fuck AGI is near?"

Self-driving cars would definitely update me significantly toward shorter timelines. Billion-dollar models are more a downstream thing - i.e. people spending billions on training models is more a measure of how close AGI is widely perceived to be than a measure of how close it actually is. So upon seeing billion-dollar models, I don't think I'd update much, because I'd already have updated on the things which made someone spend a billion dollars on a model (which may or may not actually be strong evidence for AGI being close).

In this world, I'd also expect that models are not a dramatic energy consumer (contra your #6), mainly because nobody wants to spend that much on them. I'd also expect chatbots to not have dramatically more usage than today (contra your #7) - it will still mostly be obvious when you're talking to a chatbot, and this will mostly be considered a low-status/low-quality substitute for talking to a human, and still only usable commercially for interactions in a very controlled environment (so e.g. no interactions where complicated or free-form data collection is needed). In other words, chatbot use-cases will generally be pretty similar to today's, though bot quality will be higher. Similar story with predictive tools - use-cases similar to today, limitations similar to today, but generally somewhat better.

Replies from: ChristianKl, daniel-kokotajlo
comment by ChristianKl · 2021-05-19T11:21:54.551Z · LW(p) · GW(p)

I would expect a lot of chat bot usecases to be a mix of humans and bots. The bot can autogenerated text and then a human can check whether that's correct which takes less time then the human writing everything themselves. 

comment by Daniel Kokotajlo (daniel-kokotajlo) · 2021-05-19T10:42:04.829Z · LW(p) · GW(p)

Interesting. I think what you are saying is pretty plausible... it's hard for me to reason about this stuff since I'm conditionalizing on something I don't expect to happen (no singularity by 2040).

comment by Robert Miles (robert-miles) · 2021-05-19T11:49:05.911Z · LW(p) · GW(p)

On point 12, Drone delivery: If the FAA is the reason, we should expect to see this already happening in China?

My hypothesis is, the problem is noise. Even small drones are very loud, and ones large enough to lift the larger packages would be deafening. This is something that's very hard to engineer away, since transferring large amounts of energy into the air is an unavoidable feature of a drone's mode of flight. Aircraft deal with this by being very high up, but drones have to come to your doorstep. I don't see people being ok with that level of noise on a constant, unpredictable basis.

Replies from: daniel-kokotajlo, None
comment by Daniel Kokotajlo (daniel-kokotajlo) · 2021-05-19T16:19:02.219Z · LW(p) · GW(p)

Good point. OTOH, I feel like there are some cities in the world (maybe in China?) where it's super noisy most of the time anyway, with lots of honking cars and whatnot. Also there are rural areas where you don't have neighbors to annoy.

Replies from: CarlShulman
comment by CarlShulman · 2021-05-20T13:51:30.346Z · LW(p) · GW(p)

There is at least one firm doing drone delivery in China and they just approved a standard for it.

comment by [deleted] · 2021-05-21T01:22:38.108Z · LW(p) · GW(p)

Lawnmowers are also very loud yet is widely tolerated (more or less). Plus, delivery drones need only to drop off the package and fly away; the noise pollution will only last for a few seconds. I also don't see why it would necessarily be unpredictable; drones don't get stuck in traffic. Maybe a dedicated time window each day becomes an industry standard.

But the real trouble I see with delivery drones is: what's the actual point? What problem is being solved here? Current delivery logistics work very well, I don't see much value being squeezed out of even faster/more predictable delivery. Looks like another solution in search of a problem to me.

Replies from: lsusr
comment by lsusr · 2021-05-21T03:18:14.649Z · LW(p) · GW(p)

To me, the most important thing isn't speed or predictability. It's price. Current delivery methods require a human being. People are expensive. If a delivery drone removes the human being from the equation then that could remove a significant fraction of the price.

Replies from: ChristianKl, None
comment by ChristianKl · 2021-05-22T23:31:04.288Z · LW(p) · GW(p)

There are multiple land based delievery methods that don't require a human: https://www.gearbrain.com/autonomous-food-delivery-robots-2646365636.html

comment by [deleted] · 2021-05-21T16:12:09.707Z · LW(p) · GW(p)

Why couldn't land-based delivery vehicle become autonomous though? That would also cut out the human in the loop.

One reason might be that autonomous flying drone are easier to realize. It is true that air is an easier environment to navigate than ground, but landing and taking off at the destination could involve very diverse and unpredictable situations. You might run into the same long tail problem as self-driving cars, especially since a drone that can lift several kilos has dangerously powerful propellers.

Another problem is that flying vehicles in general are energy inefficient due to having to overcome gravity, and even more so at long distances (tyranny of the rocket equation). Of course you could use drones just for the last mile, but that's an even smaller pool to squeeze value out of.

In general, delivery drones seem less well-suited for densely populated urban environments where landing spots are hard to come by and you only need a few individual trips to serve an entire apartment building. And that's where most of the world will be living anyway.

comment by Daniel Kokotajlo (daniel-kokotajlo) · 2022-12-06T04:14:37.284Z · LW(p) · GW(p)

The underlying assumption of this post is looking increasingly unlikely to obtain. Nevertheless, I find myself back here every once and a while, wistfully fantasizing about a world that might have been.

I think the predictions hold up fairly well, though it's hard to evaluate, since they are conditioning on something unlikely, and because it's only been 1.5 years out of 20, it's unsurprising that the predictions look about as plausible now as they did then. I've since learned that the bottleneck for drone delivery is indeed very much regulatory, so who knows whether it'll exist in 2040. We still don't have flying cars, after all, for basically-regulatory reasons. The military technology I outlined is looking ever-more-plausible thanks to the war in Ukraine illustrating the importance of drones of various kinds.

comment by Sam Clarke · 2021-05-19T10:04:25.334Z · LW(p) · GW(p)

Thanks for this, really interesting!

Meta question: when you wrote this list, what did your thought process/strategies look like, and what do you think are the best ways of getting better at this kind of futurism?

More context:

  • One obvious answer to my second question is to get feedback - but the main bottleneck there is that these things won't happen for many years. Getting feedback from others (hence this post, I presume) is a partial remedy, but isn't clearly that helpful (e.g. if everyone's futurism capabilities are limited in the same ways). Maybe you've practised futurism over shorter time horizons a lot? Or you expect that people giving you feedback have?
  • After reading the first few entries, I spent 20 mins writing my own list before reading yours. Some questions/confusions that occurred:
    • All of my ideas ended up with epistemic status "OK, that might happen, but I'd need to spend at least a day researching this to be able to say anything like "probably that'll happen by 2040" "
      • So I'm wondering if you did this/already had the background knowledge, or if I'm wrong that this is necessary
    • My strategies were (1) consider important domains (e.g. military, financial markets, policymaking), and what better LMs/deep RL/DL in general/other emerging tech will do to those domains; (2) consider obvious AI/emerging tech applications (e.g. customer service); (3) look back to 2000 and 1980 and extrapolate apparent trends.
      • How good are these strategies? what other strategies are there? how should they be weighed?
    • How much is my bottleneck to being better at this (a) better models for extrapolating trends in AI capabilities/other emerging tech vs (b) better models of particular domains vs (c) better models of the-world-in-general vs (d) something else?
Replies from: daniel-kokotajlo
comment by Daniel Kokotajlo (daniel-kokotajlo) · 2021-05-19T10:36:41.775Z · LW(p) · GW(p)

Thanks! Good idea to make your own list before reading the rest of mine--I encourage you to post it as an answer.

My process was: I end up thinking about future technologies a lot, partly for my job and partly just cos it's exciting. Through working at AI Impacts I've developed a healthy respect for trend extrapolation as a method for forecasting tech trends; during the discontinuities project I was surprised by how many supposedly-discontinuous technological developments were in fact bracketed on both sides by somewhat-steady trends in the relevant metric. My faith in trend extrapolation has made successful predictions at least once, when I predicted that engine power-to-weight ratios would form a nice trend over two hundred years and yep [LW · GW].

As a result of my faith in trend extrapolation, when I think about future techs, the first thing I do is google around for relevant existing trends to extrapolate. Sometimes this leads to super surprising and super important claims, like the one about energy being 10x cheaper. (IIRC extrapolating the solar energy trend gets us to energy that is 25x cheaper or so, but I was trying to be a bit conservative).

As for the specific list I came up with: This list was constructed from memory, when I was having trouble focusing on my actual work one night. The things on the list were things I had previously concluded were probable, sometimes on the basis of trend extrapolation and sometimes not.

I wouldn't be surprised if I'm just wrong about various of these things. I don't consider myself an expert. Part of why I made the post is to get pushback, so that I could refine my view of the future.

I don't know what your bottleneck is, I'm afraid. I haven't even seen your work, for all I know it's better than mine.

I agree feedback by reality would be great but alas takes a long time to arrive. While we wait, getting feedback from each other is good.