PSA. The report includes a Colab notebook that allows you to run Ajeya’s model with your own estimates for input variables. Some of the variables are “How many FLOP/s will a transformative AI run on?”, “How many datapoints will be required to train a transformative AI?”, and “How likely are various models for transformative AI (e.g. scale up deep learning, recapitulate learning in human lifetime, recapitulate evolution)?”. If you enter your estimates, the model will calculate your personal CDF for when transformative AI arrives.
Here is a screenshot from the Colab notebook. Your distribution (“Your forecast”) is shown alongside the distributions of Ajeya, Tom Davidson (Open Philanthropy) and Jacob Hilton (OpenAI). You can also read their explanations for their distributions under “Notes”. (I work at Ought and we worked on the Elicit features in this notebook.)
These are patients who had a positive test in April. Most infected people without symptoms or with mild symptoms did get tested in April in the US. We know about 20-40% are asymptomatic, with higher % among younger people. So actual rate based on this study would be upper bounded by 1/4 (not 1/3) and point estimate closer to 1/5. (I also agree with SDM).
To be clear, I think the 71% result needs more investigation and (on priors) is probably lower. Yes, there is reason to expect overshoot. It seems the amount of overshoot would vary based on (a) NPIs being taken at the time (e.g. are some people never leaving the house) and (b) proportion of people who have cross-immunity or innate reduced susceptibility. (In principle, you could imagine 80% of people in a town live as normal and 20% won't leave the house till the pandemic is over.) Again, I think if we did a lot of studies, we'd get a sense of both the minimum herd immunity threshold and the variability in overshoot.
This naive model is not a straw man! Such obvious nonsense models are the most common models quoted by the press, the most common models quoted by so-called ‘scientific experts’ and the most common models used to determine policy.
I think you underestimate the sophistication of the top epidemic modelers: Neil Ferguson, Adam Kucharski, Marc Lipsitch, and others. I tend to agree we need urgent empirical work on herd immunity thresholds (see my other comment) but the top epi people are aware of the considerations you raise. Communicating with the public is very challenging under the current circumstances and so it's reasonable these people would choose words carefully.
Your statement is also empirically false. One of the most influential models is the "Imperial Model", which certainly impacted UK policy and probably US and European policy too. Other countries did versions of the model. The lead researcher on the model literally became a household name in the UK. The Imperial Model is an agent-based model (not an SIR model). It has a very detailed representation of how exposure/contact differ among different age groups (work vs. school) and in regions with different population densities. It doesn't assume the only intervention is immunity, and follow up work has tested many different interventions. (AFAIK, it does assume equal susceptibility. But as it's an agent-based model you could experiment with heterogeneity in susceptibility. And I think evidence for variable susceptibility for reasons other than age remains fairly weak: https://twitter.com/OwainEvans_UK/status/1268873649202909185)
IMO what's needed here is detailed empirical analysis. There are many places round the world that have had spread that was only weakly controlled. If you get the % seropositive for a bunch of places, you could (to some extent) extrapolate to Europe/US/East Asia, where there's currently more control. Here's where I'd look:
Brazil has had a raging epidemic for quite a few months. % positive tests is currently >70%. It seems very likely that some towns have hit herd immunity. Similar story for South Africa and Mexico. (Many other countries have similarly bad epidemics, but these three have relatively good data.)
Some Indian states had >25% seropositive in studies that started in early July and there are huge number of new cases since then. Again, some towns have probability hit herd immunity.
Could also look at villages near Bergamo in Italy.
This study found 16% seropositive in a small town in Germany (e.g. with low population density). This town was locked down after an outbreak and so the 16% almost certainly underestimates the herd immunity threshold. This study was done pretty carefully (though the lead author has an axe to grind).
Probably the only engineering fields that are doing really well are computer science and maybe, at this point, petroleum engineering. And most other areas of engineering have been bad career decisions the last 40 years … Nuclear engineering, aerospace engineering [were catastrophic fields to go into]
Where's his evidence on this? This data suggests average salaries for engineers outside software engineering were not much different from software engineering. I'd guess there's more exciting new companies in computing than in aerospace, but it doesn't mean it was a "catastrophic career move". US companies also sell a lot of products abroad and there's been huge growth in use of aircraft, cars, and other engineered products worldwide (due to catch up growth).
Why did all the rocket scientists go to work on Wall Street in the ‘90s to create new financial products?
Because the Cold War ended. There's no big mystery. If you weren't "allowed" to make rockets, how to explain SpaceX (started in 2002)? Not to say regulation doesn't limit innovation, but I'd want to see actual data on this and not just bluster.
You are understanding correctly. Here are some things to keep in mind:
The reproductive number R before lockdowns was estimated at 2-3. People are infectious for 4-7 days. The average person has contact with about 10 other people daily (paper). So there could be 20-50 unique contacts over 4-7 days. Maybe 10-30 of those are higher risk contacts (long duration, close proximity). So only 5-33% of higher risk contacts are being infected (using these very rough numbers). So I'd say that Covid is not very contagious. Note that R for measles is 12-18!
There is probably some overdispersion. Say 20% of people do 60-80% of all infecting. So many people cause zero new infections. I'd presume such people just aren't very infectious and so even if they spend a lot of time with household members they won't infect them.
The 30% is averaging over all household members, which includes children. Children are probably less susceptible (e.g. they might have 50% lower risk of infection).
Once someone develops Covid symptoms, many households will intervene to reduce exposure. If adult children get sick, they are likely to try to isolate from their older parents.
There is a small number of studies that distinguish spouse from other relationships. See Figure S5 of this paper. I don't think there's enough data to draw a strong empirical conclusion. Most of our data for estimating SAR is from China/Korea/Taiwan and I'd guess these are mostly nuclear families or extended family (not many group house / flatmates).
I've talked extensively over many posts about why I think herd immunity is a bigger deal than people think
I understood the argument as "there'll be herd immunity faster in specific locations (e.g. subway riders or people under 20 in some neighborhood)". The logic makes sense but I'd guess the effect is small, due to population mixing / small-world network effects. Young people are probably getting infected more but they are still far from HI everywhere and they are probably well mixed. I haven't seen any positive empirical evidence for your view over my take (big first wave --> people take precautions more seriously and have slower reopening + 20-30% drop in R due to fewer susceptible).
There's Google/Apple style mobility (which actually records amount of time spent in work/home/retail/public transit) and questionnaires that ask for "number of contacts per day". People have used both to model cases/deaths and they are both pretty useful. Somepapers (China) and UK. The point is that we know you can predict spread using these proxies for contact. So you can actually see if the amount of predicted contact is lower in NYC, London, Madrid and Lombardy vs. places that didn't have a big first wave (e.g. LA, Miami, Phoenix). And the predicted contact was lower in the former places. (But I haven't done a careful study).
2. Sweden did badly, but it's important to notice that it did far less badly than a naive model would expect it to do. Why did things end up getting contained when they did? Why wasn't it much worse?
Public transit use was down 55% in Sweden at peak and is still at -7%. Norway was down 65%. Swedes stopped going to the cinema and other high-risk venues were way down. Without a formal lockdown, there was a huge change of behavior in Sweden. I'd guess Swedes were aware that all the countries around them had tighter restrictions and much lower death tolls. So they acted to reduce risk. (People in the UK also reduced risk more than was required by government.) So I don't see any mystery in Sweden. The real mysteries: Vietnam, Thailand, Cambodia, Laos and Indonesia. And I'm surprised how well the SF Bay has done.
4. It's shocking because those people are having very intimate contact over extended periods of time
Agree it goes against the naive model. But if you take seriously that 20% of people do 80% of infecting (or maybe a bit less than that), then it's likely that a decent proportion are essentially not infectious. Also note that many household members are younger children, who are harder to infect.
There has not been a substantial second wave anywhere there was a strong first wave. This implies that herd immunity, as I’ve noted here, is likely playing a big role.
Iran had a big outbreak (much bigger than official numbers) and now has a clear second wave. Some of this is in different cities but I haven't found a careful analysis.
Have you looked at the mobility data? Most (maybe all) of the places in Europe that had a strong first wave kept mobility low (esp. public transport and workplace) and have adopted strict social distancing in public spaces and some level of masking. I haven't seen any good evidence that herd immunity is playing more role than we would expect (e.g. 15-30% reduction in R in worst hit places like NYC, Lombardy, London, Madrid).
I think there's pretty good evidence that most adults are susceptible from the huge outbreaks in prisons, meat plants and hospitals with bad PPE.
2. Sweden didn’t come out of this as the hero, but things were nowhere near as bad as the critics predicted for it, and cases managed to peak and then steadily decline.
This seems wrong-headed. Sweden has been terrible compared to ALL its geographic neighbors (which are very similar culturally and started from the same position). They have a very high death rate and a poor rate of testing. They have suffered substantial economic damage despite not locking down. AFAIK, the hospital system "survived" the peak in part because they did not treat the most vulnerable. As well as low testing, they have very low mask use, and so they are poorly set up for the end of summer (when people will be inside much more). I also think Sweden is pretty different from the US. I'd expect compliance with testing, contact tracing and isolation to be high in Sweden, while it seems compliance in NYC is not high.
4. Household infection rate is shockingly low.
Why is it shocking? If R=2.5 with zero social distancing and tons of superspreader events, it's just not that infectious. We also know there is variability in infectiousness across individuals (overdispersion). SARS-1 had a household secondary attack rate of 8%, H1N1 flu had rates that varied across studies but they are mostly lower than 38% (e.g. 15-20% is common).
Regarding children, this meta-analysis finds adults have a 1.4x higher risk of infection. Not a huge difference. Kids are much more likely to have mild symptoms or be asymptomatic.
A detailed investigation of outbreak in a South African hospital found pretty good evidence for transmission by surfaces or indirect contact (nurse touches infected person A and then later touches susceptible person B). Doesn't mean risk from deliveries is significant but worth being wary of surfaces in general.
According to current evidence, SARS‐CoV‐2 is transmitted between people through respiratory droplets and contact routes. Droplet transmission may also occur through fomites so transmission of the virus can occur by direct contact with an infected person or indirect contact with surfaces in the immediate environment of that person or with objects used on the infected person (e.g. stethoscope or thermometer). The spatial distribution of cases and exposed individuals who became infected on the wards suggests that indirect contact via health care workers or fomite transmission were the predominant modes of transmission between patients in this outbreak. Direct droplet or contact transmission would be plausible where the people that were exposed were located in close proximity to an infectious case, e.g. P4 in the bed directly opposite P3 on MW1 between 13 ‐ 16 March (Figure 7); or X1 and X3 sharing a four‐bedded bay with P7 on MW1 between 27 March ‐ 2 April (Figure 10). However, in other examples the exposed individuals were located in different rooms and different areas of the ward, making indirect contact via health care workers or fomite transmission more plausible. We also present evidence suggestive of direct droplet transmission from a symptomatic health care worker to two patients on the neurology ward.
Every country should be going crazy doing studies and RCTs to understand (a) most likely causes of transmission, (b) PPE and ways of organizing work/public spaces to minimize risk. Under lockdown, it should be much easier to work out the cause of infection. There are also enormous numbers of hospitals and large care homes all around the world (E.g. janitors and cleaners would seem higher risk if surfaces/aerosol is a major mode of transmission.)
We also need much more work on outdoor/indoor transmission, as this would be a super cheap intervention if it helped. Is there any study of infections by job, say in Germany where the volume of tests is high? Under lockdown, most infections will be from work. So compare indoor vs. outdoor jobs. (What about houses with gardens/decks vs. not in the same area, in a place where the weather is nice in March/April?)
For human interaction while avoiding droplet transmission: we need an app for this. You see a stranger 10m away and want to talk to them. The app would ping their phone via bluetooth and initiate a call. So you can see/gesture to the person but the app enforces a safe distance. Everyone interacting with the public (stores, public transport) should use this app. (Over time, the app could factor in indoor/outdoor and even estimate the safety of indoor spaces based on ventilation).
At many superspreader events (e.g. Korean call center) there are a bunch of people who seem to have very similar exposure to the virus. Yet a substantial proportion (50% in the call center case) don't get infected. This will partly be due to randomness in droplets. But I'd a substantial part of this is variance in infectability. Are some people less infectable in general, or just relative to a particular person? (Younger people seem less infectable in general, and I've heard the suggestion that antibodies to other coronaviruses may provide weak immunity to covid). Natural (weak) immunity would also help explain why if someone in your household is infected you have only a ~20% chance of being infected by them). Someone should use the SSE case studies and try to tabulate this.
New study from South Korea of spread in a crowded call center. There were 94 infections on one floor (43% of workers on the floor). As most people had symptom onset during a three day period, this suggests 1-2 people were superspreaders. They have a seating chart, which suggests the secondary attack rate was significantly higher for people sitting in the same room (eyeballing maybe 60%). It's notable that some people don't get infected, despite spending 4-5 days full workdays being exposed to a superspreader and possibly other infectious people. Only 4% were asymptomatic for the whole period of the study.
They tested households of the infected office workers and get a household secondary attack rate of 16%. How much were people trying to avoid infecting their families? It's hard to say from the study, but we know the following:
1. This was around the peak of cases in South Korea. People would be primed to take Covid-like symptoms seriously.
2. After a few days where many workers developed symptoms, the office was closed. At this point, it seems very likely that most workers took efforts to isolate from their families.
3. 72% of subjects are women, with mean age 38. It seems that having roommates is relatively rare among Koreans. I'd guess these are nearly all people living older parents and nuclear families. (It's easier for someone to isolate from their parents or older children than from spouse or young kids).
4. From other studies, under 18s are less likely than adults to get secondary infections and the number is very low for under 10s. It's not clear whether children were tested, but they list 2.3 household contacts per person, which suggests they are. If 1/5 of contacts were younger children, and we removed them, you'd get a secondary attack rate of ~20%.
So what about roommates living together? I'd guess:
1. If people are fairly sensitive to Covid symptoms and make some efforts to isolate, 15-25% secondary attack rate.
2. If people don't make any effort to isolate after onset of symptoms, 20-40%.
The spread in the call center and other studies of choirs/restaurants suggest that direct physical contact is not necessary for very effective spread. So roommates spending time together in common spaces would be at high risk.
Various places got a lot of traffic from Wuhan before it was shut down: Singapore, Thailand, the US, Europe, Korea, Australia, etc. It's clear that Europe's outbreak is worse than the US/Australia/Singapore. It seems likely that things are worse in the colder parts of the US (vs. Texas or Florida).
Iran was not testing/reporting. There are many tropical / Southern Hemisphere places that could have had an Iran style outbreak and which had a lot more traffic from Wuhan than Iran does. Why Iran?
There's also lots of artistic concepts where the dependence on the medium is highly significant
Great examples. I agree the physical medium is really important in human art: see my Section 1.3.1.
It seems like it's not a surprise that NNs would be good at perspective compared to humans, since there's a cleaner inverse between the perceptive and the creation of perspective from the GAN's point of view than the human's (who has to use their hands to make it, rather than their inverted eyes).
I like the point about hands vs. "inverted eyes". At the same time, the GANs are trained on a huge number of photos, and these photos exhibit a perfect projection of a 3D scene onto a finite-size 2D array. The GAN's goal is to match these photos, not to match 3D scenes (which it doesn't know anything about). Humans invented perspective before having photos to work with. (They did have mirrors and primitive projection techniques.)
I think most humans have pretty good facility with creating and understanding 'stick figures' that comes from training on a history of communicating with other humans using stick figures, rather than simply generalizing from visual image recognition,
I agree that our facility with stick figures probably depends partly on the history of using stick figures. However, I think our general visual recognition abilities make us very flexible. For example, people can quickly master new styles of abstract depiction that differ from the XKCD style (say in a comic or a set of artworks). DeepMind has a cool recent paper where they learn abstract styles of depiction with no human imitation or labeling.
We might want to look for find concepts that are easier for humans than NNs; when I talk to people about ML-produced music, they often suggest that it's hard to capture the sort of dependencies that make for good music using current models (in the same way that current models have trouble making 'good art' that's more than style transfer or realistic faces or so on; it's unlikely that we could hook a NN up to a DeviantArt account and accept commissions and make money).
Currently humans play a major role in the interesting examples of neural art. Getting more artist-like autonomy is probably AI-complete, but I can imagine neural nets being more and more widely used in both visual art and music. I agree there’s great potential in neural music! (I suggest some experiments in my conclusion but there's tons more that could be tried).
You'd need a third and separate scheme to make Kandinskys, and then I'd just bring up another artist not covered yet.
Again, replicating all human art is probably AGI-complete. However, there are some promising strategies for generating non-representational art and I’d guess artists were (implicitly) using some of them. Here are some possible Sensory Optimization objectives:
1. Optimize the image to be a superstimulus for random sets of features in earlier layers (this was already discussed).
2. Use Style Transfer to constrain the low-level features in some way. This could aim at grid-like images (Mondrian, Kelly, Albers) or a limited set of textures (Richter). This is mentioned in Section 1.3.1.
3. If you want the image to evoke objects (without explicitly depicting them), then you could combine (1) and (2) with optimizing for some object labels (e.g. river, stairs, pole). This is simpler than your Kandinsky example but could still be effective.
4. In addition to (1) and (2), optimize the image for human emotion labels (having trained on a dataset with emotion labels for photos). To take a simplistic example: people will label photos with lots of green or blue (e.g. forest or sea or blue skies) as peaceful/calming, and so abstract art based on those colors would be labeled similarly. Red or muddy-gray colors would produce a different response. This extends beyond colors to visual textures, shapes, symmetry vs. disorder and so on. (Compare this Rothko to this one).
Maybe you could train an AI on patriotic paintings and then it could produce patriotic paintings, but I think only by working on theory of mind would an AI think to produce a patriotic painting without having seen one before.
I agree with your general point about the relevance of theory of mind. However, I think Sensory Optimization could generate patriotic paintings without training on them. Suppose you have a dataset that's richer than ImageNet and includes human emotion and sentiment labels/captions. Some photos will cause patriotic sentiments: e.g. photos of parades or parties on national celebrations, photos of a national sports team winning, photos of iconic buildings or natural wonders. So to create patriotic paintings, you would optimize for labels relating to patriotism. If there are emotional intensity ratings for photos, and patriotic scenes cause high intensity, then maybe you could get patriotic paintings by just optimizing for intensity. (Facebook has trained models on a huge image dataset with Instagram hashtags -- some of which relate to patriotic sentiment. Someone could run a version of this experiment today. However, I think it's a more interesting experiment if the photos are more like everyday human visual perception than carefully crafted/edited photos you'll find on Instagram.)
I was thinking of how some things aren't art if they're normal sized, but if you make them really big, then they're art.
Again, I expect a richer training set would convey lots of this information. Humans would use different emotional/aesthetic labels on seeing unusually large natural objects (e.g. an abnormally large dog or man, a huge tree or waterfall).
For "limited," I imagined something like Dennett's example of the people on the bridge. The artist only has to paint little blobs, because they know how humans will interpret them.
Some artworks depend on idiosyncratic quirks of human visual cognition (e.g. optical illusions). It's probably hard for a neural net to predict how humans will respond to all such works (without training on other images that exploit the same quirk). This will limit the kind of art the Sensory Optimization model can generate. Still, this doesn't undermine my claim that artists are doing something like Sensory Optimization. For example, humans have a bias towards seeing faces in random objects -- pareidolia. By exploiting this, artists exploit an image that looks like two things at once. (The artist knows the illusion will work, because it works on his or her own visual system).
My impression is that DeepDream et al. have been trained to make visual art - by hyperparameter tuning (grad student descent).
I think this first blogpost on Deep Dream and the original paper on Style Transfer already were already very impressive. The regularization tweak for Deep Dream is very simple and quite different from what I mean by "training on visual art". (It's less surprising that a GAN trained on visual art can generate something that looks like visual art -- although it is surprising how well they can deal with stylized images.)
I agree there's great variety and intellectual sophistication in art. My paper argues that the Sensory Optimization model captures *some* (not all) key properties of visual art. The model is simple, easy to experiment with (e.g. generating art-like images), and captures a surprising amount. That said, there are probably simple computational models that could do better and I'd be excited to see concrete proposals.
The paper does touch on some of your concerns. Feature Visualization can generate non-representational images (Section 1.2). I suspect these images could be made more aesthetic and evocative by training on datasets with captions that include human emotional and aesthetic responses (Section 2.3), and the same goes for art that's strongly rooted in emotions (Section 2.3.3). Do you have examples in mind when you mention "human experience" and "embodiment" and "limited agents"? I don't really address art where the artist has different knowledge/understanding than the audience and that's an important topic for further work (Section 2.3.4 is related).
I agree that lots of art (including some painting) is "heavily linguistic, or social, or relies on ... thinking on the part of the audience". Having a computational model that can generate this kind of art is plausibly AGI-complete. Yet (as already noted) it's likely we can do better than my current model.
(In general, I’m optimistic about what neural nets can create by Sensory Optimization and related techniques. Current neural nets have zero experience of the physical act of painting or drawing. They have no understanding of how animals or humans move and act in the world or of human values or interests. Yet even with zero prior training on visual art they can make pretty impressive images by human lights. I think this was surprising to most people both in and outside deep learning. I'm curious whether this was surprising to you.)
Regarding your last paragraph, I want to make some clarifications. I don't express a view about whether Deep Dream makes art. I claim that by combining ideas from Deep Dream and Style Transfer with richer datasets we could create something close to a basic form of human visual art. I don't claim that the creative process for humans is like optimization by gradient descent. Instead, humans optimize by drawing on their general intelligence (e.g. hierarchical planning, analytical reasoning, etc.).
Even so, you'd hope people would notice that on the particular puzzle of the First Cause, saying "God!" doesn't help. It doesn't make the paradox seem any less paradoxical even if true. How could anyone not notice this?
Thinking well is difficult, even for great philosophers. Hindsight bias might skew our judgment here.
"About two years later, I became convinced that there is no life after
death, but I still believed in God, because the "First Cause" argument
appeared to be irrefutable. At the age of eighteen, however, shortly
before I went to Cambridge, I read Mill's Autobiography, where I found
a sentence to the effect that his father taught him the question "Who
made me?" cannot be answered, since it immediately suggests the
further question "Who made God?" This led me to abandon the "First
Cause" argument, and to become an atheist."
– Bertrand Russell, Autobiography of Bertrand Russell, Vol. 1, 1967.