Deep limitations? Examining expert disagreement over deep learning

Interviews were conducted in 2019 to early 2020

For context, this timing implies that all of these are pre-GPT-3/CLIP/DALL-E/MLP-Mixer & my scaling-hypothesis writeup, possibly pre-MuZero and much of the recent planning/model-based DRL work (eg MuZero Unplugged, Decision Transformer), and pre much of the Distill.pub Circuits work & the semi-supervised revolution. Already some of the quotes are endearingly obsolete:

“[For instance] using convolutional attention mechanisms and applying it to graphs structures and training to learn how to represent code by training it on GitHub corpora…that kind of incremental progress would carry us to [...] superintelligence.” (P21).

(Convolutions? OK grandpa. But he's right that the program synthesis Transformers trained on Github are pretty sweet.*) Unfortunately still contemporary are the pessimistic quotes:

“Those people who say that’s going to continue are saying it as more of a form of religion. It’s blind faith unsupported by facts. But if you have studied cognition, if you have studied the properties of language… [...] you recognise that there are many things that deep learning [...] right now isn’t doing.” (P23).

“My hunch is that deep learning isn’t going anywhere. It has very good solutions for problems where you have large amounts of labelled data, and fairly well-defined tasks and lots of compute thrown at problems. This doesn’t describe many tasks we care about.” (P10).

“If you think you can build the solution even if you don’t know what the problem is, you probably think you can do AI” (P2).

I assume if interviewed now, they'd say the same things but even more loudly and angrily - the typical pessimism masquerading as intellectual seriousness.

* I wrote this before OA/GH Copilot, but it makes the point even more strongly.

Planned summary for the Alignment Newsletter:

This paper reports on the results of a qualitative survey of 25 experts conducted in 2019 and early 2020, on the possibility of deep learning leading to high-level machine intelligence (HLMI), defined here as an “algorithmic system that performs like average adults on cognitive tests that evaluate the cognitive abilities required to perform economically relevant tasks”. Experts disagreed strongly on whether deep learning could lead to HLMI. Optimists tended to focus on the importance of scale, while pessimists tended to emphasize the need for additional insights.

Based on the interviews, the paper gives a list of 40 limitations of deep learning that some expert pointed to, and a more specific list of five areas that both optimists and pessimists pointed to as in support of their views (and thus would likely be promising areas to resolve disagreements). The five areas are (1) abstraction, (2) generalization, (3) explanatory, causal models, (4) emergence of planning, and (5) intervention.

Came here from this thread about whether ML experts can readily provide examples of barriers between today and AGI. What a blast from the past. Seems like most of the alleged barriers that ML experts cited in this 2020 survey have already been smashed through in 2023. (I'm looking at Table 1, which contains things like "Insufficient ability to construct and decompose sentences according to grammatical rules.")

Here's my opinions on what deep learning can do, FWIW - 
1 (abstraction) yes, but they aren't sample efficient ! 
2. (generalization) eh, not if you define generalization as going out of distribution (note: that's not how it's normally defined in ML literature).  Deep learning systems can barely generalize outside their training data distribution at all. The one exception I know is how GPT-3 learned addition but even then it broke down at large numbers. Some GPT-3 generalization failures can be seen here.
3. (causality) maybe?  
4. (long term planning) - I think DL can do this, but maybe not necessarily using the same kind of hierarchical planning framework that humans seem to use
5.  (need for intervention) - is this getting at embodiment? I'm a bit unclear. In any case, it doesn't seem like this is real critical to me.

So for me, the main issues I have with deep learning are the low sample efficiency and lack of generalization ability. Which is why I'm skeptical that just scaling up the deep learning methods we have now can lead to true AGI, although it might get to something which is for many practical purposes pretty close.  

3: No, that will never work with DL by itself (e.g. as fancy regressions).

4: No, that will never work with DL by itself (e.g. as fancy regressions).

5: I don't understand this question, but people already use DL for RL, so the "support" part is already true.  If the question is asking whether DL can substitute for doing interventions, then the answer is a very qualified "yes," but the secret sauce isn't DL, it's other things (e.g. causal inference) that use DL as a subroutine.

---

The problem is, most folks who aren't doing data science for a living themselves only view data science advances through the vein of hype, fashion trends, and press releases, and so get an entirely wrong sense of what is truly groundbreaking and important.