What were the biggest discoveries / innovations in AI and ML?

post by elityre · 2020-01-06T07:42:11.048Z · score: 9 (4 votes) · LW · GW · No comments

This is a question post.


    27 jacobjacob
    6 FactorialCode
    4 elityre
    3 bgold
    3 elityre
    2 Björn Tegelund
    2 elityre
    2 elityre
    2 elityre
    2 elityre
No comments

These could be theoretical breakthroughs (like "the idea of a perceptron" or [something Judea Pearl did]), or they could be watershed developments / new applications that don't necessarily involve much new theory (like AlexNet or AlphaGo). Anything that seems like an important development in AI, is fair game.

I want an independently-generated list of all the interesting developments of AI, over the whole life of the fields, for a research project that I'm working on.

Feel free to include ones that you, personally, think were a big deal in some way, even if most people don't think so.



answer by jacobjacob · 2020-01-06T08:38:10.843Z · score: 27 (8 votes) · LW(p) · GW(p)

Check the section called "derivations" here: it links to a document attempting to list all conceptual breakthroughs in AI, of at least a certain significance, ever http://mediangroup.org/insights with related discussion on forecasting implications here: https://ai.metaculus.com/questions/2920/will-the-growth-rate-of-conceptual-ai-insights-remain-linear-in-2019/

comment by elityre · 2020-01-06T09:28:49.061Z · score: 3 (2 votes) · LW(p) · GW(p)

Ooo. Thank you!

answer by FactorialCode · 2020-01-06T16:22:14.363Z · score: 6 (4 votes) · LW(p) · GW(p)

-Reverse Mode Autodiff

-Using GPUs for computing.

These are the two big ones. Yes there are some others, but those two ideas together are the backbone of the current AI and ML boom.

answer by elityre · 2020-01-06T07:48:46.707Z · score: 4 (2 votes) · LW(p) · GW(p)

AlexNet in 2012. I'm not super clear on the details, but it seems to be the first time a deep neural net substantially outperformed other AI methods, and thereby kicked off the deep learning revolution.

answer by bgold · 2020-01-06T19:25:11.754Z · score: 3 (2 votes) · LW(p) · GW(p)

NeurIPS best paper awards will likely contain good leads.

answer by elityre · 2020-01-06T07:53:39.329Z · score: 3 (2 votes) · LW(p) · GW(p)

Deep Blue: a chess engine beats the reigning world chess champion, Gary Kasparov.

answer by Björn Tegelund · 2020-01-08T06:22:49.381Z · score: 2 (2 votes) · LW(p) · GW(p)

From A 'Brief' History of Neural Nets and Deep Learning, Part 4:

So, why indeed, did purely supervised learning with backpropagation not work well in the past? Geoffrey Hinton summarized the findings up to today in these four points:

  1. Our labeled datasets were thousands of times too small.
  2. Our computers were millions of times too slow.
  3. We initialized the weights in a stupid way.
  4. We used the wrong type of non-linearity.

I think this blog series might help provide a partial answer to your question.

answer by elityre · 2020-01-06T07:51:44.790Z · score: 2 (1 votes) · LW(p) · GW(p)


answer by elityre · 2020-01-06T07:46:26.182Z · score: 2 (1 votes) · LW(p) · GW(p)

Frank Rosenblatt develops the perceptron algorithm.

answer by elityre · 2020-01-06T07:46:05.418Z · score: 2 (1 votes) · LW(p) · GW(p)


comment by Pattern · 2020-01-06T18:33:11.275Z · score: 1 (1 votes) · LW(p) · GW(p)

It seems like this one should be combined with the one below it - AlphaGo.

answer by elityre · 2020-01-06T07:45:46.343Z · score: 2 (1 votes) · LW(p) · GW(p)


No comments

Comments sorted by top scores.