The 27 papers
post by EZ97 · 2024-05-30T08:46:39.587Z · LW · GW · 2 commentsContents
2 comments
List of 27 papers (supposedly) given to John Carmack by Ilya Sutskever: "If you really learn all of these, you’ll know 90% of what matters today."
The list has been floating around for a few weeks on Twitter/LinkedIn. I figure some might have missed it so here you go.
Regardless of the veracity of the tale I am still finding it valuable.
https://punkx.org/jackdoe/30.html
- The Annotated Transformer (nlp.seas.harvard.edu)
- The First Law of Complexodynamics (scottaaronson.blog)
- The Unreasonable Effectiveness of RNNs (karpathy.github.io)
- Understanding LSTM Networks (colah.github.io)
- Recurrent Neural Network Regularization (arxiv.org)
- Keeping Neural Networks Simple by Minimizing the Description Length of the Weights (cs.toronto.edu)
- Pointer Networks (arxiv.org)
- ImageNet Classification with Deep CNNs (proceedings.neurips.cc)
- Order Matters: Sequence to sequence for sets (arxiv.org)
- GPipe: Efficient Training of Giant Neural Networks using Pipeline Parallelism (arxiv.org)
- Deep Residual Learning for Image Recognition (arxiv.org)
- Multi-Scale Context Aggregation by Dilated Convolutions (arxiv.org)
- Neural Quantum Chemistry (arxiv.org)
- Attention Is All You Need (arxiv.org)
- Neural Machine Translation by Jointly Learning to Align and Translate (arxiv.org)
- Identity Mappings in Deep Residual Networks (arxiv.org)
- A Simple NN Module for Relational Reasoning (arxiv.org)
- Variational Lossy Autoencoder (arxiv.org)
- Relational RNNs (arxiv.org)
- Quantifying the Rise and Fall of Complexity in Closed Systems: The Coffee Automaton (arxiv.org)
- Neural Turing Machines (arxiv.org)
- Deep Speech 2: End-to-End Speech Recognition in English and Mandarin (arxiv.org)
- Scaling Laws for Neural LMs (arxiv.org)
- A Tutorial Introduction to the Minimum Description Length Principle (arxiv.org)
- Machine Super Intelligence Dissertation (vetta.org)
- PAGE 434 onwards: Komogrov Complexity (lirmm.fr)
- CS231n Convolutional Neural Networks for Visual Recognition (cs231n.github.io)
2 comments
Comments sorted by top scores.
comment by Amalthea (nikolas-kuhn) · 2024-05-30T09:34:08.043Z · LW(p) · GW(p)
Might be good to estimate the date of the recommendation - as the interview where Carmack mentioned this was in 2023, a rough guess might be 2021/22?
comment by metachirality · 2024-05-30T23:53:10.738Z · LW(p) · GW(p)
I like this format and framing of "90% of what matters" and someone should try doing it with other subjects.