LessWrong 2.0 Reader

I think that if you take into account all the improvements to transformers published since their initial invention in the 2010s, that there is well over 1000x worth of improvement.

I can list a few of these advancements off the top of my head, but a comprehensive list would be a substantial project to assemble.

Data Selection: 

• DeepMind JEST: https://arxiv.org/abs/2406.17711
• SoftDedup: https://arxiv.org/abs/2407.06654 

Activation function improvements, e.g. SwiGLU

FlashAttention: https://arxiv.org/abs/2205.14135 

GrokFast: https://arxiv.org/html/2405.20233v2 

AdEMAMix Optimizer https://arxiv.org/html/2409.03137v1 

Quantized training

Better parallelism

DPO https://arxiv.org/abs/2305.18290 

Hypersphere embedding https://arxiv.org/abs/2410.01131 

Binary Tree MoEs: https://arxiv.org/abs/2311.10770  https://arxiv.org/abs/2407.04153 

And a bunch of stuff in-the-works that may or may not pan out:

• hybrid attention and state-space models (e.g. mixing in some Mamba layers)
• multi-token prediction (including w potentially diffusion model guidance): https://arxiv.org/abs/2404.19737   https://arxiv.org/abs/2310.16834 

Here's a survey article with a bunch of further links: https://arxiv.org/abs/2302.01107 

But that's just in response to defending the point that there has been at least 1000x of improvement. My expectations of substantial improvement yet to come are based not just on this historical pattern, but also on reasoning about a potential for an 'innovation overhang' of valuable knowledge that can be gleaned from interpolating between existing research papers (something LLMs will likely soon be good enough for), and also from reasoning from my neuroscience background and some specific estimates of various parts of the brain in terms of compute efficiency and learning rates compared to models which do equivalent things. 

Yep, definitely a bug. Should be fixed soon.

Bug: I can no longer see the number of agreement-votes (which is distinct from the number of Karma-votes). It shows the Agreement Downvote tooltip when hovering over the agreement score (the same for Karma score works correctly, saying for example "This comment has 31 overall karma (17 Votes)").

Edit: The number of agreement votes can be seen when hovering over two narrow strips, probably 1 pixel high, one right above and one right below the agreement rating.

Interesting stuff! I've been dabbling in some similar things, talking with AE Studio folks about LLM consciousness.

but have so far only found relatively incremental improvements to transformers (in the realm of 1000x improvement)

What 1000x improvement? Better hardware and larger scale are not algorithmic improvements. Careful study of scaling laws to get Chinchilla scaling and set tokens per parameter more reasonably^[1] is not an algorithmic improvement. There was maybe 5x-20x algorithmic improvement, meaning the compute multiplier, how much less compute one would need to get the same perplexity on some test data. The upper bound is speculation based on published research for which there are no public results of large scale experiments, including for combinations of multiple methods, and absence of very strong compute multiplier results from developers of open weights models who publish detailed reports like DeepSeek and Meta. The lower bound can be observed in the Mamba paper (Figure 4, Transformer vs. Transformer++), though it doesn't test MoE over dense transformer (which should be a further 2x or so, but I still don't know of a paper that demonstrates this clearly).

Recent Yi-Lightning is an interesting example that wins on Chatbot Arena in multiple categories over all but a few of the strongest frontier GPT-4 level models [LW(p) · GW(p)] (original GPT-4 itself is far behind). It was trained for about 2e24 FLOPs [LW(p) · GW(p)], 10x less than original GPT-4, and it's a small overtrained model, so its tokens per parameter are very unfavorable, that is it was possible to make it even more capable with the same compute.

You may want higher density, but I don't think you can say that I want high density at the expense of legibility.

It takes a lot to make me notice layout, and I rarely notice fonts at all... unless they're too small. I'm not as young as I used to be. This change made me think I must have zoomed the browser two sizes smaller. The size contrast is so massive that I have to actually zoom the page to read comfortably when I get to the comment section. It's noticeably annoying, to the point of breaking concentration.

I've mostly switched to RSS for Less Wrong^[1]. I don't see your fonts at all any more, unless I click through on an article. The usual reason I click through is to read the comments (occasionally to check out the quick takes and popular comments that don't show up on RSS). So the comments being inaccessible is doubly bad.

My browser is Firefox on Fedora Linux, and I use a 40 inch 4K monitor (most of whose real estate is wasted by almost every Web site). I usually install most of the available font packages, and it says it's rendering this text in "Gill Sans Nova Medium".

Thanks for the offer! DMed you. We shot with:
- Camera A (wide shot):  FX3
- Camera B, C: FX30

From what I have read online, the FX30 is not "Netflix-approved" but it won't matter (for distribution) because "it only applies to Netflix produced productions and was really just based on some tech specs to they could market their 4k original content." (link). Basically, if the film has not been commissioned by Netflix, you do not have to satisfy these requirements. (link)

And even for Netflix originals (which won't be the case here), they're actually more flexible on their camera requirements for nonfiction work such as documentaries (they used to have a 80% on camera-approved threshold which they removed).

For our particular documentary, which is primarily interview-based in controlled lighting conditions, the FX30 and FX3 produce virtually identical image quality.

K complexity is separable into two components, one corresponding to randomness, and the other corresponding to deterministic structure

Corresponds with how bayesian Reinforcement Learning models, the randomness component corresponds with minimizing aleatoric uncertainty while the epistemic component corresponds with minimizing epistemic uncertainty.

The point is that the "controller" of a "controllable AI" is a role that can be filled by an AI and not only by a human or a human institution. AI is going to quickly grow the pie to the extent that makes current industry and economy (controlled by humans) a rounding error, so it seems unlikely that among the entities vying for control over controllable AIs, humans and human institutions are going to be worth mentioning. It's not even about a takeover, Google didn't take over Gambia.

Another important one: Height/Altitude is authority. Your boss is "above" you, the king, president or CEO is "at the top", you "climb the corporate ladder"