LessWrong 2.0 Reader

My best guess is you are a few months behind in your takes? The latest generation of thinking models can definitely do agentic frontend development and build small projects autonomously. It definitely still makes errors, and has blindspots that require human supervision, but in terms of skill level, the systems feel definitely comparable and usually superior to junior programmers (but when they fail, they fail hard and worse than real junior programmers).

Do you really think p(everyone dies) is >99%?

We have not yet tried 4.5 as it's so expensive that we would not be able to deploy it, even for limited sections.

Still seems like potentially valuable information to know: how much does small-model smell cost you? What happens if you ablate reasoning? If it is factual knowledge and GPT-4.5 performs much better, then that tells you things like 'maybe finetuning is more useful than we think', etc. If you are already set up to benchmark all these OA models, then a datapoint from GPT-4.5 should be quite easy and just a matter of a small amount of chump change in comparison to the insight, like a few hundred bucks.

I agree that birth order inversely correlating with capability is the most plausible resolution of this puzzle, though I need to check the effect sizes more studiously, and am still haunted by the ghost of Judith Rich Harris. 

As for the traits being selected, we obviously don't know, though the idea is that selecting for homosexuality gifts the selectors an obvious manner of control of whomever makes it into the college of cardinals. 

Ah ok. For the record I was referring to tetraphobia.

No worries, I appreciate the concept and think some aspects of it are useful. I do worry at a vibes level that if we're not precise about which human-child-rearing methods we expect to be useful for AI training, and why, we're likely to be misled by warm fuzzy feelings.

And yes, that's true about some (maybe many) humans' vengeful and vindictive and otherwise harmful tendencies. A human-like LLM could easily be a source of x-risk, and from humans we already know that human child rearing and training and socializing methods are not universally effective at addressing this. Among humans, we have so far been successful at not putting anyone who would destroy the world in the position of being able to do so at the time when they would choose to.

As for generational perspectives: this is a useful heuristic among humans. It is not automatic or universal. Not every perspective is worthy of respect, not on every issue. Some ought to be abandoned or condemned in the light of information or reasoning that wasn't/isn't available or accessible in other places and times. Some should be respected but only with many caveats. Having your perspective respected is earned. We assume among humans that we should try to respect the perspectives of adults, and sometimes must disabuse ourselves of this in particular cases, but it is pure convention because most humans at a certain age are mature enough for it to be a useful default. I do not have anything like strong reasons to apply this heuristic to LLMs as they currently exist.

I shot out an email, sadly, I've seen this about a month too late! If opportunities exist for staffers/applicants post-deadline, I'd be interested :)

22.3 percent of cardinals are reported as eldest children. That compares to 21.6 percent which are youngest children. Eldest children are still favored.

I kept waiting for you to discuss this point: in considering analysis of cardinals (as opposed to ordinary random people), what about the other relevant birth-order effects? Like the... first-born eldest birth order effect, where first-borns are smarter, more extraverted, stabler, higher-SES etc. All of which sounds exactly like the sort of thing you need to rise through an extreme hierarchy to the top.

After all, surely homosexuality is not the primary trait the Catholic Church hierarchy is trying to select for?

>Blackwell is a one-time thing that essentially fixes a bug in Ampere/Hopper design (in efficiency for LLM inference)

Wait, I feel I have my ear pretty close to the ground as far as hardware is concerned, and I don't know what you mean by this?

Supporting 4-bit datatypes within tensor units seems unlikely to be the end of the road, as exponentiation seems most efficient at factor of 3 for many things, and presumably nets will find their eventual optimal equilibrium somewhere around 2 bits/parameter (explicit trinary seems too messy to retrofit on existing gpu paradigms).

Was there some "bug" with the hardware scheduler or some low level memory system, or perhaps an issue with the sparsity implementation that I was unaware off? There were of course general refinements across the board for memory architecture, but nothing I'd consider groundbreaking enough to call it "fixing a bug". 

I reskimmed through hopper/blackwell whitepapers and LLM/DR queried and really not sure what you are referring to. If anything, there appear to be some rough edges introduced with NV-HBI and relying on a virtualized monolithic gpu in code vs the 2x die underlying. Or are you perhaps arguing that going MCM and beating the reticle limit was itself the one time thing? 
 

Ok can we put some rationality to this. Your prior seems to be that when a field is stuck, it is almost entirely because of politics, hegemon etc. So how do we update that prior?

What would you expect to see in a field that is not stuck bc of inertia etc. You would still get people like Hossenfelder saying we are doing things wrong, and such people would get followers.

You suggested metrics before but havn't provided any that I can see.

Evidence I take into account:

#1 There is not a significant group of young capable researchers saying we are taking things in the wrong direction, but a smaller number of older ones. Unless you are going to go so far as to say they are afraid to speak out, even anon, then that to me is evidence against your position. There are two capable experts on this blog from what I can see, one enthusiastic about string theory, and another who has investigated other theories in detail. Both disagree with your claim.

#2. There is not broad disagreement about what the major issues in physics are, so unlikely to be disagreements on metrics. You mentioned this as mattering, and if it does, I count this as evidence against your position.

Can you point to evidence that actually supports your prior? I can only see that which opposes it or is neutral. (In all fields no matter how things are progressing you get some people who think the establishment is doing it all wrong and have their own ideas. That can't count as evidence in favor for a specific field unless it is happening to a greater degree)

I don't disagree with your position in general. In fact it was clear to me that AI before neural networks was stuck with GOFAI and I believed that NN were clearly the way to go. I followed Hinton et all before they became famous. I saw the paradigm thing play out in front of my eyes there. Physics seems different.