LessWrong 2.0 Reader

It seems like you are failing to get my points at all. First, I am defending the point that blue LEDs are unworthy because the blue LED is not worthy of the award, but I corrected your claiming it was my example. Second, you are the only one making this about snubbing at all. I explicitly told you that I don't care about snubbing arguments. Comparisons are used for other reasons than snubbing. Third, since this isn't about snubbing, it doesn't matter at all whether or not the LED could have been given the award.

Makes sense. I agree that concern is important though not quite sure how to approach it.

Mostly IMO the bad outcomes in a concrete sense were PR and monetary concerns.

his AI girlfriend told him to

Which AI told him this? What exactly did it say? Had it undergone RLHF for ethics/harmlessness?

This is not to do with ethics though?

Air Canada Has to Honor a Refund Policy Its Chatbot Made Up

This is just the model hallucinating?

prevention of another Sydney.

But concretely, what bad outcomes eventuated because of Sydney?

Why would less RL on Ethics reduce productivity? Most work-use of AI has nothing to do with ethics.

In fact since RLHF decreases model capability AFAIK, would skipping this actually increase productivity because the models would be better?

918,367 kg

An average chimp is 45 kg

918,367 kg / 45 (kg / chimp)

= 20,408 chimps

You do mention the biggest issue with this showing introspection, "Models only exhibit introspection on simpler tasks", and yet the idea you are going for is clearly for its application to very complex tasks where we can't actually check its work. This flaw seems likely fatal, but who knows at this point? (The fact that GPT-4o and Llama 70B do better than GPT-3.5 does is evidence, but see my later problems with this...)

I addressed this point here [LW(p) · GW(p)]. Also see section 7.1.1 in the paper.

I have used HDBSCAN in a variety of instances in my data science career. The noise-aware aspect is definitely a mixed blessing. Often I find the best results come from using a variety of clustering algorithms, and figuring out how to do an ensemble of the results (e.g. treating the output of each clustering algorithm as a dimension in a similarity vector). Did you experiment with other clustering algorithms also?

Additionally, UMAP is outdated, please use PaCMAP instead: https://www.lesswrong.com/posts/C8LZ3DW697xcpPaqC/the-geometry-of-feelings-and-nonsense-in-large-language?commentId=Deddnyr7zJMwmNLBS [LW(p) · GW(p)]