LessWrong 2.0 Reader

I think not, because in my test the snippet didn't really contain such a quote that would have answered the question directly.

Since '23 my answer to that question would have been "well the first step is for researchers like you to produce [basically exactly the paper OpenAI just produced]"

So that's done. Nice. There are lots of follow-up experiments that can be done. 

I don't think trying to shift the market/consumers as a whole is very tractable.

But talking to your friends at the companies, getting their buy-in, seems valuable. 

This work was co-authored by Jordan Stone, Darryl Wright, and Youssef Saleh, whose names appear on the EA Forum post but not on this cross post to LW.
 

They are now open for trading:

Will the company that produces the first AGI have prioritized Corrigibility?.

Will AGI create a consensus among experts on how to safely increase AI capabilities?.

Will prioritizing corrigible AI produce safe results?.

Yes, the privacy constraints make the implications of these improvements less legible to the public. We have multiple plans for how to disseminate info within this constraint, such as publishing author survey comments regarding the reasoning traces and our competition at the end of the month to establish a sort of human baseline.

Still, I don't know that the privacy of FrontierMath is worth all the roundabout efforts we must engage in to explain it. For future projects, I would be interested in other approaches to balancing preventing models from training on public discussion of problems vs being able to clearly show the world what the models are tackling. Maybe it would be feasible to do IMO-style releases? "Here's 30 new problems we collected this month. We will immediately test all the models and then make the problems public."

Do you have a sense of what I, as a researcher, could do?

I sense that having users/companies want faithful CoT is very important.. In-tune users, as. nostalgebraist points out, will know how to use CoTs to debug LLMs. But I'm not sure whether this represents only 1% of users, so big labs just won't care. Maybe we need to try and educate more users about this. Maybe reach out to people who tweet about LLM best use cases to highlight this?

Obvious point - I think a lot of this comes from the financial incentives. The more "out of the box" you go, the less sure you can be that there will be funding for your work.

Some of those that do this will be rewarded, but I suspect many won't be.

As such, I think that funders can help more to encourage this sort of thing, if they want to. 

We got spam and had to reset the link. To get the new link, append the suffix "BbILI8HzX3zgJF8i" to the prefix "https://chat.whatsapp.com/IUIZc3". Hopefully spambots can't yet do that automatically.

Interesting. Borderline or PTSD rather than cyclothymia?
 

I don't disagree that's where a standard clinical interview would end up, but aren't these basically residual categories where to put people who aren't sane but don't clearly fit any of the other boxes? Like, not false, but it doesn't exactly constrain the space of where that weird outlier mind of his might be going next.

I'd be very interested in what would happen if he couldn't have his phone for a week.

If you have people in eastern canada, I have direct LE there and can maybe help you.