LessWrong 2.0 Reader

You mean this substance? https://en.wikipedia.org/wiki/Mesembrine

Do you have a recommended brand, or places to read more about it?

Hyperbolic (like 1/x). I feel like you're hinting the answer is exponential, but that implies a constant doubling time, which isn't what we have here.

I'm voting against including this in the Review, at max level, because I think it too-often mischaracterizes the views of the people it quotes. And it seems real bad for a post that is mainly about describing other people's views and the drawing big conclusions from that data to inaccurately describe those views and then draw conclusions from inaccurate data.

I'd be interested in hearing about this from people who favor putting this post in the review. Did you check on the sources for some of Elizabeth's claims and think that she described them well? Did you see some inaccuracies but figure that the post is still good enough? Did you trust Elizabeth's descriptions without checking yourself on what the person said?

I spent a fair amount of time spot checking Elizabeth's first section, on Martin Soto, which got my attention because it seemed like it could be one of her strongest and it was the first. This claim from Elizabeth in that section seems clearly false: "The charitable explanation here is that my post focuses on naive veganism, and Soto thinks that’s a made-up problem". The first few paragraphs quoted in this post are sufficient to falsify this interpretation, and the first comment [LW(p) · GW(p)] that Martin left on Elizabeth's post is too. Other parts of the description of Martin's views which are more central to Elizabeth's argument also seem off, though sorting them out requires getting more in the weeds. e.g. AFAICT he didn't say he opposed talking about the whole topic of vegan nutrition; he did say something along the lines of 'you didn't say anything false, but I don't like the way you presented things because it'll have bad consequences', but that's a pretty normal type of opinion - Elizabeth said something like that [LW · GW] about Will MacAskill in another post in this series.

Other places where this post felt off include Elizabeth's description of what people were trying to claim when they brought up the Adventist study, and the claim that this comment [LW(p) · GW(p)] by Wilkox involved frame control (it doesn't look like Wilkox was trying to force their frame on the conversation; rather, it looks like Elizabeth brought a strong frame to the "Change my mind" post, Wilkox didn't immediately buy into it and was trying to think through the overall frame that Elizabeth brought and the specific concrete claims that Elizabeth made).

There are other examples in the comments, e.g. this comment [LW(p) · GW(p)] by Wilkox (currently at +12 net agree-vote, w/o a vote from me) gives 6 examples where the post's "description of what was said seems to misrepresent the source text", with some overlap with my examples and some I haven't looked into.

Before doing these spot checks I was inclined to vote against this post for the review at -1 because it didn't seem to live up to the title. It was trying to do a hard thing and didn't pull it off -- or at least, I didn't get a particularly clear sense of the nature and extent of epistemic problems within EA vegan advocacy and had just cached the post as 'Elizabeth's upset about EA vegan epistemics'. After digging in to some of it more closely, it looks like it did a worse job than I'd thought, so I've moved my vote downward and written this review.

I said "one of the best movies about", not "one of the best movies showing you how to".

Right. I actually don't worry much about the likely disastrous recession. I mostly worry that we will all die after a takeover from some sort of misaligned AGI. So I am doing - doing alignment research. I guess preparing to reap the rewards if things go well is a sensible response if you're not going to be able to contribute much to alignment research. I do hope you'll chip in on that effort!

Part of that effort is preventing related disasters like global recession contributing to political instability and resulting nuclear- or AGI-invented-even-worse-weapons wars; see my If we solve alignment, do we die anyway? [LW · GW].

I think preventing a global recession is probably possible and would also up the odds of us surviving. Making some money wont' keep you and yours alive if this all goes off the rails - which it very well might on the current trajectory. It's not a matter for optimism or pessimism, it's a matter for understanding and doing something about it before it happens.

If John Wentworth is correct about that being the biggest danger, making AI produce less slop would be the clear best path. I think it might be a good idea even if the dangers were split between misalignment of the first transformative AI, and it being adequately aligned but helping misalign the next generation.

From my comment on that post:

I'm curious why you think deceptive alignment from transformative AI is not much of a threat. I wonder if you're envisioning purely tool AI, or aligned agentic AGI that's just not smart enough to align better AGI?

I think it's quite implausible that we'll leave foundation models as tools rather than using the prompt "pretend you're an agent and call these tools" to turn them into agents. People want their work done for them, not just advice on how to do their work.

I do think it's quite plausible that we'll have aligned agentic foundation model agents that won't be quite smart enough to solve deeper alignment problems reliably, and sycophantic/clever enough to help researchers fool themselves into thinking they're solved. Since your last post to that effect it's become one of my leading routes to disaster. Thanks, I hate it.

OTOH, if that process is handled slightly better, it seems like we could get the help we need to solve alignment from early aligned LLM agent AGIs. This is valuable work on that risk model that could help steer [AI development] orgs away from likely mistakes and toward better practices.

Following on that logic, I think making our first transformative AI less prone to slop/errors is a good idea. The problem is that most such efforts probably speed up progress to getting there.

I'm starting to feel pretty sure that refusing to speed up progress and hoping we get enough time or a complete stallout is unrealistic. Accepting that we're on a terrifying trajectory and trying to steer it seems like the best response.

I think routes of reducing slop also contributes to aligning the first really competent LLM-based agents. One example is engineering such an agent to review its important decisions to see if they either make important errors or change/violate their central goals. I've written about that here [AF · GW] but I'm publishing an updated and expanded post soon.

So yes, I think this is probably somethings we should be doing. It's always going to be a judgment call of whether you publicize any particular idea. But there are more clever-to-brilliant people working on capabilities every day. Hoping they just won't have the same good ideas seems like a forlorn hope. Sharing the ones that seem to have more alignment relevance seems like it will probably differentially advance alignment over capabilities. 

(For the concrete scenario, feel free to stop at the point which is obviously unrecoverable and where AIs have misaligned goals, e.g. it can end with massive AI-runs companies maximizing profit (with media control and the ability to ostracize any member of the government because of sth that they secretly did 20 years ago that became deeply shameful because of cultural evolution), and whose shareholders are grandmas held in an eternal coma who technically become richer and richer but can never spend their money.)

If you de-slopify the models, how do you avoid people then using them to accelerate capabilities research just as much as safety research? Why wouldn't that leave us with the same gap in progress between the two we have right now, or even a worse gap? Except that everything would be moving to the finish line even faster, so Earth would have even less time to react.

Is the idea that it wouldn't help safety go differentially faster at all, but rather just that it may preempt people latching on to false slop-solutions for alignment as an additional source of confidence that racing ahead is fine? If that is the main payoff you envision, I don't think it'd be worth the downside of everything happening even faster. I think time is very precious, and sources of confidence already abound for those who go looking for them.

Here’s my grandmother’s borscht recipe and a recipe for “pasta navy style”.

Some relevant dessert recipes:

• medovik (honey cake)
• smetannik (sour cream cake)
• wafer torte
• птичье молоко (“bird’s milk cake”; actually an agar-based mousse on a cake base)