LessWrong 2.0 Reader

Wednesday yes, sorry.

I procrastinated on this, but donated $200 today (the day after the fundraiser ended). I have been reading the site for 4 years, and it has been worth significantly more than $50 per year.

According to the site metrics there are around 15,000 logged-in monthly active users, so if everyone donated $200 that would exactly equal the $3,000,000 target. This made $200 seem like a reasonable number. 

Yes, and (for certain mainstream interpretations) nothing in quantum mechanics is probabilistic at all: the only uncertainty is indexical.

on extensions: note you can use ublock origin^[1] to stop sites from loading by setting a filter against their main elements^[2]. this is preferable to minimize the amount of extensions which can read site data^[3], at least if you already use ublock.

1. ^{^}

   https://github.com/gorhill/uBlock/wiki/uBlock-Origin-is-completely-unrelated-to-the-web-site-ublock.org

2. ^{^}

   (for example, now if i go to reddit, only the header bar loads and the rest is white; if i go to youtube, no video panes load, so i can only open videos found elsewhere, like by searching for a song and clicking the link)

3. ^{^}

   i remember finding an extension which claimed to block some individual site, but tried to request read access on all sites

I had a vaguely favorable reaction to this post when it was first posted.

When I wrote my recent post on corrigibility [LW · GW], I grew increasingly concerned about the possible conflicts between goals learned during pretraining and goals that are introduced later. That caused me to remember this post, and decide it felt more important now than it did before.

I'll estimate a 1 in 5000 chance that the general ideas in this post turn out to be necessary for humans to flourish.

Thanks very much! Yeah, I agree political will seems like a big issue. But I also hear people saying that they don't know what to push for, so I wanted to try to offer a concrete example of a system that wasn't as destructive to any constituency's interests as e.g. a total pause.

Stable is not a virtue, nor is our equilibrium well-tolerated. The problems it causes in terms of health, cost and homelessness are central political issues and have been for a long time.

I also have no idea why you assume I’m “ignoring” these “lessons” you’re handwaving at. It’s a pretty annoying rhetorical move.

These three prompts are very cherry-picked. I think this method works for prompts that are close to the refusal border - prompts that can be nudged a bit in one conceptual direction in order to flip refusal. (And even then, I think it is pretty sensitive to phrasing.) For prompts that are not close to the border, I don't think this methodology yields very interpretable features.

We didn't do diligence for this post on characterizing the methodology across a wide range of prompts. I think this seems like a good thing to investigate properly. I expect there to be a nice way of characterizing a "borderline" prompt (e.g. large magnitude refusal gradient, perhaps).

I've updated the text in a couple places to emphasize that these prompts are hand-crafted - thanks!

Darn, exactly the project I was hoping to do at MATS! :-)

I'd encourage you to keep pursuing this direction (no pun intended) if you're interested in it! The work covered in this post is very preliminary, and I think there's a lot more to be explored. Feel free to reach out, would be happy to coordinate!

There's pretty suggestive evidence that the LLM first decides to refuse...

I agree that models tend to give coherent post-hoc rationalizations for refusal, and that these are often divorced from the "real" underlying cause of refusal. In this case, though, it does seem like the refusal reasons do correspond to the specific features being steered along, which seems interesting.

Looking through Latent 2213,...

Seems right, nice!

The Pile was created from Reddit datasets between 2005 and 2020. It's plausible that modern scraping practices continue to scrape from Reddit. Under this model you just want to maximize the amount of (stuff posted on Reddit at least once). Multiple copies doesn't help since the Pile is subsequently de-duped