LessWrong 2.0 Reader

Hmmm, my long term strategy is to build wealth and then do it myself, but I suppose that would require me to leave academia eventually :)

I wonder if MIRI would fund it? Doesn't seem likely.

Haha, well that dosage probably would probably cause weight loss.

All of the sources I can find give the density as exactly 4 oz = 1/2 cup, although maybe this is just an approximation that's infecting other data sources?

https://www.wolframalpha.com/input?i=density+of+butter+*+(1%2F2+cup)+in+ounces

But 1/2 cup of butter weighs 4 ounces according to every source I can find: https://www.wolframalpha.com/input?i=density+of+butter+*+(1%2F2+cup)+in+ounces

Which means a 4 ounce stick of butter is 1/2 cup by volume.

How much longer did your timelines get?

Yes! Very exciting stuff. Twas an update towards longer timelines for me.

Why downvote? you can tell me anonymously:

https://docs.google.com/forms/d/e/1FAIpQLSca6NOTbFMU9BBQBYHecUfjPsxhGbzzlFO5BNNR1AIXZjpvcw/viewform

to do a do a

There seem to by a typo here :)

Do we want minecraft alignment evals?

My main pitch:

There were recently some funny examples of LLMs playing minecraft and, for example, 

1. The player asks for wood, so the AI chops down the player's tree house because it's made of wood
2. The player asks for help keeping safe, so the AI tries surrounding the player with walls

This seems interesting because minecraft doesn't have a clear win condition, so unlike chess, there's a difference between minecraft-capabilities and minecraft-alignment. So we could take an AI, apply some alignment technique (for example, RLHF), let it play minecraft with humans (which is hopefully out of distribution compared to the AI's training), and observe whether the minecraft-world is still fun to play or if it's known that asking the AI for something (like getting gold) makes it sort of take over the world and break everything else.

Or it could teach us something else like "you must define for the AI which exact boundaries to act in, and then it's safe and useful, so if we can do something like that for real-world AGI we'll be fine, but we don't have any other solution that works yet". Or maybe "the AI needs 1000 examples for things it did that we did/didn't like, which would make it friendly in the distribution of those examples, but it's known to do weird things [like chopping down our tree house] without those examples or if the examples are only from the forest but then we go to the desert"

I have more to say about this, but the question that seems most important is "would results from such an experiment potentially change your mind":

1. If there's an alignment technique you believe in and it would totally fail to make a minecraft server be fun when playing with an AI, would you significantly update towards "that alignment technique isn't enough"?
2. If you don't believe in some alignment technique but it proves to work here, allowing the AI to generalize what humans want out of its training distribution (similarly to how a human that plays minecraft for the first time will know not to chop down your tree house), would that make you believe in that alignment technique way more and be much more optimistic about superhuman AI going well?

Assume the AI is smart enough to be vastly superhuman at minecraft, and that it has too many thoughts for a human to reasonably follow (unless the human is using something like "scalable oversight" successfully. that's one of the alignment techniques we could test if we wanted to)

When you write,

a draft research paper or proposal that frames your ideas into a structured and usable format,

who is "you"?