LessWrong 2.0 Reader

I'm unclear on what the distinction is exactly. This is a tutorial that works for catching a talented liar but also creating common knowledge between yourself and a bad liar.

Will you publish the curriculum online? :)

is it the bulleted list in the beginning? (fwiw i didn't put it there, it was my friends/editors who added that. (saying this because they don't have the financial incentive i may))

any suggestions for how to talk about this stuff without having it read like an advertisement? i'm genuinely interested in the idea of one-shotting and legibilizing evidence that quick growth is possible

I agree and I am building a "negotiable society of digital twins" at Kwaai based on this work.

Their definition of "Price gouging occurs in a competitive market when lowering the price from the market-clearing level would increase total Utilitarian welfare" is a bit sneaky: it means that any time I say "here's an example of where price gouging helps improve disaster response" they can just say "but that's not real price gouging, since a lower price wouldn't increase welfare".

It also doesn't look to me like the paper's approach gives a good framework for thinking about long-term investment incentives and preparation for future disasters, or people selling/renting possessions they wouldn't normally put on the market sell (air purifiers, renting spare rooms).

The paper's division of circumstances into price gouging vs not isn't a good match for the real world, and leads them to support policies like the current ones that normally don't do anything and then suddenly make large impacts in a disaster. Instead I'd like to see recognition that it's hard to determine welfare-maximizing pricing in real time and that price signals can reach very far, and instead use a mechanism that allows price increases to occur but redistributes some of the profits.

AI assistants are weird. Here's a Perplexity Pro search I did for an EY tweet about finding the sweet spot between utilitarianism & deontology. Perplexity Pro immediately found the correct tweet:

Eliezer Yudkowsky, a prominent figure in the rationalist community, has indeed expressed a view that suggests finding a balance between utilitarianism and deontology. In a tweet, he stated: "Go three-quarters of the way from deontology to utilitarianism and then stop. You are now in the right place. Stay there at least until you..."

But I wondered why it didn't provide the full quote (which is just a few more words, namely "Stay there at least until you have become a god."), and I just couldn't get it to do so, even with requests like "Just quote the full tweet from here: <URL>". Instead, it invented alternative versions like this:

Go three-quarters of the way from deontology to utilitarianism and then stop. You are now in the right place. Stay there at least until you understand why.

or this:

Go three-quarters of the way from deontology to utilitarianism and then stop. You are now in the correct place. Stay there at least until you understand why you shouldn't go any further.

I finally provided the full quote and asked it directly:

Does the following quote represent Yudkowsky's tweet with 100% accuracy?

"Go three-quarters of the way from deontology to utilitarianism and then stop. You are now in the right place. Stay there at least until you have become a god."

And it still doubled down on the wrong version.

We can build the Artificial Static Place Intelligence – instead of creating AI/AGI agents that are like librarians who only give you quotes from books and don’t let you enter the library itself to read the whole books. Why not expose the whole library – the entire multimodal language model – to real people, for example, in a computer game?

To make this place easier to visit and explore, we could make a digital copy of our planet Earth and somehow expose the contents of the multimodal language model to everyone in a familiar, user-friendly UI of our planet.

We should not keep it hidden behind the strict librarian (AI/AGI agent) that imposes rules on us to only read little quotes from books that it spits out while it itself has the whole output of humanity stolen.

We can explore The Library without any strict guardian in the comfort of our simulated planet Earth on our devices, in VR, and eventually through some wireless brain-computer interface (it would always remain a game that no one is forced to play, unlike the agentic AI-world that is being imposed on us more and more right now and potentially forever).

If you found it interesting, we discussed it here recently [EA · GW]

Do you have hope that someone else does the required research, so that it's ready by the time the first superbabies are created?

If not, do you think it's okay to create superintelligent babies without it?

I'm not actually relying on a heuristic, I'm compressing https://www.lesswrong.com/posts/vvgND6aLjuDR6QzDF/my-model-of-what-is-going-on-with-llms [LW · GW]

If you extrapolate capability graphs in the most straightforward way, you get the result that AGI should arrive around 2027-2028. Scenario analyses (like the ones produced by Kokotajlo and Aschenbrenner) tend to converge on the same result.

If you extrapolate log GDP growth or the value of the S&P 500, superintelligence would not be anticipated any time soon. If you extrapolate then number of open mathematical theorems proved by LLMs you get ~a constant at 0. You have to decide which straight line you expect to stay straight - what Aschenbrenner did is not objective, and I don't know about Kokotajlo but I doubt it was meaningfully independent.

 We mostly solved egg frying and laundry folding last year with Aloha and Optimus, which were some of the most long-standing issues in robotics. So human level robots in 2024 would actually have been an okay prediction. Actual human level probably requires human level intelligence, so 2027.

Interesting, link? 

This reasoning feels a little motivated though - I think it would be obvious if we had human(-laborer)-level robots because they'd be walking around doing stuff. I've worked in robotics research a little bit and I can tell you that setting up a demo for an isolated task is VERY different from selling a product that can do it, let alone one product that can seamlessly transition between many tasks.