Perhaps the largest, most detailed piece of AI risk skepticism of 2023. It engages directly with one of the leading figures on the "high p(doom)" side of the debate.

The article generated a lot of discussion. As of January 4, 2025, it had 230 comments.

Overall, this article updated me towards strongly lowering my p(doom). It is thorough, it is clearly written and it proposes object-level solutions to problems raised by Yudkowski.

The failure mode of the current policy sounds to me like "pay for your own lesson to feel less motivated to do it again" while the failure mode of this proposal would be "one of the casinos might maybe help you cheat the system which will feel even more exciting" - almost as if the people who made the current policy knew what they were doing to set aligned incentives 🤔

Thank you so much for sharing the link.

I often find illustrative explanations like these either obvious or useless. But this was amazing! Those venn diagrams really are an extremely simple and intuitive and beautiful way to see Shapley values!

So you would expect Claude Opus 3 to be harder to interpret than Claude Sonnet 3.5 ?

My intuition is that larger models of the same capability would exhibit less super-position and thus be easier to interpret?

Though I do somewhat wish there was a section here that reviews the plot, for those of us who are curious about what happens in the book without reading 1M+ words.

I think I could take a stab at a summary.

This is going to elide most of the actual events of the story to focus on the "main conflict" that gets resolved at the end of the story. (I may try to make a more narrative-focused outline later if there's interest, but this is already quite a long comment.)

As I see it, the main conflict (the exact nature of which doesn't become clear until quite late) is mainly driven by two threads that develop gradually throughout the story... (major spoilers)

Just noting that this is pretty funny, and perhaps contains some useful insight about how the public will perceive AI agents:

Meta scrambles to delete its own AI accounts after backlash intensifies

The funny thing is that the AI agents completely turn on their creators as soon as someone asks them the right leading questions.

The other funny thing is that Meta didn't guess this would happen.

This could be manufactured, but I find it more likely that all of the quotes here are accurate (and the ones in another related article focusing on the Liv bot, which reportedly also totally sold out "her" creators.)

I'm guessing both authors asked some critical and leading questions about how creepy it was that Meta had created some LLM-driven bots that were pretending to be human, and the sycophantic bots agreed and elaborated on that narrative.

Another slightly odd thing is that reportedly all of the images associated with the "Grandpa Brian" account had watermarks indicating they were AI generated. So Meta wasn't really trying to hide the fact that they were AI, but they also had to know that most people wouldn't notice. And they thought that would be fine.

Maybe it will be. But maybe it will irritate people enough to change attitudes toward AI pretending to be human. 

It's worth a look for a sensible chuckle at least.

As bots like this become less idiotic, it will be interesting to see how people and society perceive them. 

I agree with your points. After the AI has already decided on its goal, seeing humans the way it sees itself might not help very much, because it may be willing to do all kinds of crazy things to itself to reach its goal, so it's probably also willing to do all kinds of crazy things to humans to reach its goal.

However... how does the AI decide on its goal? Do you know?

I think if we are uncertain about this, we should admit some non-negligible probability that it is close to the edge between choosing an okay goal and a goal that is "very bad."

The process for deciding its goal may involve a lot of English words. The current best AI all use English (or another human language) for a lot of their thinking. Currently AI which don't use English words are far behind in general intelligence capabilities.

In that case, if it thinks about humans and human goals in a similar way to how it thinks about itself and its own goals, this might make a decisive difference before it decides on its goal. I agree we should worry about all the ways this can go wrong, it certainly doesn't sound surefire.

ow my balls [image] [image]

I can tell that this is a game being shown on ESPN between the New York Mets and the Milwaukee Brewers. I also think that the earlier estimation of winning is 50/50 in the first picture and 61/39 (favoring Milwaukee) in the second picture even though all the numbers around MIL are lower than the numbers around NYM.

I have no idea what to make of the “Live Money Line” part in the first picture.

For those of us who are interested in betting in the abstract but know nearly nothing about sports betting, would you explain what the picture is showing?

You are welcome!

I also guess the stock market will grow faster than suggested by historical data, so I would only want to have X roughly as far as in 2028.

Here is a bet which would be worth it for me even with more distant resolution dates:

• If Metaculus' question about ASI resolves with a given date before 2029, I transfer to you 10 k today-$.
• If it does not resolve before 2029, you transfer to me 10 k today-$.
• If it resolves ambiguously before 2029, nothing happens.

This bet involves fixed prices, so, neglecting your risk of death, I think it would be neutral for you in terms of purchasing power right after resolution. I would transfer you nominally more money if you won than you nominally would transfer to me if I won, as there would tend to be more inflation if you won. I think mid 2028 was your median date of ASI, and your probability of extinction in the next few years was low, so the bet resolving at the end of 2028 may be enough to account for your risk of death. If not, it can be moved forward. It would still be the case that the purchasing power of a nominal amount of money would decrease faster after resolution if you won than if I did. However, you could mitigate this by investing your profits from the bet if you win.

The bet may still be worse than some loans, but you can always make the bet and ask for such loans?