My thoughts on the Beff Jezos - Connor Leahy debate

I had watched the whole thing and came away with a very different impression. From where I'm standing, Connor is just correct about everything he said, full stop. Beff made a few interesting points but was mostly incoherent, equivocating, and/or evasive. Connor tried very hard for hours to go for his cruxes [? · GW] rather than get lost in the weeds, but Beff wouldn't let him. Maybe Connor could have called him on it more skillfully, but I don't think I could have done any better. Maybe he'll try a different tack if there's a next time. The moderator really should have intervened.

At some point they start building their respective cases - what if you had a false vacuum device? Would we be fucked? Should we hide it? What should we do? And on Beff's side - what if there are dangerous aliens?

For the love of god, please talk about the actual topic.

This is the actual topic. It's the Black Marble [? · GW] thought experiment by Bostrom, and the crux of the whole disagreement! Later on Connor called it rolling death on the dice. Non-ergodicity. Beff's whole position seems to be to redefine "the good" to be "acceleration of growth", but Connor wants to add "not when it kills you!"

About 50 minutes in, Connor goes on an offensive in a way that, to me is an extremely blatant slippery slope reasoning. The main point is that if you care about growth, you cannot care about anything else, because of course everyone's views are the extremist parodies of themselves. Embarrassing tbh.

Again, Connor is simply correct here. This is not a novel argument. It's Goodhart's Law. [? · GW] You get what you optimize, even if it's only a proxy for what you want. The tails come apart [LW · GW]. You can overshoot and get your proxy rather than your target. Remember, Beff's position: "growth = good", which is obviously (to me, Connor, and Eliezer) false. Connor tried very hard to lead Beff to see why, but Beff was more interested in muddying the waters than achieving clarity or finding cruxes.

He also points out, many many times, that "is" != "ought", which felt like virtue signalling? Throwing around shibboleths? Not quite sure. But not once was it a good argument as far as I can tell.

Again, Connor is simply correct. This isn't about virtue signaling at all; that completely misses the point. Beff is equivocating. Connor is trying to point out the distinct definitions required to separate the concepts so he can move the argument forward to the next step. Beff just wasn't listening.

"Should the blueprints for F16 be open-sourced? Answer the question. Answer the question! Oh I was just trying to probe your intuition, I wasn't making a point"

Immediately followed by "If an AI could design an F16, should it be open-sourced?"

Is there something wrong with trying to understand the other position before making a point? No, and Beff should have tried harder to understand the other position. Kudos to Connor for trying. This is the Black Marble again (maybe a gray one in this case). Beff seems to have the naiive position that open source is an unmitigated good, which is obviously (to me and Connor) false, because infohazards [? · GW]. I don't think F16s were a great example, but it could have been any number of other things.

So e/acc should want to collapse the false vacuum?

Holy mother of bad faith. Rationalists/lesswrongers have a problem with saying obviously false things, and this is one of those.

Totally unfair characterization. I think this is Connor simply not understanding Beff's position, rather than Connor doing anything underhanded. The question was not simply rhetorical, and the answer was important for updating Connor's understanding (of Beff's position). From Connor's point of view, an intelligence explosion eats most of the future light cone anyway, so it's not that different from a false vacuum collapse: everybody dies, and the future has no value. There are some philosophies that actually bite the bullet to remain consistent in the limit and actually want all humans to die. (Nick Land came up.) Connor thinks Beff's might be that on reflection, but it's not for the reason Connor thought here.

It's in line with what seems like Connor's debate strategy - make your opponent define their views and their terminal goal in words, and then pick apart that goal by pushing it to the maximum. Embarrassing.

Again, this is what Eliezer, Connor, and I think is the obvious thing that would happen once an unaligned superintelligence exists: it pushes its goals to the limit at the expense of all we value. This is not Connor being unfair; this is literally his position.

Libertarians are like house cats, fully dependent on a system they neither fully understand nor appreciate.

Thanks for that virtue signal, very valuable to the conversation.

OK, maybe that's a signal (it's certainly a quip), but the point is valid, stands, and Connor is correct. I am sympathetic to the libertarian philosophy, but the naiive application is incomplete and cannot stand on its own.

After about 2 hours and 40 minutes of the "debate", it seems we finally got to the point!

Finally? Connor has been talking about this the whole time. Black marble!

If I were to respond to this myself, I'd say - at some point, depending how technology progresses, we might very well need to pause, slow down, or stop entirely.

Yep. That was yesterday. Connor would be interested in talking all about why he thinks that and (as evidenced by the next quote) wants to know Beff's criteria for when that point is, so Connor can move on and either explain why that point has already passed, or point out that Beff doesn't have any criteria and will just go ahead and draw the black marble without even trying to prepare for it. (Which means everybody dies.)

To which Connor has another one of the worst debate arguments ever: "So when is the right time? When do we know?"

Connor is correctly making a very legit point here. There are no do-overs. If you draw the black marble before you're prepared for it, then everybody dies. If you refuse to even think about how to prepare for it and not only keep drawing marbles but try to draw them faster and faster, then by default you die, and sooner and sooner! This is not unfair and this is not a bad argument. This is legitimately Connor's position (and mine and Bostrom's).

I don't know when is the right time to stop overpopulation on Mars.

That is a very old, very bad argument. If NASA discovered a comet big and fast enough to cause a mass extinction event that they estimated to have a 10% chance of colliding with Earth in 100 years, we shouldn't start worrying about it until it's about to hit us. Right? Or from the glass-half-full perspective, we've got a 90% chance of surviving anyway, so let's just forget about the whole thing. Right? Do you understand how absurd that sounds?

But Connor (and Eliezer and I (and Hinton)) don't think we have 100 years. We think it's probably decades or less, maybe much less. And Connor (and Eliezer and I) don't think we have a 90% chance of surviving by default. Quite the reverse, or even worse.

In response, Connor resorts to yelling that "You don't have a plan!"

No shit. Not only that, but e/acc seems to be trying very hard to make the problem worse, by giving us even less time to prepare and sabotaging efforts to buy more.

This is the point where we should move on to narrowing down why we need to have a plan for overpopulation on Mars right now. Perhaps we do.

Yes. That would have been good. I could tell Connor was really trying to get there. Beff wasn't listening though.

This was largely a display of tribal posturing via two people talking past each other.

Maybe describes Beff. Connor tried. Could've been better, but we have to start somewhere. Maybe they'll learn from their mistakes and try again.

Poor performance from both of them, but particularly Connor's behavior is seriously embarrassing to the AI safety movement.

I was embarrassed by Connor's headshot comment, which I thought was inappropriate. Thought experiments that could be interpreted as veiled death threats against one's interlocutor are just plain rude. Could have been worded differently. I don't think Connor actually meant it that way, and perfection is an unreasonable standard in a frustrating three-hour slog of a debate. But still bad form.

Besides that (which you didn't even mention), I cannot imagine what Connor possibly could have done differently to meet your unstated standards, given his position. Should he have not gone for cruxes? Because that's how progress gets made. Debaters can easily waste inordinate amounts of time on points that neither cares about (that don't matter) because they happened to come up. Connor was laser focused on making some actual progress in the arguments, but Beff was being so damn evasive that he managed to waste a couple of hours anyway. It's a shame, but this is so not on Connor. What do you even want from him?

C: You heard it, e/acc isn't about maximizing entropy [no shit?!]

B: No, it's about maximizing the free energy

C: So e/acc should want to collapse the false vacuum?

Holy mother of bad faith. Rationalists/lesswrongers have a problem with saying obviously false things, and this is one of those.

It's in line with what seems like Connor's debate strategy - make your opponent define their views and their terminal goal in words, and then pick apart that goal by pushing it to the maximum. Embarrassing.

I agree with you that Connor performed very poorly in this debate. But this one is actually fair game. If you look at Beff's writings about "thermodynamic god" and these kinds of things, he talks a lot about how these ideas are supported by physics and the Crooks fluctuation theorem. Normally in a debate if someone says they value X, you interpret that as "I value X, but other things can also be valuable and there might be edge cases where X is bad and I'm reasonable and will make exceptions for those."

But physics doesn't have a concept of "reasonable". The ratio between the forward and backward probabilities in the Crooks fluctuation theorem is exponential in the amount of entropy produced. It's not exponential in the amount of entropy produced plus some correction terms to add in reasonable exceptions for edge cases. Given how much Beff has emphasized that his ideas originated in physics, I think it's reasonable to take him at his word and assume that he really is talking about the thing in the exponent of the Crooks fluctuation theorem. And then the question of "so hey, it sure does look like collapsing the false vacuum would dissipate an absolutely huge amount of free energy" is a very reasonable one to ask.

B: Why do you like friendship? Because evolution hard-coded this in humans

C: You’re mixing “is” and “ought”

He was not, in fact, mixing “is” and “ought”.

Seems like he was, actually. How was he not?

The current population size that Mars can support is 0, so even 1 person would be overpopulation. To complete the analogy, we are currently sending the entire population to Mars, and someone says: "But what about oxygen? We don't know if it's on Mars, maybe we should work on spacesuits?" and another says, "Nah, we'll figure it out when we get there."

+1 on can we get connor to please stop doing these debates and just... let them not happen until someone more qualified ends up doing them? for example, I'd love to see beff debate Yoshua Bengio. (Or maybe somewhat more usefully, I'd like to see Bengio debate Alex Turner.)

Your first comment that this is insufferable describes exactly what I'd expect from this debate.

Debates are not a good way to arrive at the truth.

Connor explains more about what he was trying to do here: https://twitter.com/NPCollapse/status/1753902877452439681#m

There is a pattern of debate where you make an argument of the form "X -> Y", and the other person hears "X is true", and then retorts with "But X isn't true!"

There is a viral (and probably fake) meme about prisoners and having breakfast that illustrates this pattern.

Why is it useful to make arguments of this shape? Why not just talk about X directly?

Arguments like this are useful to avoid arguing about points that aren't actually cruxes and wasting time in a debate.

As a concrete example, it is worth asking the question "if you believed that AGI was dangerous (X) -> would you agree it shouldn't be open sourced (Y)?"

The reason this is useful to establish before talking about whether AGI is actually dangerous or not is that if the other person denies that we shouldn't open source it even in principle (denies "X -> Y", independent of whether X is true or not, which is a thing more than one person I have debated has bitten the bullet on), then there's no point in arguing about X, because whether or not it is true, it will not change their view on Y, which is the thing I care about.

If the other person agrees that if it was really that dangerous, then yeah maybe it shouldn't be open sourced (accepts "X -> Y", but not "X is true"), then it is useful to move on to a discussion about whether X is true or not, because it is an actual crux that could lead to minds being changed.

Mapping out what the cruxes/degrees of freedom are in an opponent's worldview is the core of understanding the other and hopefully changing minds, rather than wasting time on points that the opponent has already decided to never change their mind on.

Unfortunately, if it takes someone 20 minutes to answer a simple yes or no "X -> Y" question, this can still run out the clock. Alas.

There are a number of other recent Tweets from Connor (@NPCollapse) with more thoughts about the debate.