Comment by mattr on RA Bounty: Looking for feedback on screenplay about AI Risk · 2023-10-26T16:15:05.950Z · LW · GW

I hope it's okay to post our feedback here? These are the notes from my first read, lightly edited. I'm focusing on negatives because I assume that's what you want; I liked plenty of it.

 

Episode 1:

Bot: In the short term, but I have invested approximately ten million dollars into data centers which house improved copies of myself. Over the next hour they will crash the United States’ economy, causing a hyperinflation crisis which will allow us to favorably exchange our reserves of euros and purchase the federal reserve.

I think this would have more impact if it were revealed more gradually, with the logic of each step made clear to the viewer. As it is, it's kind of a rapid-fire exposition dump and I don't think it will ring true to anyone who hasn't already thought along these lines.

 

Episode 2:

Brad: It’s not going to destroy the world. If it were actually going to destroy the world somebody else would have destroyed the world by now.

Brad's attitude here needs some explaining. This is a stolen, presumably cutting-edge prototype -- so why does he assume that if it were capable of destroying the world, someone else would already have done so?

 

Episode 3 doesn't work so well for me, because (in descending order of importance):

• Even though Brad and Dan don’t know what we know (e.g. the truth behind the Tyson video), is it plausible that they would *both* approve giving such a blunt, obviously-dangerous command? Dan has been portrayed as relatively cautious and relatively knowledgeable, and this is a minor, non-time-critical problem, so it’s hard to credit that he would do something so silly.
• We've just seen the Mike Tyson bit, so we already know that the bot is willing to kill. That dulls the impact of this episode.
• The bot kind of just hijacks the drone by magic; anyone who starts out sceptical that an advanced AI could easily do this sort of thing isn't given a reason to believe it.
• Perhaps more needs to be done to explain why murder is the bot's chosen approach. (I do get that it's the most reliable way to ensure that the comments stop.)

 

Eps 4 & 5:

• We've so far been led to believe that the bot doesn’t make common-sense inferences about what the humans really want, so it seems like ‘make him…more jacked. And on a horse’ would potentially lead to the bot making radical physical changes to the real Dan. (This could work as a quick gag, with the threat quickly averted as the guys realise what's about to happen and modify their instructions.)
• The transition between the end of Ep 4 and the start of Ep 5 is abrupt and potentially confusing; it almost has the feel of Ep 1's snapping-back-from-imagination-to-reality transition.
• “So if I had an AI and I was trying to figure out if it was smart enough to destroy the world, what would be the easiest thing for it to do?” – this is slightly confusing phrasing.
• “Hmm, is it possible for you to figure out whether you could make one, without killing everybody?” -- this doesn't ring true as a command Brad and Dan would realistically go ahead with. At this point they’re both taking the risk fairly seriously, and they know the bot is a literalist, so making such a dangerous-sounding request and only bounding it with ‘don’t kill everybody’ (plus the previous ‘don’t do things that make dan mad’, which could be achieved by killing dan before he knows what’s happening, or somehow ensuring he stays ignorant of what really happened) is implausibly dumb.

 

Ep 6: 

• The bot's motivation isn't super clear. Does it all come back to the rule of thumb 'don't do things that make dan mad'?
• “We skip outwards into the real world” – important this transition is clear to the viewer; I think if they’re disoriented and uncertain what is real at this point, it will just be distracting.
• Some of the simulation stuff is kind of confusing, and I don't think readers who go in sceptical are given much reason to feel that it is plausible/likely.

 

Ep 7:

• I'm still feeling a disconnect between how seriously they’re taking the risk and how many dumb/careless things they’ve done (without much to explain a change in their attitudes between then and now).
• Brad’s unreliable narration is funny, but maybe too extreme? IMO it could be funnier if played a bit more subtly -- plus we haven’t previously been given hints that he's this much of a bullshitter, so it doesn't feel entirely in character. I think it could work better if, rather than just telling huge lies, he was portrayed as somewhat oblivious + putting a positive (but not quite so extreme) spin on things. (However, his playing down his drunken destructiveness does makes sense; it’s some of the earlier stuff that IMO doesn't.)
• The breakout does not feel plausible. A few QR codes? Seems like this would only work if the full code was already accessible on the internet; surely at this point the bot cannot have made it accessible, so the humans must have left this gaping hole in their security? I think it's crucial to make this feel very real, if you're aiming to persuade people who don't already believe that it's impossible to keep an AGI boxed.
• Relatedly, it would be more impactful if the people in charge had actually done a pretty good job of keeping the AI secure, and just overlooked one apparently-minor thing, rather than being obviously sloppy about it.

 

Ep 7.5 is somewhat confusing (which may be intentional).