Comment by sharps030 on Blackmail · 2019-02-20T12:58:55.330Z · score: 5 (5 votes) · LW · GW

"It's obviously bad. Think about it and you'll notice that. I could write a YA dystopian novel about how the consequences are bad." <-- isn't an argument, at all. It assumes bad consequences rather than demonstrating or explaining how the consequences would be bad. That section is there for other reasons, partially (I think?) to explain Zvi's emotional state and why he wrote the article, and why it has a certain tone.

Comment by sharps030 on Blackmail · 2019-02-20T12:57:45.764Z · score: 6 (5 votes) · LW · GW
I am not sure why you pick on blackmail specifically

This is in response to other writers, esp. Robin Hanson. That's why.

Comment by sharps030 on Blackmail · 2019-02-20T12:52:41.240Z · score: 15 (6 votes) · LW · GW
This only looks at the effects on Alice and on Bob, as a simplification. But with blackmail "carrying out the threat" means telling other people information about Bob, and that is often useful for those other people.

When the public interest motivates the release of private info, it's called 'whistleblowing' and is* legally protected and considered far more moral than blackmail. I think that contrast is helpful to understanding why that's not enough to make blackmail moral.

*in some jurisdictions, restrictions may apply, see your local legal code for a full list of terms & conditions.

I think you're right that it's not trivially negative sum because it can have positive outcomes for third parties. Still expect a world of legal blackmail to be worse.

Comment by sharps030 on Open Thread August 2018 · 2018-08-05T03:50:30.249Z · score: 3 (3 votes) · LW · GW

If you're throwing your AI into a perfect inescapable hole to die and never again interacting with it, then what exact code you're running will never matter. If you observe it though, then it can affect you. That's an output.

What are you planning to do with the filtered-in 'friendly' AIs? Run them in a different context? Trust them with access to resources? Then an unfriendly AI can propose you as a plausible hypothesis, predict your actions, and fake being friendly. It's just got to consider that escape might be reachable, or that there might be things it doesn't know, or that sleeping for a few centuries and seeing if anything happens is a option-maximizing alternative to halting, etc. I don't know what you're selecting for -- suicidality, willingness to give up, halts within n operations -- but it's not friendliness.

Comment by sharps030 on 5 general voting pathologies: lesser names of Moloch · 2018-04-14T04:59:53.142Z · score: 6 (3 votes) · LW · GW

The first link in this post should go ^ here to your voting theory primer. Instead, for me, it links here: