Stupid Questions Open Thread Round 3

I would say that blackmail is the intersection of {things that are indistinguishable from threats} and {things that are indistinguishable from trade}. And yes, from the perspective of the blackmailers that includes both some things that are trade and some that are threats.

I disagree that the essential element to blackmail is it must be done only to make me suffer. To this end I offer a scenario. (I've made it a little more like a story just for giggles).

You've just won the lottery, and the TV people interviewed you and your wife. Hurray! Shortly after, Angela your mistress calls you up.
"Congratulations. I saw on TV that you won the lottery... I also saw you had a wife. Things are over between us!"
"Angela. I'm sorry for lying baby, but you've got a husband."
"Irrelevant. But, you know, you're rich now. How about you give me $4,000 a week for the next ten years, and I don't tell your wife."
"Oh come on, you've got a husband!"
"Yeah, and you're rich. I expect the money in a week." click
Well that's what you get for cheating. Suddenly, Julia, your other mistress calls you up.
"Congratulations. I saw on TV that you won the lottery... I also saw you had a wife. Things are over between us!"
"Julia. I'm sorry for lying baby."
"Fuck you. I should tell her right now. But, you know, you're rich now. How about you give me $4,000 a week for the next ten years, and I don't tell your wife."
"Oh come on, not again"
"What's that mean? Besides, you're rich. I expect the money in a week." click
Way to go, two timer. However, unfortunately for you, the TV show calls you up and lets you know that there was a printing error and you did NOT in fact win the lottery. You quickly call your mistresses back.
"Angela, baby, turns out I didn't win the lottery. I can't possibly pay you."
"Don't baby me. Well, I guess I won't tell your wife then as long as you don't tell my husband. I don't want anyone to know this."
"Thanks Angela, you're the best."
Hurray, bullet dodged!
"Julia, baby, turns out I didn't win the lottery. I can't possibly pay you."
"Don't baby me. Well, I guess I'll tell your wife then asshole. Since you won't pay me I'm gonna go post on facebook now!"
"But, I can't pay you because the money never belonged to me."
"Irrelevant. Sucks to be you!"

I argue that both of them were attempting to blackmail and Julia's desire to follow through with it anyways doesn't change anything. The actions would both feel like blackmail to me if I were on the receiving end, and the police would treat both of them as blackmail as well. Blackmail is just an attempt to get money in exchange for not releasing information; the mindset of the blackmailer does not affect it. This is why I agree with Vladimir Nesov, that blackmail exists in a blurry spot on the continuum of trade.

If you don't classify Julia's actions as blackmail, I would be curious what you do call it.

I still classify it as blackmail.

Something similar to this happened to Cameron Diaz although the rights to resell the photos were questionable. She posed topless in some bondage shots for a magazine, but they were never printed. The photographer kept the shots and the recording of the photo shoot for ten years until one of the Charlie's Angel's films was about to come out. He offered them to her for a couple of million or he would sell them to the highest bidder. The courts didn't buy that he was just offering her first right of refusal and sentenced him for attempted grand theft (blackmail), forgery, and perjury (for modifying release forms and lying about it). Link

But you're forgetting the man's best option: get lots of money from the movie star and get a smaller amount from the press.

And be a lot more vulnerable to criminal charges for the blackmail.

I'm assuming that the movie star is at least reasonably smart. The first thing that comes to mind is periodic payments that decrease over time, with the value hovering just above what magazines are willing to pay + bragging rights, since people are less impressed with a 10 year old sex tape than a brand new one.

Eventually the payments would be stopped when either the man has more to lose from releasing the tape than staying quiet (eg, he's settled down and married now) or the movie star values money more than the loss of prestige from a scandal (eg, another scandal breaks or she stops getting roles anyway).

I'm sure there are other ways to solve the problem as well, but regardless it's a technical hurdle rather than an absolute one.

Actually, Pascal did advise "going through the motions" as a solution to being unable to simply will oneself into belief. The wager might not be strong apologetics, but I give Pascal some credit for his grasp of cognitive dissonance.

Yes, this is what I was trying to say. I see how the phrase "conditionality of the reward on your assessed probability" could describe Pascal's Wager, but not how it could describe Pascal's Mugging.

'I don't understand this' usually means 'Would somebody please explain?'.

Maybe the right thing to do is to mix and match different presentations of the problem? I.e. one person might be all like "huh??" or "this is stupid" whenever Newcomb's problem is discussed, but be like "oh NOW I get it" when it's presented in terms of AI and source code. Somebody else might be the opposite.

I think we are simply having a definitional dispute.

That seems likely. If moral realists think the morality is a one-place word, and anti realists think it's a two place word, we would be better served by using two distinct words.

It is somewhat unclear to me what moral realists are thinking of, or claiming, about whatever it is they call morality. (Even after taking into account that different people identified as moral realists do not all agree on the subject.)

So I think what I am calling "moral realism" is something like what you are calling "Friendliness realism."

I defined 'Friendliness (to X)' as 'behaving towards X in the way that is best for X in some implied sense'. Obviously there is no Friendliness towards everyone, but there might be Friendliness towards humans: then "Friendliness realism" (my coining) is the belief that there is a single Friendly-towards-humans behavior that will in fact be Friendly towards all humans. Whereas Friendliness anti-realism is the belief no one behavior would satisfy all humans, and it would inevitably be unFriendly towards some of them.

Clearly this discussion assumes many givens. Most importantly, 1) what exactly counts as being Friendly towards someone (are we utilitarian? what kind? must we agree with the target human as to what is Friendly towards them? If we influence them to come to like us, when is that allowed?). 2) what is the set of 'all humans'? Do past, distant, future expected, or entirely hypothetical people count? What is the value of creating new people? Etc.

My position is that: 1) for most common assumed answers to these questions, I am a "Friendliness anti-realist"; I do not believe any one behavior by a superpowerful universe-optimizing AI would count as Friendliness towards all humans at once. And 2), inasfar as I have seen moral realism explained, it seems to me to be incompatible with Friendliness realism. But it's possible some people mean something entirely different by "morals" and by "moral realism" than what I've read.

If you believed that there were objective moral facts and knew the content of those facts, wouldn't you try to adjust your beliefs and actions to conform to those facts

That's a tautology: yes I would. But, the assumption is not valid.

Even if you assume there exist objective moral facts (whatever you take that to mean), it does not follow that you would be able to convince other people that they are true moral facts! I believe it is extremely likely you would not be able to convince people - just as today most people in the world seem to be moral realists (mostly religious), and yet hold widely differing moral beliefs and when they convert to another set of beliefs it is almost never due to some sort of rational convincing.

It would be nice to live in a world where you could start from the premise that "people believe that there are objective moral facts and know the content of those facts". But in practice we, and any future FAI, will live in a world where most people will reject mere verbal arguments in favor of new morals contradicting their current ones.

First, a terminology point: CEV.HUMANITYCURRENTLYALIVE() != CEV.ALLHUMANITYEVER

Sure sure. But CEV.ALLHUMANITYEVER is also not the same as all CEV.ALLPOSSIBLEAGENTS.

Second, it seems likely that fairly large groups (i.e. the population of London) already have some {P, ~P}.

Some subroutines are probably inverted but there probably aren't people with fully negated utility functions from other people. Trade-offs needn't mean irreconcilable differences. Like I doubt there is anyone in the world who cares as much as you do about the exact opposite of everything you care about.

Human value is really complex so it's a difficult thing to a)model it and b) prove the model.

I don't understand what proving the model means in this context.

Show with some confidence that it doesn't lead to terrible outcomes if implemented.

Morality is true, simple, and knowable" seems like an empirical proposition: it just turns out to be false. It isn't obvious to me that simple moral realism is necessarily false in the way that 2+5=7 is necessarily true.

I'm not sure that it is. But when I said "still" possible I meant that we have more than enough evidence to rule out the possibility that we are living in such a world. I didn't mean to imply any beliefs about necessity. That said I am pretty confused about what it would mean for there to be objective facts about right and wrong. Usually I think true beliefs are supposed to constrain anticipated experience. Since moral judgments don't do that... I'm not quite sure I know what moral realism would really mean.

How does the world look different if morality is real and inaccessible vs. not real?

I imagine it wouldn't look different but since there is no obvious way of proving a morality logically or empirically I can't see how moral realists would be able to rule it out.

Pace certain issues about human appetites as objective things, I am an anti-realist - in case that wasn't clear.

Oh I understand that. I just meant that when you ask:

If I'm a moral anti-realist, do necessarily I believe that provably Friendly AI is impossible?

I'm wondering "Opposed to what?". I'm having trouble imagining the person for whom the prospects of Friendly AI are much brighter because they are a moral realist.

However, one thing that seems pretty clear to me is that for EY's intended philosophical purposes, there really is no important distinction between "wavefunction realism" (in the context of NRQM) and "spacetime state realism" (in the context of QFT).

I'm a little unclear about what EY's intended philosophical purposes are in this context, so this might well be true. One possible problem worth pointing out is that spacetime state realism involves an abandonment of a particular form of reductionism. Whether or not EY is committed to this form of reductionism somebody more familiar with the sequences than I would have to judge.

According to spacetime state realism, the physical state of a spacetime region is not supervenient on the physical states of its subregions, i.e. the physical state of a spacetime region could be different without any of its subregions being in different states. This is because subregions can be entangled with one another in different ways without altering their local states. This is not true of wavefunction realism set in configuration space. There, the only way a region of configuration space could have different physical properties is if some of its subregions had different properties.

Also, I think it's possible that the fact that the different "worlds" in spacetime state realism are spatially overlapping (as opposed to wavefunction realism, where they are separated in configuration space) might lead to interesting conceptual differences between the two interpretations. I haven't thought about this enough to give specific reasons for this suspicion, though.

Especially since I consider this post to be mostly wrong: locality in configuration space is what matters, and configuration space is a vector space (specifically a Hilbert space) -- there is no preferred (orthonormal) basis.

I'm not sure exactly what you're saying here, but if you're rejecting the claim that MWI privileges a particular basis, I think you're wrong. Of course, you could treat configuration space itself as if it had no preferred basis, but this would still amount to privileging position over momentum. You can't go from position space to momentum space by a change of coordinates in configuration space. Configuration space is always a space of possible particle position configurations, no matter how you transform the coordinates.

I think you might be conflating configuration space with the Hilbert space of wavefunctions on configuration space. In this latter space, you can transform from a basis of position eigenstates to a basis of momentum eigenstates with a coordinate transformation. But this is not configuration space itself, it is the space of square integrable functions on configuration space. [I'm lying a little for simplicity: Position and momentum eigenstates aren't actually square integrable functions on configuration space, but there are various mathematical tricks to get around this complication.]

What concerns me is whether any actual mathematical lies are being told -- such as integrals being assumed to converge when they haven't yet been proved to do so. Or something like the early history of the Dirac delta, when physicists unashamedly spoke of a "function" with properties that a function cannot, in fact, have.

If this is your standard for lack of rigor, then perhaps QFT hasn't been rigorously formulated yet, but the same would hold of pretty much any physical theory. I think you can find places in pretty much every theory where some such "mathematical lie" is relied upon. There's an example of a standard mathematical lie told in NRQM earlier in my post.

In many of these cases, mathematicians have formulated more rigorous versions of the relevant proofs, but I think most physicists tend to be blithely ignorant of these mathematical results. Maybe QFT isn't rigorously formulated according to the mathematician's standards of rigor, but it meets the physicist's lower standards of rigor. There's a reason most physicists working on QFT are uninterested in things like Algebraic Quantum Field Theory.

You said you suspect this is necessary, but that you hope we can recover a similar MWI, but isn't it more reasonable to expect that at the planck scale something else will explain the quantum weirdness?

When I talk about recovering MWI, I really just mean absorbing the lesson that our theory does not need to deliver determinate measurement results, and ad hoc tools for satisfying this constraint (such as collapse or hidden variables) are otiose. Of course, the foundations of our eventual theory of quantum gravity might be different enough from those of quantum theory that the interpretational options don't translate. How different the foundations will be depends on which program ends up working out, I suspect. If something like canonical quantum gravity or loop quantum gravity turns out to be the way to go, then I think a lot of the conceptual work done in interpreting NRQM and QFT will carry over. If string theory turns out to be on the right track, then maybe a more radical interpretational revision will be required. The foundations of string theory are now thought to lie in M-theory, and the nature of this theory is still pretty conceptually opaque. It's worth noting though that Bousso and Susskind have actually suggested that string theory provides a solid foundation for MWI, and that the worlds in the string theory landscape are the same thing as the worlds in MWI. See here for more on this. The paper has been on my "to read" list for a while, but I haven't gotten around to it yet. I'm skeptical but interested.

Have you given Gerard 't Hoofts idea of cellular automata which he claims salvage determinism, locality and realism any thought?

I know of 't Hooft's cellular automata stuff, but I don't know much about it. Speaking from a position of admitted ignorance, I'm skeptical. I suspect the only way to construct a genuinely deterministic local realist theory that reproduces quantum statistics is to embrace superdeterminism in some form, i.e. to place constraints on the boundary conditions of the universe that make the statistics work out by hand. This move doesn't seem like good physics practice to me. Do you know if 't Hooft's strategy relies on some similar move?

The Landau pole for QED goes away when coupled with QCD, but I believe another one appears with the Higgs field.

If you don't like the question I'm answering, complain to Komponisto, not me.
But what would you count as a conceptual problem?

because until you make it a disagreement about the territory, disagreeing about maps cashes out as squabbling.

Tell that to EY.

If there are five murder worlds branching off from myself at 17, then there are five no matter what.

That's equivalent to saying "if at the moment of my 17th birthday there is a probability 5% that I will murder someone, then in that moment there is a probability 5% that I will murder someone no matter what". I agree with this.

there's still a fixed number of murder worlds extending from the day I was born, and there's nothing I can do to change that.

That's equivalent to saying "if at the day I was born there is an X% chance that I will become a murderer, there is nothing I can do to change that probability on that day". True; you can't travel back in time and create a counterfactual universe.

Is that a faulty case of determinism?

It is explained here, without the Many Words.

Short summary: You are mixing together two different views -- timeful and timeless view. In timeful view you can say "today at 12:00 I decided to kill my neighbor", and it makes sense. Then you switch to a position of a ceiling cat, an independent observer outside of our universe, outside of our time, and say "I cannot change the fact that today at 12:00 I killed my neighbor". Yes, it also makes sense; if something happened, it cannot non-happen. But we confusing two narrators here: the real you, and the ceiling cat. You decided to kill your neighbor. The ceiling cat cannot decide that you didn't, because the ceiling cat does not live in this universe; it can only observe what you did. The reason you killed your neighbor is that you, existing in this universe, have decided to do so. You are the cause. The ceiling cat sees your action as determined, because it is outside of the universe.

If we apply it to Many World hypothesis, there are 100 different yous, and one ceiling cat. From those, 5 yous commit murder (because they decided to do so), and 95 don't (because they decided otherwise, or just failed to murder successfully). Inside the universes, the 5 yous are murderers, the 95 are not. The ceiling cat may decide to blame those 95 for the actions of those 5, but that's the ceiling cat's decision. It should at least give you credit for keeping the ratio 5:95 instead of e.g. 50:50.

Would you be willing to commit to an a priori ethical principle such that ought implies can?

That's tricky. In some sense, we can't do anything unless the atoms in our bodies do it; and our atoms are following that laws of physics. In some sense, there is no such thing as "can", if we want to examine things on the atom level. (And that's equally true in Many Worlds as in One World; only in One World there is also a randomness in the equations.) In other sense, humans are decision-makers. But we are decision-makers built from atoms, not decision-makers about the atoms we are built from.

So my answer would be that "ought" implies psychological "can"; not atomic "can". (Because the whole ethics exists on psychological level, not on atomic level.)

Would you be willing to commit to an a priori ethical principle such that ought implies can?

I'm not Viliam Bur, but I wouldn't quite agree with this, in that time matters. It's not incoherent to talk about a system that can't do X, could have done X, and ought to have done X, for example. It's similarly not incoherent to talk about a system that can't do X now but ought to have acted in the past so as to be able to do X now.

But yes, in general I would say the purpose of ethics is to determine right action. If we're talking about the ethical status of a system with respect to actions we are virtually certain the system could not have taken, can not take, and will not be able to take, then we're no longer talking about ethics in any straightforward sense.

I would certainly say you're justified in not caring about hypothetical murders. I would also say you're justified in not caring about murders in other MW branches.

What you seem to want to say here is that because murders in other MW branches are "actual", you care about them, but since murders in my imagination are not "actual", you don't.

I have no idea what the word "actual" could possibly refer to so as to do the work you want it to do here.

There are certainly clusters of consistent experience to which a hypothetical murder of a hypothetical person corresponds. Those clusters might, for example, take the form of certain patterns of neural activation in my brain... that's how I usually model it, anyway. I'm happy to say that those are "actual" patterns of neural activation. I would not say that they are "actual" murdered human beings.

That said, I'm not really sure it matters if they are. I mean, if they are, then... hold on, let me visualize... there: I just "actually" resurrected them and they are now "actually" extremely happy. Was their former murder still evil? At best, it seems all of my preconceived notions about murder (e.g., that it's a permanent state change of some kind) have just been thrown out the window, and I should give some serious thought to why I think murder is evil in the first place.

It seems something similar is true about existence in a Big World... if I want to incorporate that into my thinking, it seems I ought to rethink all of my assumptions. Transplanting a moral intuition about murder derived in a small world into a big world without any alteration seems like a recipe for walking off conceptual cliffs.

So what does adding up to normal mean?

It means that if you do not observe pink unicorns daily, no new weird and wonderful theory should claim that you should have. Or, as EY puts it "apples didn't stop falling, planets didn't swerve into the Sun". Another name for this is the correspondence principle.

If your ethics requires for you to be the first tune whistler in the multiverse, not just in this world, it's not a useful ethics.

So what does adding up to normal mean?

It means that if in your branch you are the first one to whistle the tune, there is no one else in your branch to contradict you. (Just as you would expect in One World.) In some other branch someone else was first, and in that branch you don't think that you were the first, so again no conflict.

if my normal is Newtonian physics

Then "adding up to normal" means that even when Einstein ruins your model, all things will behave the same way as they always did. Things that within given precision obeyed the Newtonian physics, will continue to do it. You will only see exceptions in unusual situations, such as GPS satellites. (But if you had GPS satellites before Einstein invented his theory, you would have seen those exceptions too. You just didn't know that would happen.)

In case of morality it means that if you had a rule "X is good" because it usually has good consequences (or because it follows the rules, or whatever), then "X is good" even with Many Worlds. The exception is if you try to apply moral significance to a photon moving through a double slit.

An explanation may change: for example it was immoral to say "if the coin ends this side up, I will kill you", and it is still immoral to do so, but the previous explanation was that "it is bad to kill people with 50% probability" and the new explanation is "it is bad to kill people in 50% of branches" (which means killing them with 50% probability in a random branch).

One that doesn't classify evolution as intelligent.

Cockroaches and the behavior of humans.

In itself that doesn't seem to be relevant evidence. "There exist species that humans cannot eradicate without major coordinated effort". It doesn't follow that either the same would hold for far more powerful AIs, nor that we should model AI-human relationship on humans-cockroaches rather than humans-kittens or humans-smallpox.

If humans can get into space, the lightspeed barrier might let far-flung tribes of "human fundamentalists," to borrow a term from Charles Stross, to survive

It's easy to imagine specific scenarios, especially when generalizing from fictional evidence. In fact we don't have evidence sufficient to even raise any scenario as concrete as yours to the level of awareness.

I could as easily reply that AI that wanted to kill fleeing humans could do so by powerful enough directed lasers, which will overtake any STL ship. But this is a contrived scenario. There really is no reason to discuss it specifically. (For one thing, there's still no evidence human space colonization or even solar system colonization will happen anytime soon. And unlike AI it's not going to happen suddenly, without lots of advanced notice.)