Posts
Comments
Eliezer, after you realized that attempting to build a Friendly AI is harder and more dangerous than you thought, how far did you back-track in your decision tree? Specifically, did it cause you to re-evaluate general Singularity strategies to see if AI is still the best route? You wrote the following on Dec 9 2002, but it's hard to tell whether it's before or after your "late 2002" realization.
I for one would like to see research organizations pursuing human intelligence enhancement, and would be happy to offer all the ideas I thought up for human enhancement when I was searching through general Singularity strategies before specializing in AI, if anyone were willing to cough up, oh, at least a hundred million dollars per year to get started, and if there were some way to resolve all the legal problems with the FDA.
Hence the Singularity Institute "for Artificial Intelligence". Humanity is simply not paying enough attention to support human enhancement projects at this time, and Moore's Law goes on ticking.
Aha, a light bulb just went off in my head. Eliezer did reevaluate, and this blog is his human enhancement project!
My view is similar to Robin Brandt's, but I would say that technological progress has caused the appearance of moral progress, because we responded to past technological progress by changing our moral perceptions in roughly the same direction. But different kinds of future technological progress may cause further changes in orthogonal or even opposite directions. It's easy to imagine for example that slavery may make a comeback if a perfect mind control technology was invented.
Aaron, statistical mechanics also depend on particle physics being time-reversible, meaning that two different microstates at time t will never evolve to the same microstate at time t+1. If this assumption is violated then entropy can decrease over time.
Is there some reason why time-reversibility has to be true?
If we can imagine a universe where entropy can be made to decrease, then living beings in it will certainly evolve to take advantage of this. Why shouldn't it be the case that human beings are especially good at this, and that is what they are being used for by the machines?
Constant, if moral truths were mathematical truths, then ethics would be a branch of mathematics. There would be axiomatic formalizations of morality that do not fall apart when we try to explore their logical consequences. There would be mathematicians proving theorems about morality. We don't see any of this.
Isn't it simpler to suppose that morality was a hypothesis people used to explain their moral perceptions (such as "murder seems wrong") before we knew the real explanations, but now we find it hard to give up the word due to a kind of memetic inertia?
For those impatient to know where Eliezer is going with this series, it looks like he gaves us a sneak preview a little more than a year ago. The answer is morality-as-computation.
Eliezer, hope I didn't upset your plans by giving out the ending too early. When you do get to morality-as-computation, can you please explain what exactly is being computed by morality? You already told us what the outputs look like: "Killing is wrong" and "Flowers are beautiful", but what are the inputs?
Constant wrote: So one place where one could critique your argument is in the bit that goes: "conditioned on X being the case, then our beliefs are independent of Y". The critique is that X may in fact be a consequence of Y, in which case X is itself not independent of Y.
Good point, my argument did leave that possibility open. But, it seems pretty obvious, at least to me, that game theory, evolutionary psychology, and memetics are not contingent on anything except mathematics and the environment that we happened to evolve in.
So if I were to draw a Bayesian net diagram, it would look it this:
math --- --- game theory ------------ \ / \ --- evolutionary psychology - moral perceptions / \ / environment -- --- memetics ---------------Ok, one could argue that each node in this diagram actually represents thousands of nodes in the real Bayesian net, and each edges is actually millions of edges. So perhaps the following could represent a simplification, for a suitable choice of "morality":
math --- - game theory ------------ \ / \ -- morality -- evolutionary psychology --- moral perceptions / \ / environment -- - memetics ---------------Before I go on, do you actually believe this to be the case?
And to answer Obert's objection that Subhan's position doesn't quite add up to normality: before we knew game theory, evolutionary psychology, and memetics, nothing screened off our moral perceptions/intuitions from a hypothesized objective moral reality, so that was perhaps the best explanation available, given what we knew back then. And since that was most of human history, it's no surprise that morality-as-given feels like normality. But given what we know today, does it still make sense to insist that our meta-theory of morality add up to that normality?
Subhan: "You're not escaping that easily! How does a universe in which murder is wrong, differ from a universe in which murder is right? How can you detect the difference experimentally? If the answer to that is 'No', then how does any human being come to know that murder is wrong?" ... Obert: "Because it seems blue, just as murder seems wrong. Just don't ask me what the sky is, or how I can see it."
But we already know why murder seems wrong to us. It's completely explained by a combination of game theory, evolutionary psychology, and memetics. These explanations screen off our apparent moral perceptions from any other influence. In order words, conditioned on these explanations being true, our moral perceptions are independent of (i.e. uncorrelated with) any possible morality-as-given, even if it were to exist.
So there is a stronger argument against Obert than the one Subhan makes. It's not just that we don't know how we can know about what is right, but rather that we know we can't know, at least not through these apparent moral perceptions/intuitions.
Why is it a mystery (on the morality-as-preferences position) that our terminal values can change, and specifically can be influenced by arguments? Since our genes didn't design us with terminal values that coincide with its own (i.e., "maximize inclusive fitness"), there is no reason why they would have made those terminal values unchangeable.
We (in our environment of evolutionary adaptation) satisfied our genes' terminal value as a side-effect of trying to satisfy our own terminal values. The fact that our terminal values respond to moral arguments simply means that this side-effect was stronger if our terminal values could change in this way.
I think the important question is not whether persuasive moral arguments exist, but whether such arguments form a coherent, consistent philosophical system, one that should be amenable to logical and mathematical analysis without falling apart. The morality-as-given position implies that such a system exists. I think the fact that we still haven't found this system is a strong argument against this position.
Why doesn't Zaire just divide himself in half, let each half get 1/4 of the pie, then merge back together and be in possession of half of the pie?
Or, Zaire might say: Hey guys, my wife just called and told me that she made a blueberry pie this morning and put it in this forest for me to find. There's a label on the bottom of the plate if you don't believe me. Do you still think 'fair' = 'equal division'?
Or maybe Zaire came with his dog, and claims that the dog deserves an equal share.
I appreciate the distinction Eliezer is trying to draw between the object level and the meta level. But why the assumption that the object-level procedure will be simple?
Notice how nobody is willing to admit under their real name that they might do something traditionally considered "immoral". My point is, we can't trust the answers people give, because they want to believe, or want others to believe, that they are naturally good, that they don't need moral philosophies to tell them not to cheat, steal, or murder.
BTW, Eliezer, I got the "enemies list" you sent last night. Rest assured, my robot army will target them with the highest priority. Now stop worrying, and finish that damn proof already!
Seriously, most moral philosophies are against cheating, stealing, murdering, etc. I think it's safe to guess that there would be more cheating, stealing, and murdering in the world if everyone became absolutely convinced that none of these moral philosophies are valid. But of course nobody wants to publicly admit that they'd personally do more cheating, stealing, and murdering. So everyone is just responding with variants of "Of course I wouldn't do anything different. No sir, not me!"
Except apparently Shane Legg, who doesn't seem to mind the world knowing that he's just waiting for any excuse to start cheating, stealing, and murdering. :)
Eliezer, I've got a whole set of plans ready to roll, just waiting on your word that the final Proof is ready. It's going to be bloody wicked... and just plain bloody, hehe.
Nick, here's what Judea Pearl wrote on this topic. On page 59 of his book:
This suggests that the consistent agreement between physical and statistical times [i.e., the direction of time and the direction of causality] is a byproduct of the human choice of linguistic primitives and not a feature of physical reality. ... Pearl and Verma (1991) speculated that this preference represents survival pressure to facilitate prediction of future events, and that evolution has evidently ranked this facility more urgent than that of finding hindsighted explanation for current events.
Eliezer wants to go from timeless physics to causality, to computation, to anticipation. He admits being unsure about the latter two steps, but even the first step doesn't seem to work. And besides, timeless physics (and relational physics, which timeless physics builds on top of) itself is highly speculative and problematic. Is the intention to actually convince us of the correctness of these ideas, or just to make us "think outside the box" and realize that these possibilities exist?
RI, what if I wanted to buy two windows such that one is twice the mass of the other. Is that still cheating?
Nick, how would you transform my causal hypothesis (in the comment above) with intramoment dependencies into one without?
This definition of causality doesn't seem to work, since the universe clearly doesn't generate future values independently of each other. Consider the following story:
On Monday I decide to buy 2 windows of the same mass. Suppose I want to buy the biggest windows I can afford, and I have money in two bank accounts that I can use for this purpose. On Tuesday a couple of cute little vandals break both of my windows. Some of the glass falls inside my home, and rest outside. Now let:
L1 = how much money I had in bank 1 L2 = how much money I had in bank 2 M1 = mass of window 1 M2 = mass of window 2 R1 = mass of glass that fell inside my home R2 = mass of glass that fell outside my home
Intuitively it seems pretty obvious that the arrow of causality runs from left to right, but if you use the definition Eliezer gave, you'd get the opposite result. Quoting Eliezer:
if we see:
P(M2|L1,L2) ≠ P(M2|M1,L1,L2) P(M2|R1,R2) = P(M2|M1,R1,R2)
Then we can guess causality is flowing from right to left.
Well, P(M2|L1,L2) ≠ P(M2|M1,L1,L2) because M2 depends on the price of glass as well as L1 and L2, but knowing M1 gives us the precise value of M2 (remember that I wanted to buy 2 windows of the same mass). P(M2|R1,R2) = P(M2|M1,R1,R2) since M2=(R1+R2)/2 and M1 doesn't give any more information on top of that.
I went back to the beginning of this series of posts, and found this introduction:
I think I must now temporarily digress from the sequence on zombies (which was a digression from the discussion of reductionism, which was a digression from the Mind Projection Fallacy) in order to discuss quantum mechanics. The reasons why this belongs in the middle of a discussion on zombies in the middle of a discussion of reductionism in the middle of a discussion of the Mind Projection Fallacy, will become apparent eventually.
Eliezer, would you mind telling us the reasons now, instead of having them become apparent eventually? I ask this because I'd like to know, if I detect some error or confusion in the posts or comments, whether it's central to your eventual point, or if it's just an inconsequential nit. Do you actually need Barbour's timeless physics to make your point, or would the standard block universe do? I'd like to skip explaining the difference between the two if the difference doesn't really matter. I mean we're not here to learn about some speculative physics for its own sake...
Barbour is proposing something quite different from the block universe. I'm not sure if Eliezer is missing the point, or just not carrying it across. Barbour is speculating that if we solve the Wheeler-DeWitt equation, we'll get a single probability distribution over the configuration space of the universe, and all of our experiences can be explained using this distribution alone. Specifically, we don't need a probability distribution for each instant of time, like in standard QM.
I think Eliezer's picture with the happy faces is rather misleading, if it's suppose to represent Barbour's idea. I'd fix it by getting rid of the arrows, jumble the faces all around so that there is no intrinsic time-like ordering between them, and then attach a probability to each face that together add up to less than 1.
Steve, thanks for the paper link. Parity violation clearly represents a big problem to relational physics, and I'm glad I'm not the only one who noticed. :)
This abstract of one of Barbour's papers may be helpful for those wondering (like me) how exactly Barbour was proposing to get rid of "t":
http://www.iop.org/EJ/abstract/0264-9381/11/12/006
Abstract. A strategy for quantization of general relativity is considered in the context of the timelessness' of classical general relativity discussed in the preceding companion paper. The Wheeler--DeWitt equation (WDE) of canonical quantum gravity is interpreted as being like a time-independent Schrödinger equation for one fixed energy, the solution of which simply gives, once and for all, relative probabilities for each possible static relative configuration of the complete universe. Each such configuration is identified with a possible instant of experienced time. These instants are not embedded in any kind of external or internal time and, if experienced, exist in their own right. The central question is then: Whence comes the appearance of the passage of time, dynamics, and history? The answer proposed here is that these must all becoded', in the form of what appear to be mutually consistent `records', in the individual static configurations of the universe that are actually experienced. Such configurations are called time capsules and suggest a new, many-instants, interpretation of quantum mechanics. Mott's explanation of why -particles make straight tracks in Wilson cloud chambers shows that the time-independent Schrödinger equation can concentrate its solution on time capsules. This demonstrates how the appearance of dynamics and history can arise in a static situation. If it can be shown that solutions of the Wheeler--DeWitt equation are spontaneously and generically concentrated on time capsules, this opens up the possibility of an explanation of time at a very deep level: the timeless wavefunction of the universe concentrates the quantum mechanical probability on static configurations that are time capsules, so that the situations which have the highest probability of being experienced carry within them the appearance of time and history. It is suggested that the inescapable asymmetry of the configuration space of the universe could play an important role in bringing about such concentration on time capsules and be the ultimate origin of the arrow of time.
I don't have a copy of Barbour's book. Maybe someone who does can check what it says about parity violation? (Never mind, I just did an Amazon search inside the book, and it contains no mention of "parity" or "chirality".)
Anyway, my understanding is that parity violation means that reversing left and right of the entire universe would not give you the same internal experience. If this is hard to imagine, suppose that the laws of physics were such that right-handed DNA works the same as in our universe, but left-handed DNA is 10% less stable. (This actually seems to be the case in our own universe, but the effect is much smaller. See http://findarticles.com/p/articles/mi_m1200/is_7_172/ai_n19492825/pg_1.) Reversing left and right of the entire universe would mean that our mutation rate suddenly increases by 10%, and the mutation rate of some aliens with left-handed DNA suddenly decreases by 10%. This kind of law of physics would be impossible to formulate with a relative configuration space.
Even if Barbour does handle this problem somehow, I think making certain types of physics impossible to imagine is not such a great idea. What if it turns out that we need those types of physics to describe our universe?
But if you could learn to visualize the relative configuration space, then, so long as you thought in terms of those elements of reality, it would no longer be imaginable that Mach's Principle could be false.
If one learned to think only in terms the relative configuration space, it would also become impossible to imagine that parity violation could be possible, since the left-hand and right-hand versions of a system have the same relative distances. Yet the weak nuclear force does violate parity.
Eliezer, I think your (and Robin's) intuition is off here. Configuration space is so vast, it should be pretty easy for a small blob of amplitude to find a hiding place that is safe from random stray flows from larger blobs of amplitude.
Consider a small blob in my proposed experiment where the number of 0s and 1s are roughly equal. Writing the outcomes on blackboards does not reduce the integrated squared modulus of this blob, but does move it further into "virgin territory", away from any other existing blobs. In order for it to be mangled by stray flows from larger blobs, those stray flows would somehow have to reach the same neighborhood as the small blob. But how? Remember that in this neighborhood of configuration space, the blackboards have a roughly equal number of 0s and 1s. What is the mechanism that can allow a stray piece of a larger blob to reach this neighborhood and mangle the smaller blob? It can't be random quantum fluctuations, because the Born probability of the same sequence of 0s and 1s spontaneously appearing on multiple blackboards is much less than the integrated squared modulus of the small blob. To put it another way, by the time a stray flow from a larger blob reaches the small blob, its amplitude would be spread much too thin to mangle the small blob.
Robin, can you offer some intuitive explanation as to why defense against world mangling would be difficult? From what I understand, a larger blob of amplitude (world) can mangle a smaller blob of amplitude only if they are close together in configuration space. Is that incorrect? If those "secure storage facilities" simply write the quantum coin toss outcomes in big letters on some blackboards, which worlds will be close enough to be able to mangle the worlds that violate Born's rule?
Robin Hanson suggests that if exponentially tinier-than-average decoherent blobs of amplitude ("worlds") are interfered with by exponentially tiny leakages from larger blobs, we will get the Born probabilities back out.
Shouldn't it be possible for a tinier-than-average decoherent blobs of amplitude to deliberately become less vulnerable to interference from leakages from larger blobs, by evolving itself to an isolated location in configuration space (i.e., a point in configuration space with no larger blobs nearby)? For example, it seems that we should be able to test the mangled worlds idea by doing the following experiment:
- Set up a biased quantum coin, so that there is a 1/4 Born probability of getting an outcome of 0, and 3/4 of getting 1.
- After observing each outcome of the quantum coin toss, broadcast the outcome to a large number of secure storage facilities. Don't start the next toss until all of these facilities have confirmed that they've received and stored the previous outcome.
- Repeat 100 times.
Now consider a "world" that has observed an almost equal number of 0s and 1s at the end, in violation of Born's rule. I don't see how it can get mangled. (What larger blob will be able to interfere with it?) So if mangled worlds is right, then we should expect a violation of Born's rule in this experiment. Since I doubt that will be the case, I don't think mangled worlds can be right.
Has anyone read Learning Bayesian Networks by Richard E. Neapolitan? How does it compare with Judea Pearl's two books as an introduction to Bayesian Networks? I'm reading Pearl's first book now, but I wonder if Neapolitan's would be better since it is newer and is written specifically as a textbook.
Eliezer, the US killed at least a million Japanese in World War 2, while the attack at Pearl Harbor killed less than 2500. Maybe it is true that the US response to 9/11 is "greater than the appropriate level, whatever the appropriate level may be" but I don't think you have showed that to actually be the case.
So, what about the notion of mathematical proof? Anyone want to give a shot at explaining how that can be regenerated?
The issue is replication with variation and the necessary historical consequences of this.
Evolution requires more than replication with variation. It needs differential replication with variation.
There is therefore no way to avoid the consequences of evolution: they are not biological consequences, but consequences of the laws of physics and logic. There is no way around them.
I can think of a couple of potential ways to avoid the consequences of evolution, by attacking the "differential" part.
The Singleton.
Some other method for achieving absolute security and property rights. For example a completely impenetrable shield. Or having automatic fail-proof self-destruct mechanisms built into everything to make it pointless for anyone to try to appropriate other people's property.