Explanations for Less Wrong articles that you didn't understand

If you want to build an AI that maximizes utility, and that AI can create copies of itself, and each copy's existence and state of knowledge can also depend on events happening in the world, then you need a general theory of how to make decisions in such situations. In the limiting case when there's no copying at all, the solution is standard Bayesian rationality and expected utility maximization, but that falls apart when you introduce copying. Basically we need a theory that looks as nice as Bayesian rationality, is reflectively consistent (i.e. the AI won't immediately self-modify away from it), and leads to reasonable decisions in the presence of copying. Coming up with such a theory turns out to be surprisingly hard. Many of us feel that UDT is the right approach, but many gaps still have to be filled in.

Note that many problems that involve copying can be converted to problems that create identical mind states by erasing memories. My favorite motivating example is the Absent-Minded Driver problem. The Sleeping Beauty problem is similar to that, but formulated in terms of probabilities instead of decisions, so people get confused.

An even simpler way to emulate copying is by putting multiple people in the same situation. That leads to various "anthropic problems", which are well covered in Bostrom's book. My favorite example of these is Psy-Kosh's problem.

Another idea that's equivalent to copying is having powerful agents that can predict your actions, like in Newcomb's problem, Counterfactual Mugging and some more complicated scenarios that we came up with.

I don't know about academic philosophy, but on Less Wrong there is the hope of one day coming up with an algorithm that calculates the "best", "most rational" way to act.

That's a bit of a simplification, though. It is hoped that we can separate the question of how to learn (epistemology) and what is right (moral philosophy) from the question of given one's knowledge and values, what is the "best", "most rational" way to behave? (decision theory).

The von Neumann–Morgenstern theorem is the paradigmatic result here. It suggests (but does not prove) that given one's beliefs and values, one "should" act so as to maximize a certain weighted sum. But as the various paradoxes show, this is far from the last word on the matter.

The naive view of truth:

Some things are true, some things are false, like "My name is 'Ben'." - True "My name is Alfred'." - False

When it comes to factual questions, you should believe in their truth the more you have evidence for them. If well-researched statistics indicate that one country has a higher homicide rate than another, then you should believe it (unless you have other, really good evidence to the contrary). If well-formulated studies come back in, and a certain brand of alternative medicine has been discovered to be 'ineffectual', then you should believe it (unless you have other, really good evidence to the contrary). One should not start arguing about "well, what is truth really?" or "how can we ever know anything really?". If one actually thought like this, I think it was Feynman who noted that these people would soon die of starvation, because they'd never really know if that yellow thing was a banana, and that they could eat it. These arguments are simply ways of dismissing really good evidence, and you should not use them.

The purpose of the essay, is so that when you're in an argument, you provide evidence, and the person goes "but all truth is relative" or "nothing is true, it's all just oppression of the many by the powerful" you can send it them and say "stop evading the actual evidence!".

What is that "naive view of truth"?

"Snow is white" is true if and only if snow is white.

A map works by being in correspondence with the territory.

There is nothing magical about the correspondence. It is completely reducible to physics.

Do you not understand what Eliezer is saying, or are you just disagreeing with it?

all I see is strawmanning everything Eliezer dislikes

It would probably help to have a link to specific people/books who have inspired specific sections, to see how much they were strawmanned.

I remember talking with a few people who refused to be reasonable about anything, defending it with more and more meta nonsense, typically up to "how do you even know there is such thing as a reality or truth?" Using LW lingo, you can evaluate maps by compating then with the terrority, but what if someone's map contains a large text "there is no such thing as territory"? Even if you show them that your map fits the territory better, they will point out that matching the territory counts as an improvement only according to your map, not according to their map, so you didn't really demonstrate anything beyond your maps being different, which both of you already knew.

Reading the article reminded me of some of their techniques. Yeah, that didn't really mean disproving them. But I was nice to know I wasn't the only one who finds these discussion techniques irritating.

The naive version (with added parenthesis) :

• ('Snow is white' is true) if and only if (snow is white)

I really wanted to help, because you're helping me with the free will thing, but I could only manage to skim the essay. I take it that the naive view of truth is supposed to be the disquotational or deflationary view. This is to say that the assertion

'Snow is white' is true.

is identical content-wise to the assertion

Snow is white.

To say that something is true is just to assert that thing, and asserting it is sufficient to say that it's true. In other words, we can for most purposes just do without the word 'true' (though things are more complicated for 'false').

As far as I can tell, Eliezer is arguing for the correspondence theory of truth.

The naive view is that "'true' is true."

I was reading Eliezer's cartoon proof of Lob's theorem the other day and I didn't get it.

In order to understand it, I am currently reading Forever Undecided: A Puzzle Guide to Gödel. The book features a whole chapter about Löb's theorem.

The book does not have any prerequisites. It starts out with plain English logic puzzles that you need to solve (detailed solutions at the end of each chapter), and later introduces you to formal logic by translating those puzzles into propositional logic.

I have not yet finished the book, so I can't tell if it fits the purpose of understanding Löb's theorem. But what I can already tell is that it is really engaging and fascinating. Highly recommended!

ETA: To give a taste of Raymond M. Smullyan's style, check out his 'World's shortest explanation of Gödel's theorem':

We have some sort of machine that prints out statements in some sort of language. It needn't be a statement-printing machine exactly; it could be some sort of technique for taking statements and deciding if they are true. But let's think of it as a machine that prints out statements.

In particular, some of the statements that the machine might (or might not) print look like these:

P*x (which means that the machine will print x)

NP*x (which means that the machine will never print x)

PR*x (which means that the machine will print xx)

NPR*x (which means that the machine will never print xx)

For example, NPR*FOO means that the machine will never print FOOFOO. NP*FOOFOO means the same thing. So far, so good.

Now, let's consider the statement NPR*NPR*. This statement asserts that the machine will never print NPR*NPR*.

Either the machine prints NPR*NPR*, or it never prints NPR*NPR*.

If the machine prints NPR*NPR*, it has printed a false statement. But if the machine never prints NPR*NPR*, then NPR*NPR* is a true statement that the machine never prints.

So either the machine sometimes prints false statements, or there are true statements that it never prints.

So any machine that prints only true statements must fail to print some true statements.

Or conversely, any machine that prints every possible true statement must print some false statements too.

If you already understand Gödel's first and second incompleteness theorems, then you can find a much simpler proof of Löb's theorem in this pdf, pages 6-7.

One thing I read Eliezer as saying, in Dissolving the Question, is that the phenomenology of free will is more interesting than the metaphysics:

Your homework assignment is to write a stack trace of the internal algorithms of the human mind as they produce the intuitions that power the whole damn philosophical argument.

(This comment is not a full answer to that "homework assignment.")

In other words, it is a fact that humans do reasonably reliably possess the intuition, "I have free will." We do have that intuition; our having it is something to be explained. And it is a fact that when we examine the processes that we are made of — physics — we do not (contra Penrose) see anywhere for free will to sneak in. Brains use the same atoms that billiard balls and computers do.

(I don't know if you are a coder. A "stack trace" is a snapshot of what is going on, at a particular moment, at every level of abstraction in a computer program. Stack traces are often seen when a program crashes, to let the programmer follow the trail of the code bug or bad data that let to the crash. We might obtain a stack trace of consciousness through introspective techniques such as meditation — I'm not there yet via that particular method, but I think I can follow the arguments.)

This is fine, if that's the answer. But it's not a compatibilist answer.

I take Eliezer to be (heavily) influenced by Daniel Dennett. Dennett in Elbow Room, Freedom Evolves, etc. holds that what we want out of "free will" is that we create choices to influence the future; that we can take reasoned steps to avoid predicted bad outcomes; that we could have done otherwise if we thought or believed differently. This is just as incompatible with indeterminism (wherein our seeming choices are the results of quantum indeterminacy in the neurons (Penrose) as well as with a sort of greedy mechanical determinism where our choices are produced by our bodies without conscious reflection. I take Dennett as implying that our choices are produced by our bodies, but conscious reflection is our name for the mechanism by which they are produced.

(As Eliezer points out in the Anti-Zombie posts, consciousness does have an effect on the world, notably that we talk about consciousness: we discuss our thoughts, plans, fears, dreams.)

Eliezer diagrams this in Thou Art Physics: the determinist claims that the world's future is caused by physics operating on the past, and not by me making choices. But to the computationally minded materialist, "me" is the name of the place within physics where my choices are calculated, and "me" certainly does have quite a bit of control over the future.

I am not convinced that a materialist determinist like Sam Harris or Democritus would be convinced. The fact that I draw a line around some part of physics and call it "me" doesn't mean I control what goes on in that boundary, after all.

(Computation is vital here, because computation and (selective) correlation are never free. In order for a computation to take place, it has to take place somewhere. In order for some outputs (say, my movement towards the cookie jar) to correlate with some inputs (my visual signals about the cookie jar), that correlation has to be processed somewhere. Plot my worldline, and I am in orbit around the cookie jar with some very complex equation modeling my path; but where that equation is actually computed in order to guide my feet, is inside my brain.)

But the reason that determinism worries freshman philosophy students and novice LWers is that it seems to imply fatalism — that the choices we make don't matter, because the universe is scripted in advance. This compatibilist view, though, seems to say that the choices we make do matter, because they are part of how the universe calculates what the future will bring.

Fatalism says we can't change the future, so we may as well just sit on the couch playing video games. Compatibilism says that we are the means of changing the future.

I don't think that's EY's solution - I don't think his discussion of Free Will has anything to do with moral responsibility being a matter of convention.

From what I recall, the argument is something more like this: When people talk of "Free will", it's not clear what exactly they are referring to. If you try to pin down a more precise meaning that matches people's intuitions, you get something like "the subjective sensation of evaluating different available courses of action one might take" - and that is compatible with determinism (you can run decision algorithms in a perfectly deterministic binary world e.g. a simulated tile-based game world).

Does that make sense?

I would like to know how you understand free will. But since philosophical definitions are generally useless, I would appreciate if you can give a few examples of not having free will. I asked this question on this forum once before, and some people struggled to described the sensation of not having free will, and those who did describe it gave very different descriptions. I am quite sure that your examples of lack of free will would clash with those of others, indicating that the notion is too poorly defined to be discussed productively.

This is a general problem with intuitively understood notions, people use the same term for overlapping but different concepts and/or qualia. "Existence" is one of the worst. People use it in different and contradictory ways, often in the same sentence (this is easy to detect by asking "what does it mean for to not exist?").

Anyway, just wondering about 3 examples which you personally think describe not having free will.

I think this is his conclusion:

...if we want to know which meaning to attach to a confusing sensation, we should ask why the sensation is there, and under what conditions it is present or absent.

Then I could say something like: "This sensation of freedom occurs when I believe that I can carry out, without interference, each of multiple actions, such that I do not yet know which of them I will take, but I am in the process of judging their consequences according to my emotions and morals."

This is a condition that can fail in the presence of jail cells, or a decision so overwhelmingly forced that I never perceived any uncertainty about it.

There - now my sensation of freedom indicates something coherent; and most of the time, I will have no reason to doubt the sensation's veracity. I have no problems about saying that I have "free will" appropriately defined; so long as I am out of jail, uncertain of my own future decision, and living in a lawful universe that gave me emotions and morals whose interaction determines my choices.

Certainly I do not "lack free will" if that means I am in jail, or never uncertain of my future decisions, or in a brain-state where my emotions and morals fail to determine my actions in the usual way.

Usually I don't talk about "free will" at all, of course! That would be asking for trouble - no, begging for trouble - since the other person doesn't know about my redefinition. The phrase means far too many things to far too many people, and you could make a good case for tossing it out the window.

But I generally prefer to reinterpret my sensations sensibly, as opposed to refuting a confused interpretation and then calling the sensation "false".

This sounds pretty compatibilist to me. EY gives a definition of free will that is manifestly compatible with determinism. Elsewhere in that post he argues that different definitions of free will are nonsensical and are generated by misleading intuitions.

But as the quote demonstrates, and as discussed in a different post, EY is less interested in providing a definition for free will and then asserting that people do or do not possess free will, and more interested in explaining in detail where all the intuitions about free will come from, and therefore why people talk about free will. He suggests that if you can explain what caused you to ask the question "do we have free will?" in the first place, you may not need to even bother to answer the question.

Pick a favorite ice cream flavour. Now tell me it (it's chocolate, great) and let's make a "theory of ice cream preferences". It reads "Hen favours chocolate above others". I go to a third person (let's call him Dave) and tell a story about how I got a divine mission to make a fact in the paper to come true. I wave my magic wand and afterwards Dave checks that indeed the mission was accomplished. All a natural law is a description how things happen, the law itself is not the cause of it's truth. If you try to ask "Did hen or the law of ice cream picking reduce the options to chocolate?" the answer is going to be yes, but mostly figurative for the side of law and not "literally". All natural law is descriptive and not imperative.

It is not so that people first ponder what they want to do and then pass it by the censor of "universe" in whether they are allowed to do it or not. People simply just do things. I have also seen that some hold that everything that is determined can not be the result of a free choice. This seems silly when you try to pick a ice cream flavour as making so that "Hen prefers chocolate" would rob you of your agency as now it has become "determined". It would also mean that any such choice would be impossible to make if it were to remain truly "free". Things will happen a certain way, not 0 ways or 2 ways but 1 way and that is separate from our ability to describe it and whatever happens is the way that happens and a statement describing that would be a true natural law.

1) free will is incompatible with determinism / the natural world is relevantly deterministic // we therefore do not have free will. you are wrong in the first and 3rd section. Corrected it would read like "People spout all kinds of beliefs about their ability to make choices / the natural world is relevantly deterministic / Not everything people spout can be true / What people spout is not an adequate picture of world"

or alternative version "1) free will requires determinism / natural world is relevantly determistic / we can excerise free will 2) Here is a theory why people form funny and incorrect beliefs while dealing with their ability to make choices 3) Don't let error sources cloud your understanding and use you brain to clear any fogginess up"

Maybe someone already mentioned it below, but UDT provides a formal sense in which to speak of the consequences of possible actions in a deterministic universe. I think it resolves the free will "paradox" to the extent it is meaningful at all.

I still haven't found a readable meta-overview of causation. What I would love to be able to read is a 3-10 pages article that answers these questions: what is causation, why our intuitive feeling that "A causes B" is straightforward to understand is naive (some examples), why nevertheless "A causes B" is fundamental and should be studied, what disciplines are interested in answering that question, what are the main approaches (short descriptions with simple lucid examples), which of them are orthogonal/in conflict/cooperate with each other, example of how a rigorous definition of causality is useful in some other problem, major challenges in the field.

Before I'm able to digest such a summary (or ultimately construct it in my own head from other longer sources if I'm unable to find it), I remain confused by just about every theoretical discussion of causation - without at least a vague understanding of what's known, what's unknown, what's important and what's mainstream everything sounds a little sectarian.

I sense there may be a contradiction between a decision theory that aims to be timeless and the mandate to ignore sunk costs because they're in the past. But I fear I may be terribly misunderstanding both concepts.

Hi, can you explain EDT to me (by email)? :)

Your first point was discussed in detail here. Your second point was discussed in many places on LW, most recently here, I think.