Open Thread September, Part 3

As a Turk, I strongly believe that story is fictional.

Where and how was this ban issued? Can you give more details?

You may be hearing some fictional story based on his social reforms.

See here

And the veil, currently banned in public universities, is still very much a hot button issue. Also, a large segment of the Turkish population still wears the veil. The country is deeply divided over this issue.

I don't get it, why would prostitutes be more eager to obey the law? Especially seeing as their professional success depends on their perceived beauty?

Does anyone else agree that, as a piece of expository writing, that document sucks bigtime?

111 pages! I got through about 25 and I was wondering why Eliezer thought I needed to hear about how his four friends had decided when presented with the Newcomb's soda problem and how some people refer to this problem as Solomon's problem. So, I decided to skim ahead until he started talking about TDT. So I skimmed and skimmed.

Finally, I got to section 14, entitled "The timeless decision procedure". "Aha!", I think. "Finally." The first paragraph consists of a very long and confusing sentence which at least seems to deal with the timeless decision procedure.

The timeless decision procedure evaluates expected utility conditional upon the output of an abstract decision computation - the very same computation that is currently executing as a timeless decision procedure - and returns that output such that the universe will possess maximum expected utility, conditional upon the abstract computation returning that output.

It might be easier to understand if expressed as an equation or formula containing, you know, variables and things. So I read on, hoping to find something I can sink my teeth into. But then the second paragraph begins:

I delay the formal presentation of a timeless decision algorithm because of some significant extra steps I wish to add ...

and closes with

Before adding additional complexities, I wish to justify this critical innovation from first principles.

As far as I can tell, the remainder of this section entitled "The timeless decision procedure" consists of this justification, though not from first principles, but rather using an example. And it doesn't appear that Eliezer ever gets back to the task of providing a "formal presentation of a timeless decision algorithm".

So, I skip forward to the end, hoping to read the conclusions. Instead I find:

This manuscript was cut off here, but interested readers are suggested to look at these sources for more discussion:

Followed by a bibliography containing one entry - A chapter from a 1978 collection of articles on applications of decision theory.

"...was cut off here ..."? Give me a break!

Let me know when you get it down to a dozen pages or so.

ETA: A cleaned up copy of the paper exists with a more complete bibliography and without the "manuscript was cut off here" closing.

Thanks for the link! I just read it all. The good: it's very, very smooth reading - I know how well Eliezer can write, and even I was surprised at the quality - and it has some very lucid explanations of tricky matters (like why Pearlean causality is useful). The bad: it's kinda rambling, contains many standard sci-fi LW arguments that feel out of place in a philosophy paper, and it doesn't make any formal advances beyond what we already know here (I'd hoped to see at least one). The verdict: definitely read the first half if you're confused about this whole "decision theory" controversy, it'll get you unconfused in a pinch. Take the second half with a grain of salt because it's still very raw (unmixed metaphor award!)

I wonder if it should be reformatted in LaTeX to pass item #1 from here.

I must say I'm disappointed by the lack of rigor. On the other hand, I'm slightly relieved that he didn't beat me to any of the stuff in the decision theory document I'm writing myself. So far, I have yet to see any formalization of decision theory that I would consider usable, other my own unfinished one.

I notice there seems to be an issue with the bibliography - there's only one entry in it, but I've found at least one other citation in the text (Judea Pearl's Causality cited on page 58) that's not there. Are there any good collections of decision theory paper links out there?

I notice that the ideal causal diagram used in Part 2 (and based on pearls) is isomorphic to an example I use to teach CLIP, once you apply the substitution:

sprinkler on -> a paperclip truck has over turned
rain -> a clippy has haphazardly used up a lot of metal wire
sidewalk wet -> paperclips are scattered across the ground
sidewalk slippery -> many paperclips need to be moved to the safe zone

Thanks, Larks-- how did you find out this was available? Is there a blog post or something somewhere? I didn't see it on SIAI's blog.

I'm glad to have this to read. I was surprised to find many examples and arguments that EY hadn't given before (or at least formalized this way). I liked the Newcomb's soda problem in particular. I had been worried that EY had presented enough of his TDT justification that someone could "scoop" him, but there's a lot more depth to it. (Anyone up-to-date on the chance that he could get a PhD just for this?)

And I also appreciated that the modified the smoking lesion problem to be one where people aren't distracted by their existing knowledge of smoking, and that this was the reason for transforming the example.

I read up to ~p. 35, and I think I have a better understanding now of the relevance of time consistency and how it varies across examples.

That said, I agree with the others who say it could use some mroe polish.

I scanned it. My initial reactions:

• Surprise that the document existed;
• TL;DR;
• Surprise at the quantity of work that had gone into it.

Alas, I totally failed to see the claimed "strange singularity at the heart of decision theory".

My favourite bit was probably the speculations about agent boundaries - starting on p.108. Alas, from my POV, no mention of the wirehead problem.

Update 2011-06-26 regarding the new version. The bit that reads:

This manuscript was cut off here, but interested readers are suggested to look at these sources for more discussion:

...seems to have been deleted, and 3 pages worth of references have been added. The document seems to have had negligible additions, though - the bit on p.108 has moved back onto page 107. There seem to be a few more extra lines at the end about how "change" is a harmful concept in decision theory.

Are you implying that there is an irrational focus on cooperation? I could see how this claim could be made about Eliezer or Drescher but less so about Nesov or Wei. It's not so much a focus on the aesthetics of the shiny idea of cooperation so much as the realization that if cooperation yields the best results, our decision theory should probably cooperate. It's not so much accommodating cooperation or acausal interaction as capitalizing on them. If it's impossible in practice, then the decision theory should reflect that. Currently, it seems incredibly difficult to find or define isomorphisms between computations an agent would consider itself, though people are working on it with interesting approaches. It's the ideal we'd like our decision theory to reach.

Also, I don't believe that timeless ontology is necessary -- at least, I'm not sure that it actually changes anything decision theoretically speaking. At any rate Wei Dai's and I think others' decision theory work is being done under the assumption that the agent in question will be operating in a Tegmark multiverse (or generally some kind of ensemble universe), and the notion of time doesn't really make sense in that case, even if it does make sense in 'our' multiverse (though I don't know what postulating this 'time' thing gets you, really).

Acausal trade is just a way to capitalize on comparative advantage over vast distances... it's a brilliant and frighteningly logical idea. (I believe Carl Shulman thought it up? I'm rather jealous at any rate.) Why do you think acausal trade wouldn't be a good idea, decision theoretically speaking? Or why is the concept confused, metaphysically speaking? Practically speaking, the combinatorial explosion of potential trading partners is difficult to work with, but if a human can choose between branches in the combinatorial explosion of a multiverse via basic planning on stupid faulty hardware like brains, an AGI might very well be able to do similar simulation of trading partners in an ensemble universe (or just limit the domain, of course). (I think Vladimir Nesov came up with this analogy, or something like it.)

That forthcoming essay by me ithat is mentioned here is actually online now, and is a two-part series, but I should say that it supports an evidential approach to decision theory (with some fairly major qualifications). The two essays in this series are as follows:

Almond, P., 2010. On Causation and Correlation – Part 1: Evidential decision theory is correct. [Online] paul-almond.com. Available at: http://www.paul-almond.com/Correlation1.pdf or http://www.paul-almond.com/Correlation1.doc [Accessed 9 October 2010].

Almond, P., 2010. On Causation and Correlation – Part 2: Implications of Evidential Decision Theory. [Online] paul-almond.com. Available at: http://www.paul-almond.com/Correlation2.pdf or http://www.paul-almond.com/Correlation2.doc [Accessed 9 October 2010].

I often wish I could use the terms "transitive" "equivalence relation" "partition" and "subset", and have people understand their technical meanings.

From the linked article:

It is certainly worth considering the possibility that there is no global element in the Universal Sheaf of Theories.

This sounds like a blatant map/territory confusion. Maybe we haven't found a single theory that applies to all domains. That is, we may have to use multiple inconsistent maps, at least for now. But the territory doesn't refer to our maps to figure out what to do. The territory just does its thing.

Pardon the self-promotion, but the point that post makes is similar to the structure of understanding I outlined here. The sheaf model of knowledge is what I call a Level 2 understanding, and the level that scientists can't yet achieve for General Relativity and Quantum Mechanics.

Ordinary people go through life having different theories about love, religion, politics, when you kick a table it hurts your foot, and so on, and don’t seem to worry a bit about whether the restriction maps are compatible ...

That's what I call a Level 1 understanding.

I probably could have created a better hierarchy if I had been familiar with the sheaf concept -- sounds like an ideal ontology for an AI to have since it faciliates regeneration of knowledge (Level 3) and consilience (Level 2).

I like the idea, but he seems to be using some nonstandard terminology - IIRC, restriction maps still have to be compatible in a presheaf, no?

Edit: Or maybe he's just using "compatible" to mean "can be glued together".

Also it would be nice if you had a small amount of additional explanation of how to use the interface.

A large number of these are things where my confidence on them is much too low to bet. Almost anyone already willing to bet on them would likely have a lot more of thought and relevant data. Still, some of these look interesting enough to maybe play with.

Related: affect heuristic, affective death spirals.

To clarify the question a bit, I would consider dividing software technologies into three categories:

1. Technologies developed while the necessary computing resources were still unavailable or too expensive, which flourished later when the resources became cheap enough. For example, Alan Turing famously devised a chess program which he could only run using paper and pencil.

2. Technologies that appeared very soon after the necessary computing resources became available and cheap enough, suggesting that the basic idea was fairly straightforward after all, and it was only necessary to give smart people some palpable incentive to think about it. Examples such as the first browsers and spreadsheets would be in this category.

3. Technologies for which the necessary computing resources had been cheaply available for a long time before someone finally came up with them, suggesting an extraordinary intellectual breakthrough. I cannot think of any such examples, and it doesn't seem like anyone else in this thread can either.

This reinforces my cynical view of software technologies in general, namely that their entire progress in the last few decades has been embarrassingly limited considering the amount of intellectual power poured into them.

Here's an interesting related thought experiment that reinforces my cynicism further. Suppose that some miraculous breakthrough in 1970 enabled the production of computers equally cheap, powerful, compact, and easily networked as we have today. What do we have today in terms of software technology that the inhabitants of this hypothetical world wouldn't have by 1980?

An example might be binary search, which is pretty trivial conceptually but which took many years for a correct, bug-free algorithm to be published.

e.g. see: http://googleresearch.blogspot.com/2006/06/extra-extra-read-all-about-it-nearly.html where a bug in a popular published version of the search remained undetected for decades.

This kind of thing is particularly worrying in the context of AI, which may well need to be exactly right the first time!

Would the first spreadsheet (VisiCalc) or the first browser (Mosaic) fit your bill? As far as I know, they didn't face difficult hardware bottlenecks when they appeared.

What about BitTorrent or P2P file transfers in general? Anonymous peer to peer seems to have not emerged until 2001, or peer to peer in general until November 1998. That's a bit too far back for me to have any idea what computers were like but peer to peer file transfer is an amazing software development which could be implemented on any computers that can transfer files--at least as early as 1977.

How significant of a technology are you thinking of?

For example, I would guess that most video game emulators came about when computers were much faster than the games they were emulating--if it weren't the case that fast computers were cheaper than the emulated consoles emulators wouldn't be very popular. Further, I can guarantee you that computers easily have more power than video game consoles, so any emulator produced of the latest generation of console was written when computers had far more power than necessary.

So: Does a new emulator count? It's a specific technology that is developed in a fast environment. Does an old emulator count? Emulators in general aren't new technology at all. Does an instant messenger count? Predecessors existed in times when text content was a big deal, but I would be mildly surprised to hear that the original AIM (or whatever the first instant messenger program was) was created at a time when text-over-the-internet was a big stress on computers.

Relatedly, I wish I could remember what I read recently about comparing the performance of a 2000s algorithm for a particular problem (factoring?) on 1980s hardware to a 1980s algorithm on 2000s hardware. It might've been on Accelerating Future.

Many "killer apps" would fit your criteria, I think. VisiCalc, AutoCAD, Mosaic.

Interesting question. I can't think of any examples. Do you want to argue that hardware is the limiting factor, the bottleneck of progress in IT? Thinking about your question for five minutes has made this conclusion much more plausible to me.

"In the 156845 times this simulation has run, players have won $1686353 And by won I mean they have won back $1686353 of the $156845 they spent (1075%)."

"In the 590873 times this simulation has run, players have won $2761902 And by won I mean they have won back $2761902 of the $590873 they spent (467%)."

"In the 842587 times this simulation has run, players have won $2788774 And by won I mean they have won back $2788774 of the $842587 they spent (330%)."

This part seems to fluctuate pretty wildly. But it's a very cool and intuitive way to show people the low chance of them winning the lottery.

someone should really top-level post that in the next 12 hours or so. it could be you, reader of this comment.

Is it supposed to replace the open threads?

LW now has a discussion section that serves as a permanent open thread. The link is at the top right, next to the link to the wiki.

Don't forget that if it works, you probably get immortality too. If you were already immortal, would you be willing to become mortal for $500 000?

The point of the insurance isn't to help you. The point of using insurance is because there were problems with the early cryonics organizations where people were prepped for cryonics with money supposedly going to come from their estates and then the money never materialized. The insurance makes sure that the organizations get enough funds. It doesn't make things less expensive for the person to be preserved.

I can't downvote either. I don't think I've used up my allotment.

It seems that for three days now, downvotes don't work on Less Wrong.

http://lesswrong.com/lw/2r4/open_thread_september_part_3/2pjr?c=1

Links to studies where scientists used a particular framework obviously doesn't count. Is there any evidence constantly challenging your willpower makes it stronger in the long run?

Well. Now I don't know what to say.

Should I model willpower as a muscle or as a battery?

I'm firmly in the "muscle" camp. Here's why:

When you use a muscle a lot, it tires, and you need to rest for a while before exercising it again. This is the part that resembles a battery, giving rise to that model. The difference is that, after going through this process several times, the muscle's capacity for use is greater, whereas the battery's would be smaller. In my experience, exercising willpower carefully makes it easier to use it in the future. As a fellow once said, getting better at skills is a skill, which you can get better at.

So, by that model, the problem with this oft-referenced comic is obvious. The author is doing the equivalent of going to the gym and lifting the heaviest weights she can for as long as she can stand, then going back the next day and trying to do the same thing. Of course she crashes. The way to get stronger is to push at the edge of your comfort zone only a little bit, keep doing it until it becomes comfortable, and then push a little more next time. Ask anyone who plays an instrument. You don't rush through the tricky section over and over and expect to learn it; you slow it down, break it up until it's only a little harder than what you've been doing, and then work up to play speed again. And what do they call the thing you build up that way? Muscle memory.

I lean toward battery. I'm unaware of any 'willpower muscle' evidence.

Let's make them wear hooded robes and call them Confessors.

In the discussion on Prices or Bindings, EY mentioned that it may help to organize as something that is legally a "church" (apparently there can be atheist churches) so that you can give a vow of secrecy that's stronger than psychologists can give.

Like Rational-Emotive Behavior Therapy?

Basically the idea behind REBT is to correct irrational thoughts and point out errors in thinking. One of my professors, a clinician, said he had great success treating panic disorder with agoraphobia using REBT.

Plato, Not Prozac

It's interesting to look at people's arguments against paperclip maximizers. There seem to be two related categories that make up most of the objections:

1. People who can't imagine that a sufficiently intelligent being could be that different from us. One guy tried to claim that morality is universal so of course an artificial intelligence would share our values. Another said that a superintelligence would inevitably realize that its existence was pointless, as if there could possibly be some point other than maximizing the number of paperclips. Another claimed that morality is an "emergent phenomenon", but didn't explain what that actually meant, or how humanlike morality would emerge from a being whose only goal is paperclip maximization.

2. People who think of it as a dumb machine, more akin to a drill press than an alien god. Just put an off button on it! Or require that there be a human operator with an axe ready to cut the power lines to the computer.

What these objections both have in common is that they assume that the world consists of humanlike intelligences or dumb machines. It's unintuitive to imagine something that is both intelligent and profoundly alien.

Could you elaborate on what you mean by status and how you reached the belief that it is relevant to your studying?

While being a professor has a certain amount of status in the eyes of the outside world, I'm not sure one should attribute this status to "being good at math"; indeed, it would probably be easiest to transform math into academic prestige through some other field, like bioinformatics. An anecdote about status and the outside world: there is a U of Chicago professor who is bitter that his parents think it a state school.
By being good at math, you'll gain status in the eyes of mathematicians. I think most people have the opposite problem, that being in a graduate program, surrounded by people who care about the subject causes them to think the world cares about it.

I recently started graduate school in mathematics, and I have been having some difficulty focusing on studying.

Difficulties focusing on studying math are common. There's an issue of many existing expositions being markedly suboptimal with respect to engaging the student.

I'm in my fourth year of math graduate school and I would be happy to offer suggestions specific to your situation. Feel free to PM me with your email address if you'd like to correspond.

Well, you will gain status by being good at math, unless you have a more prestigious opportunity than "professor" lined up, in which case you should take that instead.

Interestingly, they claim that choking is due to poor use of working memory:

Talented people often have the most working memory, but when worries creep up, the working memory they normally use to succeed becomes overburdened. People lose the brain power necessary to excel.

I'd like to see someone compare college students' performance on important tests after, say, 0--3 drinks. If test anxiety hurts people's scores as much as it seems to, then perhaps cheap beer will be used as a nootropic.

(A quick check on Google Scholar doesn't show any studies that have been done on this, which isn't surprising.)

Yep.

Same here. And upvotes do not have the same problem, so I don't think it's a caching issue.

This happens to me, as well.

Modafinil increased my blitz chess rating by 150 points the first time I took it. That is completely ridiculous. The effects weren't as strong after that, but still very noticeable. It definitely worked to keep me awake for long hours. One time it totally fucked me up for 3 days or so: I took it thinking I'd stay up, but then I was like nah, never mind, I'll go to sleep 'cuz I'm really tired. Like an idiot. I then got 5 or so hours of fitful sleep followed by about 5 more hours of thrashing about on my mattress follow by about 10 more hours of groggy sleep deprivation-induced pain followed by a completely messed up sleep cycle for the next few days of constant tiredness. So, it's awesome when you use it right, but be careful to stay up for the duration of the effects (12 hours or so, I think.)

I am quite obviously not a doctor, though.

I would point out that if you are interested in general information, there are many larger sites to search for information. Some quick links:

• http://www.erowid.org/
• http://www.imminst.org/forum/
• http://www.reddit.com/r/Nootropics/
• http://www.bluelight.ru/

It would be better if you asked on LW only if you had something more specific (for example, if you had developed a theory of akrasia and thought a particular nootropic might help as has been done with sulbutiamine).

(For completeness, I keep a record of my nootropics in http://www.gwern.net/Nootropics )