Suggestions for naming a class of decision theories

post by orthonormal · 2012-03-17T17:22:54.160Z · LW · GW · Legacy · 58 comments

In my recent post, I outlined 5 conditions that I'd like a decision theory to pass; TDT, UDT and ADT pass them, while CDT and EDT don't. I called decision theories that passed those conditions "advanced decision theories", but that's probably not an optimal name. Can I ask you to brainstorm some other suggestions for me? (I may be writing a follow-up soon.)

As usual, it's best to brainstorm on your own before reading any of the comments. You can write down your ideas, then check if any have already been suggested, then comment with the new ones.

Thanks!

58 comments

Comments sorted by top scores.

comment by Peter Wildeford (peter_hurford) · 2012-03-18T20:57:50.216Z · LW(p) · GW(p)

Robust decision theory?

comment by Vladimir_Nesov · 2012-03-17T21:32:49.829Z · LW(p) · GW(p)

I think "TDT-like decision theories" is adequate...

Replies from: army1987
comment by A1987dM (army1987) · 2012-03-17T23:49:35.117Z · LW(p) · GW(p)

That's what I've used so far, but I'd prefer something better.

comment by Tyrrell_McAllister · 2012-03-20T21:01:53.554Z · LW(p) · GW(p)

"Logical decision theories"?

One quality shared by these theories is that they are concerned about the logical implications (within some mathematical model of the world) that would follow from the hypothesis that the agent chooses a given action.

Replies from: Vladimir_Nesov
comment by Vladimir_Nesov · 2012-03-20T21:14:16.587Z · LW(p) · GW(p)

Not necessarily, a variant of TDT could work without logical specification of decisions, it only needs some way of referring to the decision as part of the environment, and tools for figuring out what other facts about the environment follow from the fact that is the decision. So, "self-referential", "reflexive" and "non-independent" could work, but "logical" doesn't seem to capture what's going on. See this discussion between Wei Dai and myself.

Replies from: Tyrrell_McAllister
comment by Tyrrell_McAllister · 2012-03-20T22:24:23.531Z · LW(p) · GW(p)

Not necessarily, a variant of TDT could work without logical specification of decisions ...

Would you give a concrete example of what you mean? Right now, I seem to be in the same place where Wei Dai was when he wrote

And you're still using it as a logical fact, i.e., deducing logical consequences from it, right?

I feel like you must be making a point that I'm not getting...

I'm also not seeing the relevance of your reply to him.

When you write:

... it only needs some way of referring to the decision as part of the environment, and tools for figuring out what other facts about the environment follow from the fact that is the decision ...

... in what sense do you understand the "other facts" to "follow from" the "fact that is the decision"? What does the agent work with to determine whether one "fact" "follows from" another?

I agree that the agent needn't be working with strings in some formal language. But, in the decision theories under consideration, the agent is working with some mathematical model of the world, which the agent uses to infer what follows logically from the premise that the agent decides to do X *. I agree also that the agent need not be using a first-order predicate logic to make this inference. Nonetheless, it still seems correct to me to say that what the agent is inferring is a relationship of logical implication.

By analogy, Euclid didn't use a formal first-order predicate logic, but he was still inferring relationships of logical implication.


* I am least familiar with TDT among the decision theories being considered, so, if this statement is wrong, it is most likely wrong about TDT.

comment by gRR · 2012-03-17T21:56:56.190Z · LW(p) · GW(p)

Renormalized DT-s?

comment by orthonormal · 2012-03-17T17:57:05.730Z · LW(p) · GW(p)

Results of my brainstorming, threaded into separate comments for voting. Note karma sink at the end.

Replies from: orthonormal, orthonormal, army1987, orthonormal, orthonormal, orthonormal, orthonormal, orthonormal, orthonormal, orthonormal, orthonormal, orthonormal
comment by orthonormal · 2012-03-17T17:57:41.060Z · LW(p) · GW(p)

Reflexive decision theories

Replies from: wedrifid, Richard_Kennaway
comment by wedrifid · 2012-03-18T12:49:37.378Z · LW(p) · GW(p)

After googling Reflexive the only problem I have with using "Reflexive decision theories" is that it may prevent someone using Reflexive Decision Theory for an actual instance of a well developed decision theory. The name is perfect. (And the name UDT is just STUPID! It only makes sense by redefining 'update'.)

Replies from: Wei_Dai
comment by Wei Dai (Wei_Dai) · 2012-04-24T21:47:29.654Z · LW(p) · GW(p)

And the name UDT is just STUPID!

I hope you don't blame me too much for that, because it was Eliezer who gave it that name, and I was only complicit to the extent that I didn't object.

It only makes sense by redefining 'update'.

I think "updateless" was meant to refer to the fact that UDT doesn't do explicit Bayesian updating, which doesn't seem unreasonable to me...

(BTW I remember you once said you also didn't like the language I used to describe UDT, but didn't say why when I asked. I'm still curious about that.)

Replies from: wedrifid
comment by wedrifid · 2012-04-24T22:22:10.792Z · LW(p) · GW(p)

I hope you don't blame me too much for that

Not at all, and your work on the theory was excellent. I just hope that by the time the descendent decision theories evolve to a stable state that they have picked up a new moniker. The reflexive term sounds much more significant!

BTW I remember you once said you also didn't like the language I used to describe UDT, but didn't say why when I asked.

I think I would put less emphasis on subtracting updates and more on just which kind of information should be used. After all, not doing updates isn't the important thing (or a sufficient thing), it is that the right piece of information is used at the end.

I'm afraid I've actually been negligent in my decision theory reading. I've actually forgotten a lot since I originally read your work from - what was it? - two years ago or so. I wouldn't really have high confidence in my words if I tried to really explore the issues in detail these days.

Edit: Also, wedrifid_today considers the wording (and punctuation) used by wedrifid_last_month to be rather more hyperbolic than he would endorse.

Replies from: Vladimir_Nesov
comment by Vladimir_Nesov · 2012-04-24T22:31:52.777Z · LW(p) · GW(p)

After all, not doing updates isn't the important thing, it is that the right piece of information is used at the end.

It's an important thing, in that we don't know how to do updates without getting misled in some strange situations. UDT uses other sources of information, and shows how that's sufficient in principle, but the current puzzle is how to make use of the information that UDT doesn't use, avoiding UDT's logical cornucopia (so that one has to deal with resulting logical uncertainty, and resolve it to a limited extent based on observations).

comment by Richard_Kennaway · 2012-03-17T23:33:36.364Z · LW(p) · GW(p)

Snap!

Most of the others just say "better" instead of saying what the desired sort of theory actually does.

Replies from: hairyfigment, orthonormal
comment by hairyfigment · 2012-03-18T03:31:30.384Z · LW(p) · GW(p)

Then what do we call decision theories capable of proving statements about self-modification?

Replies from: Richard_Kennaway
comment by Richard_Kennaway · 2012-03-18T08:19:40.105Z · LW(p) · GW(p)

You could include them in reflexive decision theories. When formalised, I suspect there may not be much of a distinction between theories that can know about themselves and theories that can modify themselves.

Replies from: orthonormal
comment by orthonormal · 2012-03-18T16:05:30.603Z · LW(p) · GW(p)

As a mathematician, I prefer using different notations for things until I've proved they're identical. And in this case, I rather suspect they're not.

comment by orthonormal · 2012-03-18T16:14:21.065Z · LW(p) · GW(p)

Most of the others just say "better" instead of saying what the desired sort of theory actually does.

Actually, it's precisely for this reason that "reflexive" doesn't work for what I originally wanted: I wanted to use a word to denote TDT, UDT, ADT, etc, not in terms of how they work but in terms of satisfying the five conditions, the most important of which is that they out-perform CDT. How a decision theory works is a secondary consideration to how well it works.

comment by orthonormal · 2012-03-17T17:58:35.390Z · LW(p) · GW(p)

Superrational decision theories

Replies from: orthonormal, army1987, Giles
comment by orthonormal · 2012-03-17T21:19:33.192Z · LW(p) · GW(p)

(By the way, I worry that this phrase would sound cultish to anyone who wasn't familiar with Hofstadter's superrationality.)

Replies from: army1987, wedrifid
comment by A1987dM (army1987) · 2012-03-17T23:41:25.409Z · LW(p) · GW(p)

Yeah. I had thought about the bad connotations, then remembered that the usual superlative of "rationality" around here is "x-rationality". But I hadn't thought that this doesn't change how an outsider would perceive the phrase.

comment by wedrifid · 2012-03-18T12:46:09.589Z · LW(p) · GW(p)

By the way, I worry that this phrase would sound cultish to anyone who wasn't familiar with Hofstadter's superrationality.

I learned superrationality in school. Not the name Hofstadter or anything about why superrationality is sane though.

The only problem with 'superrationality' is the connotation that CDT is rational. It's just not!

comment by A1987dM (army1987) · 2012-03-17T19:19:48.352Z · LW(p) · GW(p)

While I'm not 100% sure this is the best possible name (not that I can think of anything better myself right now), at least it has precedent (Hofstadter's superrationality, which is the only ‘mainstream’ --i.e. notable enough for a Wikipedia article-- name for something similar -- unless you count the categorical imperative, but IMO the name categorical decision theory would be better for one specific theory than for a class.)

comment by Giles · 2012-04-01T03:36:00.512Z · LW(p) · GW(p)

Rational decision theories

comment by A1987dM (army1987) · 2012-03-17T19:13:03.790Z · LW(p) · GW(p)

It's a three-value range voting (-1, 0, 1), right? Or am I supposed to choose only one? (I've downvoted three and upvoted two plus the karma sink.)

Replies from: orthonormal
comment by orthonormal · 2012-03-17T19:28:14.648Z · LW(p) · GW(p)

Yes, with this many options I think three-range voting is better than approval (0, 1) voting. We don't want to settle on one that 2/3 of people love and 1/3 of people hate, rather than one that 1/2 of people love and 1/2 of people are indifferent toward.

Replies from: army1987
comment by A1987dM (army1987) · 2012-03-17T20:11:57.461Z · LW(p) · GW(p)

The problem is that the karma sink doesn't work if abs(number of upvotes - number of downvotes) > 1...

comment by orthonormal · 2012-03-17T18:00:25.049Z · LW(p) · GW(p)

Non-independent decision theories

comment by orthonormal · 2012-03-17T17:59:34.166Z · LW(p) · GW(p)

Agreeable decision theories

comment by orthonormal · 2012-03-17T17:57:27.033Z · LW(p) · GW(p)

Advanced decision theories

comment by orthonormal · 2012-03-17T17:59:57.007Z · LW(p) · GW(p)

Acute decision theories

comment by orthonormal · 2012-03-17T17:58:13.581Z · LW(p) · GW(p)

Good decision theories

comment by orthonormal · 2012-03-17T17:59:15.975Z · LW(p) · GW(p)

Sapient decision theories

comment by orthonormal · 2012-03-17T17:57:59.517Z · LW(p) · GW(p)

Non-stupid decision theories

comment by orthonormal · 2012-03-17T17:58:53.689Z · LW(p) · GW(p)

Clever decision theories

comment by orthonormal · 2012-03-17T18:02:06.296Z · LW(p) · GW(p)

KARMA SINK: I played chicken with the universe and lost!

comment by vi21maobk9vp · 2012-03-20T06:13:30.445Z · LW(p) · GW(p)

Simulation-stable decision theories.

comment by Dmytry · 2012-03-18T16:38:17.220Z · LW(p) · GW(p)

Okay, I have a question: what will you call the decision theory where x is the action, I write down f(x) which is the pay-off, like in applied mathematics, and then I solve it for maximum f(x) using regular school algebra? I use that a lot when writing AIs for the computer game, really (when I want to find the direction in which to shoot, for instance, or want to turn minimizing the 3rd derivative)

Then I don't need to go into any recursion what so ever if i have conditionals on x inside the pay-offs (as in newcomb). I don't do some update cycle to solve the x=x , i know it doesn't fix x, and i don't recurse if i find x=1+x/2

BTW, an observation on the newcomb's problem: it seems to me that one boxing people write payoff function as

f(1)=box1(1)

f(2)=box2(2)+box1(2)

box1(1)=1000000

box1(2)=0

box2(1)=1000

box2(2)=1000

and one-box, other people write it as:

f(1)=box1

f(2)=box2+box1

box1>=0

box2>=0

and ignore the fact about prediction (boxes as function of x) altogether because they trust a world model where this is forbidden more than they trust the problem statement, which is kind of silly thing to do when you're solving a hypothetical anyway. Or maybe because they listen harder to 'the contents of boxes are fixed' than to 'prediction'. In any case to me the 1-boxing vs 2-boxing now looks like a trivial case of 'people can disagree how to transform verbal sentence into a model' . Given that there's as many versions of English as there are people speaking English, it's not very interesting. One can postulate the both boxes being transparent to make even more nonsensical version.

Replies from: orthonormal
comment by orthonormal · 2012-03-19T14:41:43.640Z · LW(p) · GW(p)

Basically, all of the decision theories are just deducing payoffs and calculating argmax, but there's a subtle complication with regard to the deduction of payoffs. I'm almost done with the post that explains it.

Replies from: Dmytry
comment by Dmytry · 2012-03-19T16:54:40.550Z · LW(p) · GW(p)

Well, you guys instead of using x for the choice and doing algebra to handle x on both sides of equations, start going meta and considering yourselves inside simulators, which, albeit intellectually stimulating, is unnecessary and makes it hard for you to think straight.

If I needed to calculate ideal orientation of a gun assuming that the enemy can predict orientation of a gun perfectly, i'd just use x for the orientation, and solve for both ballistics and enemy's evasive action.

Also, the newcomb's now sounds to me like simple case of alternative english to math conversions when processing the problem statement, not even a case of calculating anything differently. There's the prediction, but there's also the box contents being constant., you can't put both into math, you can in English but human languages are screwy and we all know it.

Replies from: orthonormal
comment by orthonormal · 2012-03-24T16:41:27.080Z · LW(p) · GW(p)

I finished the post that explains the problem with the decision theory you proposed- calculating payoffs in the most direct way risks spurious counterfactuals. (I hope you don't mind that I called it "naive decision theory", since you yourself said it seemed like the obvious straightforward thing to do.)

comment by Grognor · 2012-03-18T03:31:40.272Z · LW(p) · GW(p)

"Strong" decision theories, using the word "strong" as in, "strong principle of charity" or "strong atheism".

Replies from: Grognor
comment by Grognor · 2012-03-18T03:32:13.486Z · LW(p) · GW(p)

"Hard" decision theories, as in you can't break them with silly problems like newcomb's.

comment by lavalamp · 2012-03-17T23:14:13.520Z · LW(p) · GW(p)

"orthonormal-complete"

Replies from: orthonormal
comment by orthonormal · 2012-03-18T00:02:41.151Z · LW(p) · GW(p)

Haha no, that's #25 on this list.

comment by Dmytry · 2012-03-17T21:48:57.162Z · LW(p) · GW(p)

I wonder what sort of decision theory would try to generate a bunch of decision theories and evaluate relative performance of the decision theories on the problems identified as, well, problematic, and come up with a decision theory that performs the best. You can probably even formalize this with bounded form of Solomonoff induction - iterate over the decision theories, pick the best - except that you need to really formalize what a decision theory is.

Replies from: army1987
comment by A1987dM (army1987) · 2012-03-17T23:37:35.935Z · LW(p) · GW(p)

http://lesswrong.com/lw/15z/ingredients_of_timeless_decision_theory/1217

Replies from: Dmytry
comment by Dmytry · 2012-03-17T23:50:48.512Z · LW(p) · GW(p)

Still not quite it.

BTW, an observation: if i want to maximize the distance at which the thrown stone lands, assuming constant initial speed and zero height, I work out the algebra - I have unknown, x, the shoot angle, and I have laws of physics that express distance as function of x, and I find best x. In newcomb's, I have x=my choice, I have been given rules of the world, whereby the payoff formula includes the x itself, i calculate best x, which is one-box (not surprisingly). The smoking lesion also works fine. Once you stop invoking your built-in decision theory on confusing cases, things are plain and clear.

At this point, how well you perform depends to what sort of axiom system you are using to solve for x, and by Godel's theorem, there will be some problem that is going to get ya, i.e. cause failure.

Replies from: endoself
comment by endoself · 2012-03-18T00:19:23.983Z · LW(p) · GW(p)

At this point, how well you perform depends to what sort of axiom system you are using to solve for x, and by Godel's theorem, there will be some problem that is going to get ya, i.e. cause failure.

This doesn't seem like something that needs to be solvable. You can you diagonalization to defeat any decision theory; just award some utility iff the agent chooses the option not recommended by that decision theory. A different decision theory can choose the other option, but that decision theory has acausal influence over the right answer that prevents it from winning.

Replies from: Dmytry
comment by Dmytry · 2012-03-18T03:19:12.260Z · LW(p) · GW(p)

Yep. Just wanted to mention that every theory where you can do diagonalization, i.e. every formal one, can be defeated.

My point is that one could just make the choice be x, then express the payoff in terms of x, then solve for x that gives maximum payoff, using the methods of algebra, instead of trying to redefine algebra in some stupid sense of iteration of values of x until finding an equality (then omg it fails at x=x), and trying to reinvent already existent reasoning (in form of theorem proving).

comment by hairyfigment · 2012-03-17T20:06:28.805Z · LW(p) · GW(p)

Improved decision theories. This only works as a temporary name (hopefully) but people could still use it in historical accounts. And it has fewer syllables than "superrational". That matters, even in Newton's Experimental Philosophy.

Replies from: hairyfigment
comment by hairyfigment · 2012-03-17T20:09:48.439Z · LW(p) · GW(p)

Karma balance.

comment by Larks · 2012-03-18T11:23:39.556Z · LW(p) · GW(p)

Less Wrong Decision Theories

(in both senses of the phrase, perhaps)

Replies from: orthonormal
comment by orthonormal · 2012-03-18T16:08:02.768Z · LW(p) · GW(p)

Except that some of the variants were developed before Less Wrong existed. Gary Drescher, as much as he likes this site, might rightly object.

Replies from: Larks
comment by Larks · 2012-03-18T23:51:31.858Z · LW(p) · GW(p)

Drescherian Decision Theories

(unfortunately you can't easily mix "Less Wrong" with his name)

Replies from: orthonormal
comment by orthonormal · 2012-03-19T00:05:22.660Z · LW(p) · GW(p)

Jokingly:

"Good, Real, and Less Wrong Decision Theories"

comment by A1987dM (army1987) · 2012-03-17T23:51:58.781Z · LW(p) · GW(p)

Metarational decision theory

Replies from: army1987
comment by A1987dM (army1987) · 2012-03-17T23:53:24.177Z · LW(p) · GW(p)

Karma balance