Conjecture on Addiction to Meta-level Solutions

Tom Chi presented on some related topics at EAG, part of which I like referring to as the Tom Chi question: Can this process, in principle, lead to a robust solution to the given problem domain? So kind of a handle for the concept of checking your heuristic-problem mapping.

Edit: I remember now that this is commonly referred to as the Emperor of China's Nose problem.

When we become more rational, it's usually because we invent a new cognitive rule that:

1. Explains why certain beliefs and actions lead to winning in a set of previously observed situations that all share some property; and,
2. Leads to winning in some, if not all, heretofore unforeseen situations that also share this property.

When you learn the general rule of not-arguing-over-definitions, then, in hindsight, you understand in a very general sense why humans on a desert island will draw lines in the sand to communicate if necessary instead of, with futility, mutually drawing lines that are naively intended to communicate the fact that they are dissatisfied with their respective companions' line-drawing methods. You will foresee future instances of the general failure mode as well.

When we become more rational, it's usually because we invent a new cognitive rule that:

1. Explains why certain beliefs and actions lead to winning in a set of previously observed situations that all share some property; and,
2. Leads to winning in some, if not all, unforeseen situations that also share this property.

When you learn the rule of not-arguing-over-definitions. In hindsight, you understand in a sense - why humans on a desert island will draw lines in the sand to communicate if necessary. Instead of; with futility, mutually drawing lines that are intended to communicate the fact that they are dissatisfied with their respective companions' line-drawing methods. You will expect future instances of the failure mode as well.

You might say that one possible problem statement of solving human rationality is obtaining a complete understanding of the algorithm implicit in the physical structure of our brains that allows us to generate such new and improved rules.

Because there is some algorithm. Your new cognitive rules are output, and the question is what algorithm generates them. If you explicitly understood that algorithm, then all other insights about human rationality would simply fall out of it as consequences.

You might say that one problem of solving human rationality is obtaining an understanding of the algorithm in the physical structure of our brains that allows us to generate new rules.

There is some algorithm. Your new cognitive rules are output, and the question is: "What algorithm generates them?" If you explicitly understood that algorithm, then all other insights about human rationality would simply fall out of it as consequences.

Trying to modify to make this more readable. small changes; but your original is quite hard to parse.

Editorial note: The introduction before the quote was quite confusing to me. I would prefer the article starting with the quote about alcoholics, so I know what is the main topic. Also, perhaps a summary at the end?

This is how I understood the article:

Research suggests that alcoholics use alcohol as a tool for regulating their emotions. This could be of interesting to LW users, because it is a part of metacognition, which we consider important, but probably wouldn't look at it from this specific angle. Could we also be somehow victims of our metacognitive strategies, successful (rewarded) in short term, but harmful in long term? We may be addicted to "going meta", which results in procrastination, because we habitually escape from the object level to the meta level, just like the alcoholics escape from their daily problems to alcohol. The solution for the alcoholics (easier said than done) is to find a better way to regulate their emotions. Those of us who are addicted to "going meta", i.e. those of us who procrastinate a lot, probably need something similar.

It reminds me of a friend (a LW lurker) who said: After you have gained all the necessary insights and formulated the best strategy, the remaining component is the ability to shut up and suffer. You can (and should) reduce the unpleasant work, but you often cannot reduce it to zero, and you have to accept it and endure the necessary suffering, otherwise you never get things done.

Some of what you are saying feels slightly off, but overall I understand very well what you are getting at, so I'm just going to give you credit and understand it in the best way I can.

When we become more rational, it's usually because we invent a new cognitive rule that:

Explains why certain beliefs and actions lead to winning in a set of previously observed situations that all share some property; and, Leads to winning in some, if not all, heretofore unforeseen situations that also share this property.

This is different from my subjective experience. I started describing it here, but it grew long and I thought it was interesting enough to write a separate post: http://lesswrong.com/r/discussion/lw/ney/how_it_feels_to_improve_my_rationality/

I am thinking about optimal value salvage from foregone opportunities. To illustrate the application: I recently turned down an interview offer for a poor paying (<10k AUD annually) full time job in India. The pay was the decisive in my decision. In the future should I screen jobs by salary in advance to save on application time? What is my acceptable minimum salary for full time work? Should I disclose my error to the employer, friends and/or the public (e.g. by blogging? Commenting on existing material by others? Social media?). And meta questions like is this task formulated at the optimal level or abstraction and where is its relative importance? I have not systematically answered these questions. Writing this out gives me feeling that the cumulative reward of systematically generated answers (that i expect will be highly uncertain and full non dominated options that will be harder to regress to my preferences) is dominated by the prospect of trusting my gut :) and, I reckon its more convenient and braggable to read up on the moat proximate problem domain than to reinvent the wheel. Maybe i should read about options trading in finance...

“Ultimately, it is the desire, not the desired, that we love.”

• Friedrich Nietzsche