The ideas you're not ready to post

The Dilbert Challenge: you are working in a company in the world of Dilbert. Your pointy-haired boss comes to you with the following demand:

"One year from today, our most important customer will deliver us a request for a high-quality reliable software system. Your job and the fate of the company depends on being able to develop and deploy that software system within two weeks of receipt of the specifications. Unfortunately we don't currently know any of the requirements. Get started now."

I submit that this preposterous demand is really a deep intellectual challenge, the basic form of which arises in many different endeavors. For example, it's reasonable to believe that at some point in the future, humanity will face an existential threat. Given that we will not know the exact nature of that threat until it's almost upon us, how can we prepare for it today?

On the Care and Feeding of Rationalist Hardware

Many words have been spent here in improving rationalist software -- training patterns of thought which will help us to achieve truth, and reliably reach our goals.

Assuming we can still remember so far back, Eliezer once wrote:

But if you have a brain, with cortical and subcortical areas in the appropriate places, you might be able to learn to use it properly. If you're a fast learner, you might learn faster - but the art of rationality isn't about that; it's about training brain machinery we all have in common

Rationality does not require big impressive brains any more than the martial arts require big bulging muscles. Nonetheless, I think it would be rare indeed to see a master of the martial arts willfully neglecting the care of his body. Martial artists of the wisest schools strive to improve their bodies. They jog, or lift weights. They probably do not smoke, or eat unhealthily. They take care of their hardware so that the things they do will be as easy as possible.

So, what hacks exist which enable us to improve and secure the condition of our mental hardware? Some important areas that come to mind are:

• sleep
• diet
• practice

Incommensurate thoughts: People with different life-experiences are literally incapable of understanding each other, because they compress information differently.

Analogy: Take some problem domain in which each data point is a 500-dimensional vector. Take a big set of 500D vectors and apply PCA to them to get a new reduced space of 25 dimensions. Store all data in the 25D space, and operate on it in that space.

Two programs exposed to different sets of 500D vectors, which differ in a biased way, will construct different basic vectors during PCA, and so will reduce all vectors in the future into a different 25D space.

In just this way, two people with life experiences that differ in a biased way (due to eg socioeconomic status, country of birth, culture) will construct different underlying compression schemes. You can give them each a text with the same words in it, but the representations that each constructs internally are incommensurate; they exist in different spaces, which introduce different errors. When they reason on their compressed data, they will reach different conclusions, even if they are using the same reasoning algorithms and are executing them flawlessly. Futhermore, it would be very hard for them to discover this, since the compression scheme is unconscious. They would be more likely to believe that the other person is lying, nefarious, or stupid.

There is a topic I have in mind that could potentially require writing a rather large amount, and I don't want to do that unless there is some interest, rather than suddenly dumping a massive essay on LW without any prior context. The topic is control theory (the engineering discipline, not anything else those words might suggest). Living organisms are, I say (following Bill Powers, who I've mentioned before) built of control systems, and any study of people that does not take that into account is unlikely to progress very far. Among the things I might write about are these:

• Purposes and intentions are the set-points of control systems. This is not a metaphor or an analogy.

• Perceptions do not determine actions; instead, actions determine perceptions. (If that seems either unexceptionable or obscure, try substituting "stimulus" for "perception" and "response" for "action".)

• Control systems do not, in general, work by predicting what action will produce the intended perception. They need not make any predictions at all, nor contain any model of their environment. They require neither utility measures, nor Bayesian or any other form of inference. There are methods of designing control systems that use these concepts but they are not inherent to the nature of control.

• Inner conflict is, literally, a conflict between control systems that are trying to hold the same variable in two different states.

• How control systems behave is not intuitively obvious, until one has studied control systems.

This is the only approach to the study of human nature I have encountered that does not appear to me to mistake what it looks like from the inside for the underlying mechanism.

What say you all? Vote this up or down if you want, but comments will be more useful to me.

We are Eliza: A whole lot of what we think is reasoned debate is pattern-matching on other people's sentences, without ever parsing them.

I wrote a bit about this in 1998.

But I'm not as enthused about this topic as I was then, because then I believed that parsing a sentence was reasonable. Now I believe that humans don't parse sentences even when reading carefully. The bird the cat the dog chased chased flew. Any linguist today would tell you that's a perfectly fine English sentence. It isn't. And if people don't parse grammatic structures to just 2 levels of recursion, I doubt recursion, and generative grammars, are involved at all.

I'm kind of thinking of doing a series of posts gently spelling out step by step the arguments for Bayesian decision theory. Part of this is for myself: I've read a while back Omohundro's vulnerability argument, but felt there were missing bits that I had to personally fill in, assumptions I had to sit and think on before I could really say "yes, obviously that has to be true". Some things that I think I can generalize a bit or restate a bit, etc.

So as much as for myself, to organize and clear that up, as for others, I want to do a short series of "How not to be stupid (given unbounded computational power)" In which in each each post I focus on one or a small number of related rules/principles of Bayesian Decision theory and epistemic probabilities, and gently derive those from the "don't be stupid" principle. (Again, based on Omohundro's vulnerability arguments and the usual dutch book arguments for Bayesian stuff, but stretched out and filled in with the details that I personally felt the need to work out, that I felt were missing.)

And I want to do it as a series, rather than a single blob post so I can step by step focus on a small chunk of the problem and make it easier to reference related rules and so on.

Would this be of any use to anyone here though? (maybe a good sequence for beginners, to show one reason why Bayes and Decision Theory is the Right Way?) Or would it be more clutter than anything else?

This doesn't even have an ending, but since I'm just emptying out the drafts folder

Memetic Parasitism

I heard a rather infuriating commercial on the radio today. There's no need for me to recount it directly -- we've all heard the type. The narrator spoke of the joy a woman feels in her husband's proposal, of how long she'll remember its particulars, and then, for no apparent reason, transitioned from this to a discussion of shiny rocks, and where we might think of purchasing them.

I hardly think I need to belabor the point, but there is no natural connection between shiny rocks and promises of monogamy. There was not even any particularly strong empirical connection between the two until about a hundred years ago, when some men who made their fortunes selling shiny rocks decided to program us to believe there was.

What we see here is what I shall call memetic parasitism. We carry certain ideas, certain concepts, certain memes to which we attach high emotional valence. In this case, that meme is romantic love, expressed through monogamy. An external agent contrives to derive some benefit by attaching itself to that meme.

Now, it is important to note when describing a Dark pattern that not everything which resembles this parttern is necessarily dark. Carnation attempts to connect itself in our minds to the Burns and Allen show. Well, on reflection, it seems this is right. Carnation did bring us the Burns and Allen show. It paid the salary of each actor, each writer, each technician, who created the show each week. Carnation deserves our gratitude, and any custom which may result from it. Romantic love existed for many centuries before the shiny-rock-sellers came along, and they have done nothing to enhance it.

Of course, I think most of us have seen this pattern before. This comic makes the point rather well, I think.

So, right now, I know that the shiny-rock-sellers want to exploit me, this outrages me, and I choose to have nothing to do with them. How do we excite people's shock and outrage at the way the religions have tried to exploit them?

Buddhism.

What it gets wrong. Supernatural stuff - rebirth, karma in the magic sense, prayer. Thinking Buddha's cosmology was ever meant as anything more than an illustrative fable. Renunciation. Equating positive and negative emotions with grasping. Equating the mind with the chatty mind.

What it gets right. Meditation. Karma as consequences. There is no self, consciousness is a brain subsystem, emphasis on the "sub" (Cf. Drescher's "Cartesian Camcorder" and psychology's "system two"). The chatty mind is full of crap and a huge waste of time, unless used correctly. Correct usage includes noticing mostly-subconscious thought loops (Cf. cognitive behavioral therapy). A lot of everyday unreason does stem from grasping, which roughly equates to "magical thinking" or the idea that non-acknowledgment of reality can change it. This includes various vices and dark emotions, including the ones that screw up attempted rationality.

What rationalists should do. Meditate. Notice themselves thinking. Recognize grasping as a mechanism. Look for useful stuff in Buddhism.

Why I can't post. Not enough of an expert. Not able to meditate myself yet.

Aumann agreements are pure fiction; they have no real-world applications. The main problem isn't that no one is a pure Bayesian. There are 3 bigger problems:

• The Bayesians have to divide the world up into symbols in exactly the same way. Since humans (and any intelligent entity that isn't a lookup table) compress information based on their experience, this can't be contemplated until the day when we derive more of our mind's sensory experience from others than from ourselves.
• Bayesian inference is slow; pure Bayesians would likely be outcompeted by groups that used faster, less-precise reasoning methods, which are not guaranteed to reach agreement. It is unlikely that this limitation can ever be overcome.
• In the name of efficiency, different reasoners would be highly orthogonal, having different knowledge, different knowledge compression schemes and concepts, etc.; reducing the chances of reaching agreement. (In other words: If two reasoners always agree, you can eliminate one of them.)

This would probably have to wait until May.

I think there's a post somewhere in the following observation, but I'm at a loss as to what lesson to take away from it, or how to present it:

Wherever I work I rapidly gain a reputation for being both a joker and highly intelligent. It seems that I typically act in such a way that when I say something stupid, my co-workers classify it as a joke, and when I say something deep, they classify it as a sign of my intelligence. As best I can figure, its because at one company I was strongly encouraged to think 'outside the box' and one good technique I found for that was to just blurt out the first technological idea that occurred to me when presented with a technological problem, but to do so in a non-serious tone of voice. Often enough the idea is one that nobody else has thought of, or automatically dismissed for what, in retrospect, were insufficient reasons. Other times its so obviously stupid an idea that everyone thinks I'm making a joke. It doesn't hurt that often I do deliberately joke.

I don't know if this is a technique others should adopt or not, but I've found it has made me far less afraid of appearing stupid when presenting ideas.

Willpower building as a fundamental art. And some of the less obvious pit falls. Including the dangers of akrasia circumvention techniques which simply shunt willpower from one place to another and overstraining damaging your willpower reserves.

I need to hunt back down some of the cognitive science research on this before I feel comfortable posting it.

Some bad ideas on the theme "living to win":

• Murder is okay. There are consequences, but it's a valid move nonetheless.
• Was is fun. In fact, it's some of the best fun you can have as long as you don't get disabled or killed permanently.
• Being a cult leader is a winning move.
• Learn and practice the so called dark arts!

What would a distinctively rationalist style of government look like? Cf. Dune's Bene Gesserit government by jury, what if a quorum of rationalists reaching Aumann Agreement could make a binding decision?

What mechanisms could be put in place to stop politics being a mind-killer?

Why not posted: undeveloped idea, and I don't know the math.

I'm vaguely considering doing a post about skeptics. It seems to me they might embody a species of pseudo-rationality, like Objectivists and Spock. (Though it occurs to me that if we define "S-rationality" as "being free from the belief distortions caused by emotion", then "S-rationality" is both worthwhile and something that Spock genuinely possesses.) If their supposed critical thinking skills allow them to disbelieve in some bad ideas like ghosts, Gods, homeopathy, UFOs, and Bigfoot, but also in some good ideas like cryonics and not in other bad ideas like extraterrestrial contact, ecological footprints, p-values, and quantum collapse, then how does the whole thing differ from loyalty to the scientific community? Loyalty to the scientific community isn't the worst thing, but there's no need to present it as independent critical thinking.

I'm sure there are holes in this line of thought, so all criticism is welcome.

Yet another post from me about theism?

This time, pushing for a more clearly articulated position. Yes, I realize that I am not endearing myself by continuing this line of debate. However, I have good reasons for pursuing.

• I really like LW and the idea of a place where objective, unbiased truth is The Way. Since I idealistically believe in Aumann’s Agreement theorem, I think that we are only a small number of debates away from agreement.

• To the extent to which LW aligns itself with a particular point of view, it must be able to defend that view. I don’t want LW to be wrong, and am willing to be a nuisance to make sure.

• If defending atheism is not a first priority, can we continue using religion as a convenient example of irrationality, even as the enemy of rationality?

• There is a definite sense that theism is not worth debating, that the case is "open-and-shut". If so, it should be straight-forward to draft a master argument. (Five separate posts of analogies is not strong evidence in my Bayesian calculation that the case is open-and-shut.)

• A clear and definitive argument against theism would make it possible for theists (and yourselves, as devil's advocates) to debate specific points that are not covered adequately in the argument. (If you are about to downvote me on this comment, think about how important it would be to permit debate on an ideology that is important to this group. Right now it is difficult to debate whether religion is rational because there is no central argument to argue with.)

• Relative to the ‘typical view’, atheism is radical. How does a religious person visiting this site become convinced that you’re not just a rationality site with a high proportion of atheists?

A criticism of practices on LW that are attractive now but which will hinder "the way" to truth in the future; that lead to a religious idolatry of ideas (a common fate of many "in-groups") rather than objective detachment. For example,

(1) linking to ideas in original posts without summarizing the main ideas in your own words and how they apply to the specific context -- as this creates short-cuts in the brain of the reader, if not in the writer

(2) Use of analogies without formally defining the ideas behind them leads to content not only saying more than it intends to (or more than it strictly should) but also having meta-meanings that are attractive but dangerous because they're not explicit. [edit: "formally" was a poor choice of words, "clearly" is my intended meaning]

And any other examples people think of, now and as LW develops.

Winning Interpersonally

cousin_it would like to know how rationality has actually helped us win. However, in his article, he completely gives up on rationality in one major area, admitting that "interpersonal relationships are out."

Alex strenuously disagrees, asking "why are interpersonal relationships out? I think rationality can help a great deal here."

(And, of course, I suppose everone knows my little sob-story by now.)

I'd like to get a read from the community on this question.

Is rationality useless -- or worse, a liability when dealing with other human beings? How much does it matter if those human beings are themselves self-professed rationalists? It's been noted that Less Wrong is incredibly male. I have no idea whether this represents an actual gender differential in desire for epistemic rationality, but if it does, it means most male Less Wrongers should not expect to wind up dating rationalists. Does this mean that it is necessary for us to embrace less than accurate beliefs about, eg, our own desirability, that of our partner, various inherently confused concepts of romantic fate, or whatever supernatural beliefs our partners wish do defend? Does this mean it is necessary to make the world more rational, simply so that we can live in it?

(note: this draft was written a while before Gender and Rationality, so there's probably some stuff I'd rewrite to take that into account)

I'd be interested in reading (but not writing) a post about rationalist relationships, specifically the interplay of manipulation, honesty and respect.

Seems more like a group chat than a post, but let's see what you all think.

(rationlism:winning)::(science:results)

We've argued over whether rationalism should be defined as that which wins. I think this is isomorphic to the question whether science should be defined as that which gets good results.

I'd like to look at the history of science in the 16th-18th centuries, to see whether such a definition would have been a help or a hindrance. My priors say that it would have been a hindrance, because it wouldn't have kicked contenders out of the field rapidly.

Under position 1, "science = good results", you would have competition only on the level of individual theories. If the experimental approach to transforming metals won out over mystical Hermetic formulations, that would tell you nothing about whether you would expect an experimental approach to crop fertilization to win out over prayer to the gods.

Position 2, that science is a methodology that turns out to have good results, lets epistemologies, or families of theories, compete. You can group a whole bunch of theories together and call them "scientific", and a whole bunch of other theories together and call them "tradition", and other theories together and call them "mystic", etc.; and test the families against each other. This gives you much stronger statistics. This is probably what happened.

The ideal title for my future post would be this:

How I Confronted Akrasia and Won.

It would be an account of my dealing with akrasia, which so far resulted in eliminating two decade-long addictions and finally being able to act according to my current best judgment. I also hope to describe a practical result of using these techniques (I specified a target in advance and I'm currently working towards it.)

Not posted because:

1. The techniques are not yet tested even on myself. They worked perfectly for about a couple of months, but I wasn't under any severe stress or distraction. Also, I think they don't involve much willpower (defined as forcing myself to do something), but perhaps they do -- and the only way to find out is to test them in a situation when my 'willpower muscles' are exhausted.

2. The ultimate test would be the practical result which I plan to achieve using these techniques, and it's not yet reached.

Regarding all the articles we've had about the effectiveness of reason:

Learning about different systems of ethics may be useless. It takes a lot of time to learn all the forms of utilitarianism and their problems, and all the different ethical theories. And all that people do is look until they find one that lets them do what they wanted to do all along.

IF you're designing an AI, then it would be a good thing to do. Or if you've already achieved professional and financial success, and got your personal life in order (whether that's having a wife, having a family, whatever), and are in a position of power, it would be good to do. But if you're a grad student or a mid-level manager, it may be a big waste of time. You've already got big obvious problems to work on; you don't need to study esoteric theories of utility to find a problem to work on.

Any interest in a top-level post about rationality/poker inter-applications?

The Implications of Saunt Lora's Assertion for Rationalists.

For those who are unfamiliar, Saunt Lora's Assertion comes from the novel Anathem, and expresses the view that there are no genuinely new ideas; every idea has already been thought of.

A lot of purportedly new ideas can be seen as, at best, a slightly new spin on an old idea. The parallels between, Leibniz's views on the nature of possibility and Arnauld's objection, and David Lewis's views on the nature of possibility and Kripke's objection are but one striking example. If there is anything to the claim that we are, to some extent, stuck recycling old ideas, rather than genuinely/interestingly widening the range of views, it seems as though this should have some import for rationalists.

Scott Peck, author of "The Road Less Travelled", which was extremely popular ten years ago, theorised that people became more mature, and could get stuck on a lower level of maturity. From memory, the stages were:

1. Selfish, unprincipled
2. Rule- following
3. Rational
4. Mystical.

Christians could be either rule-following, a stage of maturity most people could leave behind in their teens, needing a big friendly policeman in the sky to tell them what to do- or Mystical.

Mystical people had a better understanding of the World because they did not expect it to be "rational", following a rationally calculable and predictable course. This fits my map in some ways: there are moments when I relate better to someone if I rely on instinct, rather than calculating what is going on, just as I can hit something better if I let my brain do the work rather than try to calculate a parabolic course for the rock.

I am not giving his "stage four" as well as he could. If you like, I would read up in his books, including "Further along the RLT" and "The RLT and beyond" and "The Different Drum" (I used to be a fan, and still hold him in respect).

You could then either decide you were convinced by Scott Peck, or come up with ways to refute him.

Would you like an article on this? Or would you rather just read about him on wikipedia?

Wikipedia says,

Stage IV is the stage where an individual starts enjoying the mystery and beauty of nature. While retaining skepticism, he starts perceiving grand patterns in nature. His religiousness and spirituality differ significantly from that of a Stage II person, in the sense that he does not accept things through blind faith but does so because of genuine belief. Stage IV people are labeled as Mystics.

I have this idea in my mind that my value function differs significantly from that of Elizier. In particular I cannot agree to blowing up Huygens in that Baby-Eater Scenario presented.

To summarize shortly: He gives a scenario which includes the following problem:

Some species A in the universe has as a core value the creation of unspeakable pain in their newborn. Some species B has as core value removal of all pain from the universe. And there is humanity.

In particular there are (besides others) two possible actions: (1): Enable B to kill off all of A, without touching humanity, but kill off a some humans in the process. (2): Block all access between all three species, leading to a continuation of the acts of A, kill significantly less humans in the process.

Elizier claims action 1 is superior over 2, and I cannot agree.

First, some reason why my intuition tells me that Elizier got it wrong: Consider the situation with wild animals, say in Africa. Lions killing gazelles in the thousands. And we are not talking about clean, nice killing, we are talking about taking bites out of living animals. We are talking about slow, agonizing death. And we can be pretty certain about the qualia of that experience, by just extrapolating from brain similarity and our own small painful experiences. Yet, I don't see anybody trying to stop the lions, and I think that is right.

For me the only argument for killing off special A goes like: "I do not like Pain" -> "Pain has negative utility" -> "Incredible pain got incredible negative utility" -> "Incredible pain needs to be removed from the Universe" That sounds wrong to me at the last step. Namely, I feel that our value function ought to (actually does) include a term which discounts things happening far away from us. In particular I think that the value of things happening somewhere in the universe which are (by the scenario) guaranteed not to have any effects on me, are exactly zero.

But more importantly it sounds wrong at the second to last step, claiming that incredible pain has incredible negative utility. Why do we dislike our own pain? Because it is the hardware response closing the feedback loop for our brain in the case of stupidity. It's evolution's way of telling us "don't do that". Why do we dislike pain in other people. Due to sympathy, i.e. due to reduces efficiency of said people in our world.

Do I feel more sympathy towards mammals than towards insects, yes. Do I feel more sympathy towards apes than towards other mammals, again yes. So the trend seems to indicate that I feel sympathy towards complex thinking things.

Maybe that's only because I am a complex thinking thing, but then again, maybe I just value possible computation. Computation generally leads to knowledge, and knowledge leads to more action possibilities. And more diversity in the things carrying out computation will probably lead to more diversity in knowledge, which I consider A Good Thing. Hence, I opt for saving species A, thus creating a lot more of pain, but also some more computation.

As you can probably tell, my line of reasoning is not quite clear yet, but I feel that I got a term in my value function here, that some other people seem to lack, and I wonder whether that's because of misunderstanding or because of different value functions.

Lurkers and Involvement.

I've been thinking that one might want to make a post, or post a survey, that attempts to determine how much folks engage with the contents on less wrong.

I'm going to assume that there are far more lurkers than commenters, and far more commenters than posters, but I'm curious as to how many minutes, per day, folks spend on this site.

For myself, I'd estimate no more than 10 or 15 minutes but it might be much less than that. I generally only read the posts from the RSS feed, and only bother to check the comments on one in 5. Even then, if there's a lot of comments, I don't bother reading most of them.

One of the reasons I don't post is that I often find it takes me 20-30 minutes to put my words into a shape that I feel is up to the rather high standard of posting quality here, and I'm generally not willing to commit that much of my time to this site.

I think the question of how much of their time an average person thinks a site is worth to them is an important metric, and one we may wish to try to measure with an eye to increasing the average for this site.

Heck, that might even get me posting more often.

Putting together a rationalist toolset. Including all the methods one needs to know, but also, and very much so the real world knowledge that helps to get along or ahead in life. Doesnt have to be reinvented, but pointed out and evaluated.

In short: I expect members of the rationality movement to dress well when its needed. To be in reasonable shape. To /not/smoke. Know about positive psychology. Know about how to deal with people. And find ways to be a rational & happy person.

There has been some calling for applications of rationality; how can this help me win? This combined with the popularity and discussion surrounding "Stuck in the middle with Bruce" gave me an idea for a potential series of posts relating to LWers pastimes of choice. I have a feeling most people here have a pastime, and if rationalists should win there should be some way to map the game to rational choices.

Perhaps articles discussing "how rational play can help you win at x" and "how x can help you think more rationally" would be worthwhile. I'm sure there are games or hobbies that multiple people share (as was discovered relating to Magic) and even if no one has played a certain game the knowledge gained from it should be generalizable and used elsewhere (as was the concept of a Bruce).

I might be able to do a piece on Counter-Strike (perhaps generalized to FPS style games) although I haven't played in several years.

I know I would be interested in more discussion of how Magic and rationality work together. In fact I almost went out an picked up a deck to try it out again (haven't played since Ice Age when I was but a child) but remembered I don't know anyone I could play with right now anyway, which is probably why I don't.

memetic engineering

The art of manipulating the media, especially news, and public opinion. Sometimes known as "spin-doctoring" I guess, but I think the memetic paradigm is probably a more useful one to attack it from.

I'd love to understand that better than I do. Understanding it properly would certainly help with evangelism.

I fear that very few people really do grok it though, certainly I wouldn't be capable of writing much relevant about it yet.

I have more to say about my cool ethics course on weird forms of utilitarianism, but unlike with Two-Tier Rationalism, I'm uncertain of how germane the rest of these forms are to rationalism.

I have a lot to say about the Reflection Principle but I'm still in the process of hammering out my ideas regarding why it is terrible and no one should endorse it.

I started an article on the psychology of rationalization, but stopped due to a mixture of time constrains and not finding many high quality studies.

It seem to me possible to create a safe oracle AI. Suppose that you have a sequence predictor which is a good approximation of Solomonoff induction but which run in reasonable time. This sequence predictor can potentially be really useful (for example, predict future siai publications from past siai publications then proceed to read the article which give a complete account of Friendliness theory...) and is not dangerous in itself. The question, of course, is how to obtain such a thing.

The trick rely on the concept of program predictor. A program predictor is a function which predict, more or less accurately, the output of the program (note that when we refer to a program we refer to a program without side effect that just calculate an output.) it take as it's input but within reasonable time. If you have a very accurate program predictor then you can obviously use it to gain a good approximation of Solomonoff induction which run in reasonable time. But of course, this just displace the problem: how do you get such an accurate program predictor?

Well, suppose you have a program predictor which is good enough to be improved on. Then, you use it to predict the program of less than N bits of length (with N sufficiently big of course) which maximize a utility function which measure how accurate the output of that program is as a program predictor given that it generate this output in less than T steps (where T is a reasonable number given the hardware you have access to). Then you run that program. Check the accuracy of the obtained program predictor. If insufficient repeat the process. You should eventually obtain a very accurate program predictor. QED.

So we've reduced our problem to the problem of creating a program predictor good enough to be improved upon. That should be possible. In particular, it is related to the problem of logical uncertainty. If we can get a passable understanding of logical uncertainty it should be possible to build such a program predictor using it. Thus a minimal understanding of logical uncertainty should be sufficient to obtain agi. In fact even without such understanding, it may be possible to patch together such a program predictor...

The Verbal Overshadowing effect, and how to train yourself to be a good explicit reasoner.

Contents of my Drafts folder:

• A previous version of my Silver Chair post, with more handwringing about why one might not stop someone from committing suicide.
• A post about my personal motto ("per rationem, an nequequam", or "through reason, or not at all"), and how Eliezer's infamous Newcomb-Box post did and didn't change my perspective on what rationality means.
• A listing of my core beliefs related to my own mind, beliefs/desires/etc., with a request for opinions or criticism.
• A post on why animals in particular and any being not capable of rationality/ethics in general don't get moral consideration. This one isn't posted only because my proof has a hole.

Great thread idea.

Frequentist Pitfalls:

Bayesianism vs Frequentism is one thing, but there are a lot of frequentist-inspired misinterpretations of the language of hypothesis testing that all statistically competent people agree are wrong. For example, note that: p-values are not posteriors (interpreting them this way usually overstates the evidence against the null, see also Lindley's paradox), p-values are not likelihoods, confidence doesn't mean confidence, likelihood doesn't mean likelihood, statistical significance is a property of test results not hypotheses, statistical significance is not effect size, statistical significance is not effect importance, p-values aren't error probabilities, the 5% threshold isn't magical.

In a full post I'd flesh all of these out, but I'm considering not doing so because it's kind of basic and it turns out Wikipedia already discusses most of this surprisingly well.

What an argument for atheism would look like.

Yesterday, I posted a list for reasons for why I think it would be a good idea to articulate a position on atheism.

I think many people are interested in theory in developing some sort of deconversion programme (for example, see this one), and perhaps creating a library of arguments and counter-arguments for debates with theists.

While I have no negative opinion of these projects, my ambition is much more modest. In a cogent argument for atheism,there would be no need to debate particular arguments. It would be much more effective (and evincing of an open-and-shut case) to just present the general line of reasoning. For example, the following argument would be a beginning (from someone who is just trying to guess what your arguments are):

Example: An Articulated Athiestic Position

First, we reject the claim that any empirical observation is the sleight of hand of a Supreme Being on the grounds that if you begin with such a hypothesis, then you find yourself in an unacceptable epistimological position in which you cannot know anything.

Therefore, we interpret all empirical evidence at face value, using the scientific method, and accept scientific theory, limited as ever by the scientist's view that theories are modified as needed to incorporate conflicting evidence. Thus, we reject a literal interpretation of the bible and the supernatural as these explicitly contradict the notion that we can interpret empirical evidence at face value.

By this, we have limited the acceptable definition of God to one that obeys the natural laws of the universe. While this leaves open the possibility of a passive, non-intrusive God, we present the principle that one does not assume the existence of something if there is no need to do so.

A definition like the one above is good in that a theist can read it, see exactly where assumptions differ, and decide if they want to engage in a debate -- or not. If they think God created an illusive world to test our faith, there's no point arguing about empirical evidence of anything.

I wanted to give this definition as a crude example of the simplicity I would like to see; I in no way intend to speak for atheists by making a final statement on the main arguments -- I would expect this to be edited.

As another example, I would like to give an argument for theism that I think could be a good articulation of their position, if someone held these particular views. The value being in that assumptions are explicit so pretty much you can find exactly where your views differ.

Example: An Articulated Thiestic Position

The existence of God can neither be proved nor demonstrated through any empirical evidence, but is knowable through reasoned reflection and intuition. All knowledge of the properties of God are inferred through experience, and thus knowledge of God is evolving and not absolute.

Thank you for this post -- I feel a bit lighter somehow having all those drafts out in the open.

Let's empty out my draft folder then....

Counterfactual Mugging v. Subjective Probability

A couple weeks ago, Vladimir Nesov stirred up the biggest hornet's nest I've ever seen on LW by introducing us to the Counterfactual Mugging scenario.

If you didn't read it the first time, please do -- I don't plan to attempt to summarize. Further, if you don't think you would give Omega the $100 in that situation, I'm afraid this article will mean next to nothing to you.

So, those still reading, you would give Omega the $100. You would do so because if someone told you about the problem now, you could do the expected utility calculation 0.5U(-$100)+0.5U(+$10000)>0. Ah, but where did the 0.5s come from in your calculation? Well, Omega told you he flipped a fair coin. Until he did, there existed a 0.5 probability of either outcome. Thus, for you, hearing about the problem, there is a 0.5 probability of your encountering the problem as stated, and a 0.5 probability of your encountering the corresponding situation, in which Omega either hands you $10000 or doesn't, based on his prediction. This is all very fine and rational.

So, new problem. Let's leave money out of it, and assume Omega hands you 1000 utilons in one case, and asks for them in the other -- exactly equal utility. What if there is an urn, and it contains either a red or a blue marble, and Omega looks, maybe gives you the utility if the marble is red, and asks for it if the marble is blue? What if you have devoted considerable time to determining whether the marble is red or blue, and your subjective probability has fluctuated over the course of you life? What if, unbeknownst to you, a rationalist community has been tracking evidence of the marble's color (including your own probability estimates), and running a prediction market, and Omega now shows you a plot of the prices over the past few years?

In short, what information do you use to calculate the probability you plug into the EU calculation?

I have an idea I need to build up about simplicity, how to build your mind and beliefs up incrementally, layer by layer, how perfection is achieved not when there's nothing left to add, but nothing left to remove, how simple minded people are sometimes being the ones to declare simple, true ideas others lost sight of, people who're too clever and sophisticate, whose knowledge is like a card house, or a bag of knots, genius, learning, growing up, creativity correlated with age, zen. But I really need to do a lot more searching about that before I can put something together.

Edit : and if I post that here, that's because if someone else wants to dig that idea, and work on it with me, that'd be with pleasure.

"Telling more than we can know" Nisbett & Wilson

I saw this on Overcoming Bias a while ago thanks to Pete Carlton www.lps.uci.edu/~johnsonk/philpsych/readings/nisbett.pdf

I hope you all read this. What are the implications? Can you tell me a story ;)

Here is my story and I am sticking to it!

I have a sailing friend that makes up physics from whole cloth it is frightening and amusing to watch an almost impossible to correct without some real drama. It seems one only has to be close in horse shoes and hand grenades and life.