Reference class of the unclassreferenceable

Meta question here: why does reference-class forecasting work at all?

Presumably, the process is that when you cluster objects by visible features, you also cluster all their invisible features too, and the invisible features are what determines the time evolution of those objects.

If the category boundary of the "reference class" is a simple one, then you can't fool yourself by interfering with the statistical correlation between visible and hidden attributes.

For example, reference class forecasting predicts that cryo will not work because cryo got clustered with all the theistic religious afterlives, and things like the alchemists' search for the Elixir of Life. The visible attribute we're clustering on is "actions that people believe will result in an infinite life, or >200 year life".

But a cryonics advocate might complain that this argument is trampling roughshod over all the careful inside view reasoning that cryonicists have done about why cryo is different than religion or superstition: namely that we have a scientific theory for what is going on.

If you drew the boundary around "Medical interventions that have a well accepted scientific theory backing them up", then cryo fares better. The different boundaries you can draw lead to focus upon different hidden attributes of the object in question: cryonics is like religion in some ways, but it is also like heart transplants.

I suggest a reference class "Predictions of technologies which will allow humans to recover from what would formerly have been considered irreversible death", with successful members such as heart transplants, CPR, and that shock thing medical shows are so fond of. (You know, where they shout "Clear!" before using it.)

So to sum up, you think you have a heuristic "On average, nothing ever happens for the first time" which beats any argument that something is about to happen for the first time. Cases like the Wright Brothers (reference class: "attempts at heavier-than-air flight") are mere unrepeatable anomalies. To answer the fundamental rationalist question, "What do you think you know and how do you think you know it?", we know the above is so because experiments show that people could do better at predicting how long it will take them to do their Christmas shopping by asking "How long did it take last time?" instead of trying to visualize the details. Is that a fair summary of your position?

http://lesswrong.com/lw/ri/the_outside_views_domain/

http://lesswrong.com/lw/vz/the_weak_inside_view/

I hereby assign all your skepticism to "beliefs the future will be just like the past" with associated correctness frequency zero.

PONG.

Your move in the wonderful game of Reference Class Tennis.

Finding a convincing reference class in which cryonics, singularity, superhuman AI etc. are highly probable

I'll nominate hypotheses or predictions predicated on materialism, or maybe the Copernican/mediocrity principle. In an indifferent universe, there's nothing special about the current human condition; in the long run, we should expect things to be very different in some way.

Note that a lot of the people around this community who take radically positive scenarios seriously, also take human extinction risks seriously, and seem to try to carefully analyze their uncertainty. The attitude seems markedly different from typical doom/salvation prophecies.

(Yes, predictions about human extinction events have never come true either, but there are strong anthropic reasons to expect this: if there had been a human extinction even in our past, we wouldn't expect to be here to talk about it!)

Alter these reference classes even tiny bit, and the result you get is basically just the opposite. For cryonics, just use the reference class of cases where people thought either a) that technology X could prolong the life of the patient, or b) that technology X could preserve wanted items, or c) that technology X could restore wanted media. Comparing it to technologies like this seems much more reasonable than taking the single peculiar property of cryonics(that it could theoretically for the first time grant us immortality) and using only that as a reference class. You could use same argument of using the peculiar property as reference class against any developing technology and consistently reach ~0% chance for it, so it works as perfectly general counter argument too.

Coming of a new world seems more reasonable reference class for singularity, but you seem to be interpreting in a bit strickter way than I would. I'd rephrase that as reference class of enormous changes in society, and there has indeed been many of such. Also, we note that processing and spreading information has been crucial to many of these, so narrowing our reference class to crucial properties of singularity(which basically just means "huge change in society due to artifical being that is able to process information better than we are"), we actually gain opposite result than what you did.

We do have a fairly good track record of making artifical beings that replicate parts of human behavior, too.

The problem with a lot of faulty outside view arguments is that the choice of possible features to focus on is too rich. For a reference class to be a good explanation of the expected conclusion, it needs to be hard to vary. Otherwise, the game comes down to rationalization, and one may as well name "Things that are likely possible" as the reference class and be done with it.

You don't even have to go as far as to cryonics and AI to come up with examples of the outside view's obvious failure. For example, mass production of 16nm processors has never happened in the course of history. Eh, technological advancement in general is a domain where outside view is useless, unless you resort to 'meta-outside views' like Kurzweil, such as predicting an increase in computing power because in the past computing power has increased.

Ultimately, I think the outside view is a heuristic that is sometimes useful and sometimes not; since actual outcomes are fully determined by "inside" causality.

A general observation - a class reference and the outside view are only useful if they are similar enough in the relevant characteristics, whatever they may be.

For general predictions of the future, the best rule is to predict only general classes of futures, the more detailed the predictions, even slightly more detailed, the significantly lower the odds of that future coming about.

I think the odds of successful cryonics to be about even. A Singularity of some sort happening within 50 years slightly better than even. And a FOOM substantially less, though because of its dangers, if it should happen, I still spend more time thinking about it than either of the others.

Also, contra your claim in the comments, there is no need for a Singularity to be "sudden"; if it happens it will be too fast for humans to adapt to. (It could even take years.)

"Now in some situations we have precise enough data that inside view might give correct answer - but for almost all such cases I'd expect outside view to be as usable and not far away in correctness."

Why? The above statement seems spectacularly wrong to me, and to be contradicted by all commonplace human experience, on a small scale or on a large scale.

"Reference class of predictions based on technology which isn't even remotely here has perhaps non-zero but still ridiculously tiny success rate."

What? Of such a tech, fairly well understood, EVER arising?

"reference class of beliefs in coming of a new world, be it good or evil, is huge and with consistent 0% success rate. "

I count several.

Why is this post so highly rated? As far as I can tell, the author is essentially saying that immortality will not happen in the future because it has not already happened. This seems obviously, overtly false.

This seems to me to argue yet again that we need to collect an explicit dataset of prior big long-term sci/tech based forecasts and how they turned out. If I assign a ~5+% chance to cryonics working, then for you to argue that goes against an outside view you need to show that substantially less than 5% of similar forecasts turned out to be wrong.

If you actually look a little deeper into cryonics you can find some more useful reference classes than "things promising eternal (or very long) life"

http://www.alcor.org/FAQs/faq01.html#evidence

1. Cells and organisms need not operate continuously to remain alive. Many living things, including human embryos, can be successfully cryopreserved and revived. Adult humans can survive cardiac arrest and cessation of brain activity during hypothermia for up to an hour without lasting harm. Other large animals have survived three hours of cardiac arrest near 0°C (+32°F ) (Cryobiology 23, 483-494 (1986)). There is no basic reason why such states of "suspended animation" could not be extended indefinitely at even lower temperatures (although the technical obstacles are enormous).

2. Existing cryopreservation techniques, while not yet reversible, can preserve the fine structure of the brain with remarkable fidelity. This is especially true for cryopreservation by vitrification. The observations of point (a) make clear that survival of structure, not function, determines survival of the organism.

3. It is now possible to foresee specific future technologies (molecular nanotechnology and nanomedicine) that will one day be able to diagnose and treat injuries right down to the molecular level. Such technology could repair and/or regenerate every cell and tissue in the body if necessary. For such a technology, any patient retaining basic brain structure (the physical basis of their mind) will be viable and recoverable.

I up-voted the post because you talked about two good, basic thinking skills. I think that paying attention to the weight of priors is a good thinking technique in general- and I think your examples of cryonics and AI are good points, but your conclusion fails- the argument you made does not mean they have 0 chance of happening, but you could take out of that more usefully, for example that any given person claiming to have created AI probably has close to 0 chance of having actually done it (unless you have some incredibly good evidence:

"Sorry Arthur, but I'd guess that there is an implicit rule about announcement of an AI-driven singularity: the announcement must come from the AI, not the programmer. I personally would expect the announcement in some unmistakable form such as a message in letters of fire written on the face of the moon." - Dan Clemmensen

). The thinking technique of abstracting and "stepping back from" or "outside of" or using "reference class forecasting" for your current situation also works very generally. Short post though, I was hoping you would expand more.

Finding a convincing reference class in which cryonics, singularity, superhuman AI etc. are highly probable - I invite you to try in comments, but I doubt this will lead anywhere.

Try the reference class of shocking things that science was predicted to never do, e.g. flying machines or transmutation of elements or travel to the planets.

I don't know if those are the right reference classes for prediction, but those two beliefs definitely fall into those two categories. That should set off some warning signals.

Most people seem to have a strong need to believe in life after death and godlike beings. Anything less than ironclad disproof leads them to strong belief. If you challenge their beliefs, they'll often vigorously demonstrate that these things are not impossible and declare victory. They ignore the distinction between "not impossible" and "highly likely" even when trying to persuade a known skeptic because, for them on those issues, the distinction does not exist.

Not that I see anyone doing that here.

It's just a warning sign that the topics invite bias. Proceed with caution.

It is not a good idea to try and predict the likelihood of the emergence of future technologies by noting how these technologies failed to emerge in the past. The reason is that cryonics, singularities, and the like, are very obviously more likely to exist in the future than they were in the past (due to the invention of other new technologies), and hence the past failures cease to be relevant as the years pass. Just prior to the successful invention of most new technologies, there were many failed attempts, and hence it would seem (looking backward and applying the same reasoning) that the technology is unlikely ever to be possible.

I think we should taboo the words "outside" and "inside" for purposes of this discussion. They obscure the actual reasoning processes being used, and they bring along analogies to situations that are qualitatively very different.

I put cryonics in the reference class of a "success of a technical project on a poorly understood system". Which means that most of medical research comes under that heading. So not good odds but not very small.

I put AGI in the same class, although it has the off putting property of possible recursion (in that it is trying to understand understanding, which is just a little hairy). Which means it might be a special case in how easy it is to solve, with the evidence so far pointing at the harder end of the spectrum.

For FOOM and the singularity I put it in the class of extrapolation from highly complex poorly understood theory. This gets a low probability of being right. But AGI is also in the reference class of potential world changing technologies (nukes), so still a good idea to tread carefully and try and get it into the class of better understood theories.

The main use of outside views is to argue that people with inside views are overconfident, presumably because they haven't considered enough failure modes or delays. Thus your reference class should include some details of the inside view.

Thus I reject your reference classes "things promising eternal life" and "beliefs in almost omnipotent good or evil beings" as not having inside views worth speaking of. "Predictions based on technology which isn't even remotely here" is OK.

likewise belief in singularity - reference class of beliefs in coming of a new world, be it good or evil, is huge and with consistent 0% success rate.

A new world did come to be following the Industrial Revolution. Another one came about twenty years ago or so, when the technology that allows us to argue this very instant came into its own. People with vision saw that these developments were possible and exerted themselves to accomplish them, so the success rate of the predictions isn't strictly nil. I'd put it above epsilon, even.

Our proposed complicated object here is "cryonics, singularity, superhuman AI etc." and I'm looking for a twist that decomposes it into separate parts with obvious references classes of objects taw finds highly probable. (Maybe. There are other ways to Transform a problem.) How about this: take the set of people who think all of those things are decently likely, then for each person apply the outside view to find out how likely you should consider to be. Or instead of people use journals. Or instead take the set of people who think none of those things are decently likely, apply the outside view to them, and combine. Or instead of "all of those things" use "almost-all of those things".

I wonder what the results of those experiments would be?

(Idea from this discussion on how to solve hard problems)

Biting the outside view bullet like me, and assigning very low probability to them.

I am going to stop using the term 'bite the bullet'. It seems to be changing meaning with repeated use and abuse.

For some things (especially concrete things like animals or toothpaste products), it is easy to find a useful reference class, while for other things it is difficult to find which possible reference class, if any, is useful. Some things just do not fit nicely enough into an existing reference class to make the method useful - they are unclassreferencable, and it is unlikely to be worth the effort attempting to use the method, when you could just look at more specific details instead. ("Unclassreferencable" suggests a dichotomy, but it's more of a spectrum.) ETA: I see this point has already been made here.

Humans naturally use an ad-hoc method that is like reference class forecasting (that may not be perfect or completely rational, but does a reasonable job sometimes). It is useful when we first encounter something and do not yet have enough specific details to evaluate it on its own terms. Once we have those details, the forecasting method is not needed. We use forecasting to get a heuristic on which things are worth us investigating further, so we can make that more detailed evaluation. Often something that is unclassreferencable is more worth investigating - we are curious about things that do not fit nicely into our existing categories.

There are a couple of ways promoters of a product/idea can exploit humans' natural forecasting habits. Sometimes the phrase "defies categorisation" or "doesn't fit into the normal genres" is applied to a new piece of music, to suggest that it is unclassreferencable and therefore worth checking out (which is better than a potential listener lumping it into a category that they don't like). On the other hand, sometimes promoters purposefully put themselves into a reference class, hoping that noone investigates finer details - like a new product claiming to be "environmentally friendly", or people wearing certain clothes to appear to have higher status.

Let me know if I'm suffering from man-with-hammer syndrome here, but it seems reference class forecasting is a useful way to think about many promotional strategies in a more systematic way.

Reference class forecasting is meant to overcome the bias among humans to be optimistic, whereas a perfect rationalist would render void the distinction between "inside view" and "outside view" -- it's all evidence.

Therefore a necessary condition to even consider using reference class forecasting for predicting an AI singularity or cryonics is that the respective direct arguments are optimistically biased. If so, which flaws do you perceive in the respective arguments, or are we humans completely blind to them even after applying significant scrutiny?

I'm perfectly willing to grant that, over the scope of human history, the reference classes for cryo/AGI/Singularity have produced near-0 success rates. I'd modify the classes slightly, however:

• Inventions that extend human life considerably: Penicillin, if nothing else. Vaccinations. Clean-room surgery.
• Inventions that materially changed the fundamental condition of humanity: Agriculture. Factories/mass production. Computers.
• Interactions with beings that are so relatively powerful that they appear omnipotent: Many colonists in the Americas were seen this way. Similarly with the cargo cults in the Pacific islands.

The point is, each of these references classes, given a small tweak, has experienced infrequent but nonzero successes--and that over the course of all of human history! Once we update the "all of human history" reference class/prior to account for the last century--in which technology has developed faster than probably the previous millennium--the posterior ends up looking much more promising.

I cannot think of any reference class in which cryonics does well. ... I invite you to try in comments

Okay: "Technologies whose success is predicated only on a) the recoverability of biological information from a pseudo-frozen state, and b) the indistinguishability of fundamental particles."

b) is well-established by repeated experiments, and a) is a combination of proven technologies.

I'd say that this is clashing with the sense that more should be possible in the world, and it has the problem that the reference classes are based on specific results. You almost sound like Lord Kelvin.

The reference class of things promising eternal life is huge, but it's also made of stuff that is amazingly irrational, entirely based on narrative, and propped up with the greatest anti-epistemology the world has ever known. Typically there were no moving parts.

The reference class for coming of a new world, to me, includes predictions like talk about the Enlightenment (I seem to remember very rosy predictions existing early on, but this is not my area of expertise) and other cases where people decided to work in a coordinated way to create a new world or people who had a somewhat coherent theory of society predicted a new world (the best-known example for this is of course Communism which was a flop, but it is not the only one.)

Almost omnipotent beings: The gods of most religions are clearly not omnipotent: they act according to the rules of drama.

Eternal life: If you remove the completely religious spam, you get stuff like the Fountain of Youth and the Philosopher's Stone, which still were things that people thought had to exist, not things that people realized should be possible to make.

Analyses of working systems trump comparisons to past nonoccurrence that seem similar to humans but were predicted for completely different reasons.

Reference class forecasting might be an OK way to criticise an idea (that is, in situations where you've done something a bunch of times, and you're doing the exact same thing and expect a different outcome despite not having any explanations that say there should be a different outcome), but the idea of using it in all situations is problematic, and it's easy to misapply:

It's basically saying 'the future will be like the past'. Which isn't always true. In cases like cryonics -- cases that depend on new knowledge being created (which is inherently unpredictable, because if we could predict it, we'd have that knowledge now) -- you can't say the future will be like the past.

To say the future will be like the past, you need an explanation for why. You can't just say, look, this situation is like that situation and therefore they'll have the same outcome.

The reason I think cryonics is likely is because a) death is a soluble problem and medical advancements are being made all the time, and b) even if it doesn't happen a couple hundred years from now, it would be pretty shocking if it wasn't solved at all, ever (including in thousands or millions of years from now). There would need to be some great catastrophe that prevents humans from making progress. Why wouldn't it be solved at some point?

This idea of applying reference class forecasting to cryonics and saying it has a 0% success rate is saying that we're not going to solve the death problem because we haven't solved it before. But that can be applied to anything where progress beyond what was done in the past is involved. As Roko said, try the reference class of shocking things science hasn't done before.

All of this reasoning doesn't depend on the future being like the past. It depends on explanations about why we think our predictions about the future are good, and the validity of these explanations doesn't depend on the outcomes of predictions of other stuff (though, again, the explanations might be criticised by saying 'hey, the outcome of this prediction which has the same explanation as the one you're using turned out to be false. So you need an explanation about why this criticism doesn't apply').

In short, I'm just saying: you can't draw comparisons between stuff without having an explanation about why it applies, and it's the explanation that's important rather than the comparison.

If something seems too good to be true, it probably is.

This is a real question concerning this quote:

reference class of beliefs in coming of a new world, be it good or evil, is huge and with consistent 0% success rate.

Are you saying that the Industrial Revolution did not has a success rate of greater than 0% to come to pass? The beliefs associated with it may not have been accurate when looking at some of the most critical or the most enthusiastic of supporters for the Industrial Revolution, but most of the Industrialists who made their fortunes from the event understood quite well that it was the end of a way of life and the beginning of a new world. It just wasn't all that either side of the polorized supporters or detractors made it out to be.

For the last two years, I have been gathering data about how the Singularity has gained its own sort of mythology, just as past profound social and technological changes have produced their own mythology.

Part of what is different with the Singularity is that we have long passed the point where much of the mythology has started to become fact. There are now sound hypotheses that can be tested.

This does not mean that the predictions of transcendence of biology that come as a consequence of some technical aspects of the Singularity are false however. It does mean that people are likely to place more hopes in them than should be the case, however.

Things like Cryonics. Most of us who have cryonics arrangements have been told by the companies themselves that they are making no promises and that we should think of this as no different than making any other form of final arrangements. Just that this arrangement carries with it a chance that isn't very likely to exist with the other arrangements (infinitesimally small with a normal burial given radically high technology - in other words next to none... And, absolutely none with cremation). So, being that we wish to give ourselves the best chances possible, and given that even $250,000 for the most expensive of plans is an astronomically low figure in a life span that is not limited by ordinary aging or other maladies... It is probably worth it.

Any wrongness can be explained as referencing a suboptimal reference class compared to an idealised reference class.

I recognize this is an old post, but I just wanted to point out that cryonics doesn't promise an escape from all forms of death, while Heaven does, meaning Heaven has a much higher burden of proof. Cryonics won't save you from a gunshot or a bomb or an accident, before or after you get frozen. Cryonics promises (a possibility of) an end to death by non-violent brain failure, specifically old age.

Science has been successful in the past at reducing the instances of death by certain non-violent failures of various organs. Open heart surgery and bypass surgery are two important ones, but there's also neurosurgery to remove formerly fatal brain tumors, hemispherectomy to cure formerly fatal epilepsy in children, and various procedures to limit the damage and reduce the lethality of strokes.

Add that to the fact that we know scientists are continuing to study the human brain, and there's no reason from the inside or the outside view to think that they'll suddenly stop, or that they'll find that old age is the one brain disease for which there's nothing that can be done even in theory, and there's reason to assign a small but non-trivial probability that if humanity survives for a couple of centuries, old age will be yet another formerly fatal disease that science has cured. That reference class is quite large, and the reference class of science doing the impossible is even larger (see flight, space travel.)

As for artificial intelligence, what about the reference class of "machines being able to do things formerly thought to be the sole domain of humanity"? Chess-playing computers, disease-diagnosing computers, computers which conduct scientific experiments, computers which compose music...I'm sure I'm missing many interesting innovations here.

ETA: A better rerefence class would be "machines doing things that were formerly thought to be the sole domain of humanity, better than humans can." Chess playing computers and calculators would fit into this category, too.

I knew that there was something else that I wanted to ask.

How closely is the Optimism Bias similar to the Dunnin-Krueger Effect?

reference class of beliefs in coming of a new world, be it good or evil, is huge and with consistent 0% success rate.

That's odd. This would imply that you don't believe in evolution of homo sapiens sapiens from previous hominids, or the invention of agriculture... . Heck, read the descriptions of heaven in the New Testament: the description of the ultimate better world (literally heaven!) is a step backward from everyday life in the West for most people.

It seems we have lots of examples of the transformation of the world for better, though I'd say there's not much room for worse than the lowest-happiness lives already in history.

Typo in para 1: "guess which will invariably turned out" => "turn out"

"Many economic and financial decisions depend crucially on their timing. People decide when to invest in a project, when to liquidate assets, or when to stop gambling in a casino. We provide a general result on prospect theory decision makers who are unaware of the time-inconsistency induced by probability weighting. If a market offers a sufficiently rich set of investment strategies, then such naive investors postpone their decisions until forever. We illustrate the drastic consequences of this “never stopping” result, and conclude that probability distortion in combination with naivité leads to unrealistic predictions for a wide range of dynamic setups.""

The whole issue of "singularity" needs a bit of clarification. If this is a physical singularity, i.e. a breakdown of a theory's ability to predict, then this is in the reference class of "theories of society claiming current models have limited future validity", which makes it nearly certain to be true.

If its a mathematical singularity (reaching infinity in finite time), then its reference class is composed nearly solely of erroneous theories.

You can get compromises between the two extremes (such as nuclear chain reactions - massive self feeding increase until a resource is exhausted), but it's important to define what you mean by singularity before assigning it to a reference class.

You will have an eternal life in heaven after your death isn't a real prediction.

A real prediction is something where you have a test to see whether or not the prediction turned out to be true. There's no test to decide whether someone has eternal life in heaven.

Predictions are about having the possibility to update your judgement after the event you predict happens or doesn't happen.