Open Thread: July 2010

Does anyone else find the idea of creating a printed magazine rather anachronistic?

I don't think there's enough content on LW to be worthwhile publishing a magazine. However, Eliezer's book on rationality should offer many of the same benefits.

While I'm not planning to pursue cryopreservation myself, I don't believe that it's unreasonable to do so.

Industrial coolants came up in a conversation I was having with my parents (for reasons I am completely unable to remember), and I mentioned that I'd read a bunch of stuff about cryonics lately. My mom then half-jokingly threatened to write me out of her will if I ever signed up for it.

This seemed... disproportionately hostile. She was skeptical of the singularity and my support for the SIAI when it came up a few weeks ago, but she's not particularly interested in the issue and didn't make a big deal about it. It wasn't even close to the level of scorn she apparently has for cryonics. When I asked her about it, she claimed she opposed it based on the physical impossibility of accurately copying a brain. My father and I pointed out that this would literally require the existence of magic, she conceded the point, mentioned that she still thought it was ridiculous, and changed the subject.

This was obviously a case of my mom avoiding her belief's true weak points by not offering her true objection, rationality failures common enough to deserve blog posts pointing them out; I wasn't shocked to observe them in the wild. What is shocking to me is that someone who is otherwise quite rational would feel so motivated to protect this particular belief about cryonics. Why is this so important?

That the overwhelming majority of those who share this intense motivation are women (it seems) just makes me more confused. I've seen a couple of explanations for this phenomenon, but they aren't convincing: if these people object to cryonics because they see it as selfish (for example), why do so many of them come up with fake objections? The selfishness objection doesn't seem like it would be something one would be penalized for making.

A factual error:

when he first announced his intention to have his brain surgically removed from his freshly vacated cadaver and preserved in liquid nitrogen

I'm fairly sure that head-only preservation doesn't involve any brain-removal. It's interesting that in context the purpose of the phrase was to present a creepy image of cryonics, and so the bias towards the phrases that accomplish this goal won over the constraint of not generating fiction.

I wonder if Peggy's apparent disvalue of Robin's immortality represents a true preference, and if so, how should an FAI take it into account while computing humanity's CEV?

That was very nearly terrifying.

Good article overall. Gives a human feel to the decision of cryonics, in particular by focusing on an unfair assault it attracts (thus appealing cryonicist's status).

The hostile wife phenomenon doesn't seem to have been mentioned much here. Is it less common than the article suggests or has it been glossed over because it doesn't support the pro-cryonics position? Or has it been mentioned and I wasn't paying attention?

This needs to be a top-level post. Even with minimal editing. Please.

(ETA: It's not so much that we need to have another go at the cryonics debate; but the above is an argument that I can't recall seeing discussed here previously, that does substantially change the picture, and that illustrates various kinds of reasoning - about scaling properties, about predefining thresholds of acceptability, and about what we don't know we don't know - that are very relevant to LW's overall mission.)

Done, 60 USD sent.

Done!

I'm sorry I didn't see that earlier; I donated $30 to the SIAI yesterday, and I probably could have waited a little while longer and donated $60 all at once. If this offer will still be open in a month or two, I will take you up on it.

Without any way of authenticating the donations, I find this to be rather silly.

First, let's calculate the concrete probability numbers. If we are to trust this calculator, the probability of finding exactly 75 big fish in a sample of a hundred from a pond where 75% of the fish are big is approximately 0.09, while getting the same number in a sample from a 25% big pond has a probability on the order of 10^-25. The same numbers hold in the reverse situation, of course.

Now, Al and Bob have to consider two possible scenarios:

1. The fish are 75% big, Al got the decently probable 75/100 sample, but Bob happened to be the first scientist who happened to get the extremely improbable 25/100 sample, and there were likely 10^(twenty-something) or so scientists sampling before Bob.

2. The fish are 25% big, Al got the extremely improbable 75/100 big sample, while Bob got the decently probable 25/100 sample. This means that Bob is probably among the first few scientists who have sampled the pond.

So, let's look at it from a frequentist perspective: if we repeat this game many times, what will be the proportion of occurrences in which each scenario takes place?

Here we need an additional critical piece of information: how exactly was Bob's place in the sequence of scientists determined? At this point, an infinite number of scientists will give us lots of headache, so let's assume it's some large finite number N_sci, and Bob's place in the sequence is determined by a random draw with probabilities uniformly distributed over all places in the sequence. And here we get an important intermediate result: assuming that at least one scientist gets to sample 25/100, the probability for Bob to be the first to sample 25/100 is independent of the actual composition of the pond! Think of it by means of a card-drawing analogy. If you're in a group of 52 people whose names are repeatedly called out in random order to draw from a deck of cards, the proportion of drawings in which you get to be the first one to draw the ace of spades will always be 1/52, regardless of whether it's a normal deck or a non-standard one with multiple aces of spades, or even a deck of 52 such aces!

Now compute the following probabilities:

P1 = p(75% big fish) * p(Al samples 75/100 | 75% big fish) * p(Bob gets to be the first to sample 25/100)
~ 0.5 0.09 1/N_sci

P2 = p(25% big fish) * p(Al samples 75/100 | 25% big fish) *p (Bob gets to be the first to sample 25/100)
~ 0.5 10^-25 1/N_sci

(We ignore the finite, but presumably negligible probabilities that no scientist samples 25/100 in either case; these can be made arbitrarily low by increasing N_sci.)

Therefore, we have P1 >> P2, i.e. the overwhelming majority of meetings between Al and Bob -- which are by themselves extremely rare, since Al usually meets someone from the other (N_sci-1) scientists -- happen under the first scenario, where Al gets a sample closely matching the actual ratio.

Now, you say:

It isn't terribly clear why Bob should discount all of his observations, since they don't seem to subject to any observation selection effect; at least from his perspective, his observations were a genuine random sample.

Not really, when you consider repeating the experiment. For the overwhelming majority of repetitions, Bob will get results close to the actual ratio, and on rare occasions he'll get extreme outlier samples. Those repetitions in which he gets summoned to meet with Al, however, are not a representative sample of his measurements! The criteria for when he gets to meet with Al are biased towards including a greater proportion of his improbable 25/100 outlier results.

As for this:

VARIANT: as before, but Charlie has a similar conversation with Bob. Only this time, he tells him he's going to introduce Bob to someone who observed exactly 75 of 100 fish to be big.

I don't think this is a well defined scenario. Answers will depend on the exact process by which this second observer gets selected. (Just like in the preceding discussion, the answer would be different if e.g. Bob had been always assigned the same place in the sequence of scientists.)

Interesting problem!

(This would be especially puzzling, since they have ALL the same information, having shared everything.)

It isn't terribly clear why Bob should discount all of his observations, since they don't seem to subject to any observation selection effect; at least from his perspective, his observations were a genuine random sample.

I think these two statements are inconsistent. If Bob is as certain as Al that Bob was picked specifically for his result, then they do have the same information, and they should both discount Bob's observations to the same degree for that reason. If Bob doesn't trust Al completely, they don't have the same information. Bob doesn't know for sure that Charlie told Al about the selection. From his point of view, Al could be lying.

VARIANT: as before, but Charlie has a similar conversation with Bob. Only this time, he tells him he's going to introduce Bob to someone who observed exactly 75 of 100 fish to be big.

If Charlie tells both of them they were both selected, they have the same information (that both their observations were selected for that purpose, and thus give them no information) and they can only decide based on their priors about Charlie stocking the pond.

If each of them only knows the other was selected and they both trust the other one's statements, same thing. But if each puts more trust in Charlie than in the other, then they don't have the same information.

One key observation is that Al made his observation after being told that he would meet someone who made a particular observation - specifically, the first person to make that specific observation, Bob. This makes Al and Bob special in different ways:

• Al is special because he has been selected to meet Bob regardless of what he observes. Therefore his data is genuinely uncorrelated with how he was selected for the meeting.
• Bob is special because he has been selected to meet Al because of the specific data he observes. More precisely, because he will be the first to obtain that specific result. Therefore his result has been selected, and he is only at the meeting because he happens to be the first one to get that result.

In the original case, Bob's result is effectively a lottery ticket - when he finds out from Al the circumstances of the meeting, he can simply follow the Natural Answer himself and conclude that his results were unlikely.

In the modified case, assuming perfect symmetry in all relevant aspects, they can conclude that an astronomically unlikely event has occurred and they have no net information about the contents of the pond.

Okay, qualitative analysis without calculations:

Let's go for a large, finite, case. Because otherwise my brain will explode.

Question 1: for any large, finite number of scientists Bob should defer MOSTLY to Alice.

First lets look at Alice; In any large finite number of scientists there is a small finite chance that NO scientist will get that result. This chance is larger in the case where 75% of the fish are big. Thus, upon finding that a scientist HAS encountered 25 fish, Alice must adjust her probability slightly towards 25% big fish.

Bob has also received several new pieces of information.

*He was the first to find 25 big fish. P[first25|found25] approaches 1/P[found25] as you increase the number of scientists. This information almost entirely cancels out the information he already had.

*All the information Alice had. This information therefore tips the scales.

Bob's final probability will be the same as Alice's.

Question two is N/A I will answer question three in a reply to this to try and avoid a massive wall of text.

What is each of their prior probabilities for this setup being true? Bob, knowing that he was selected for his unusual results, can pretty happily disregard them. If you win a lottery, you don't update to believe that most tickets win. Bob now knows of 100 samples (Al's) that relate to the prior, and accepts them. Bob's sampling is of a different prior: coin flipped, then a specific resulting sample will be found.

If they are both selected for their results, they both go to 50/50. Neither one has non-selected samples.

Is there any particular reason why one of the actors is an AI?

From Bob's perspective, he was more likely to be chosen as the one to talk to Al, if there are fewer scientist that observed exactly 25 big fish, which would happen if there are more big fish. So Bob should update on the evidence of being chosen.

First off all, I think that if Al does not see a sample, it makes the problem a bit simpler. That is, Al just tells Bob that he (Bob) is the first person that saw 25 big fishes.

I think that the number N of scientists matters, because the probability that someone will come to see Al depends on that.

Lets call B then event the lake has 75% big fishes, S the opposite and C the event someone comes, which means that someone saw 25 fishes.

Once Al sees Bob, he updates :
P(B/C)=P(B)* P(C/B)/(1/2*P(C/B)+1/2*P(C/S)).
When N tends toward infinity, both P(C/B) and P(C/S) tend toward 1, and P(B/SC) tends to 1/2.
But for small values of N, P(C/B) can be very small while P(C/S) will be quite close to 1.
Then the fact that someone was chosen lowers the probability of having a lake with big fishes.

If N=infinity, then the probability of being chosen is 0, and I cannot use Bayes' theorem.

If Charlie keeps inviting scientists until one sees 25 big fishes, then it becomes complicated, because the probability that you are invited is greater if the lake has more big fishes. It may be a bit like the sleeping beauty or the absent-minded driver problem.

Edited for formatting and misspellings

Part of the San Francisco skyline, maybe?

Thanks. This is the first time I ever noticed this. Absolutely no idea what it is or why it's there. Talk about selective blindness!

It was an early draft of the map vs territory theme that became the site header, which we intended to finish but forgot about.

If you think doom is very probable and we only survived due to the anthropic principle, then you should expect doom any day now, and every passing day without incident should weaken your faith in the anthropic explanation.

If you think all possible worlds exist, then you should expect our small bubble of ordered existence to erupt into chaos any day now, because way more copies of it are contained in chaotic worlds than in ordered ones. Every day you spend without spontaneously turning into a pheasant should weaken your faith in the multiverse.

(These arguments are not standard LW fare, but I've floated them here before and they seem to work okay.)

http://www.nickbostrom.com/papers/anthropicshadow.pdf

From what I've heard, there was a lot of talk about bomb shelters, but very few of them were built.

It makes me think of Richard Hamming talking about having "an attack".

This also reminded me that I wanted to go through the Intentional Stance by Daniel Dennett and find the good bits. Also worth reading is the wiki page.

I think he would state that the model you describe comes from folk psychology.

A relevant passage

"We have all learned to take a more skeptical attitude to the dictates of folk physics, including those robust deliverances that persist in the face of academic science. Even the "undeniable introspective fact" that you can feel "centrifugal force" cannot save it, except for the pragmatic purposes of rough-and-ready understanding it has always served. The delicate question of just how we ought to express our diminished allegiance to the categories of folk physics has been a central topic in philosophy since the seventeenth century, when Descartes, Boyle and other began to ponder the meta-physical status of color, felt warmth, and other "secondary qualities". These discussions, while cautiously agnostic about folk physics have traditionally assumed as unchallenged the bedrock of folk-psychological counterpart categories: conscious perceptions of color, sensations of warmth, or beliefs about the external "world"."

In lesswrong people do tend to discard the perception and sensation parts of folk psychology, but keep the belief and goal concepts.

You might have trouble convincing people here, mainly because people are interested in what should be done by an intelligence, rather than what is currently done by humans. It is a lot harder to find evidence for what ought to be done rather than what is done.

I'm not so interested in decision theory. I criticised it a bit here

Edit: To give a bit more background to how I view rationality: An intelligence is a set of interacting programs some of which have control of the agent at any one time. The rationality of the agent depends upon the set of programs in control of the agent. The relationship between the set of programs and rationality of the system is somewhat environmentally specific.

My kind of test would be like this:

1) Do you always seem to be able to predict the future, even as others doubt your predictions?

If they say yes ---> "That's because of confirmation bias, moron. You're not special."

There's an online test for calibration of subjective probabilities.

I would love for this to exist! I have some notes on easily-tested aspects of rationality which I will share:

The Conjunction Fallacy easily fits into a short multi-choice question.

I'm not sure what the error is called, but you can do the test described in Lawful Uncertainty:

Subjects were asked to predict whether the next card the experiment turned over would be red or blue in a context in which 70% of the cards were blue, but in which the sequence of red and blue cards was totally random. In such a situation, the strategy that will yield the highest proportion of success is to predict the more common event. For example, if 70% of the cards are blue, then predicting blue on every trial yields a 70% success rate. What subjects tended to do instead, however, was match probabilities - that is, predict the more probable event with the relative frequency with which it occurred. For example, subjects tended to predict 70% of the time that the blue card would occur and 30% of the time that the red card would occur. Such a strategy yields a 58% success rate.

You could do the positive bias test where you tell someone the triplet "2-4-6" conforms to a rule and have them figure out the rule.

You might be able to come up with some questions that test resistance to anchoring.

It might be out of scope of rationality and getting closer to an intelligence test, but you could take some "cognitive reflection" questions from here, which were discussed at LessWrong here.

The test's questions may need to be considerably dynamic to avert the possibility that people condition to specific problems without shedding the entire infected heuristic. Someone who had read Less Wrong a few times, but didn't make the knowledge truly a part of them, might return false negative for certain biases while retaining those biases in real-life situations. Don't want to make the test about guessing the teacher's password.

The test should include questions about applying rationality in one's life, not just abstract problems.

I'd suggest starting with a list of common biases and producing a question (or a few?) for each. The questions could test the biases and you could have an explanation of why the biased reasoning is bad, with examples.

It would also be useful to group the biases together in natural clusters, if possible.

Sounds like a good idea. Doesn't have to be invented from scratch; adapt a few psychological or behavioral-economics experiments. It's hard to ask about rationality in one's own life because of self-reporting problems; if we're going to do it, I think it's better to use questions of the form "Scenario: would you do a, b, c, or d?" rather than self-descriptive questions of the form "Are you more: a or b?"

Somewhat relatedly, I considered the idea of creating a 'Bias-Quotient' type test. It could go some way to popularising rationality and bias-aversion. A lot more people like the idea of being right than are actually aware of biases and other such behavioural stuff.

I anticipate that many of these people would do the test expecting to share their score somewhere online and gain relative intellect-prestige from an expected high score. On discovering that they're more biased than they believed, I believe that, provided the test's response to a low score were engaging and informative (and not annoying and pedantic), they would on net be genuinely interested in overcoming this, with a link to Less Wrong somewhere appropriate. They might share the test regardless of their low score with an annotation such as 'check this -- very interesting!'. That's all based on my model of how a lot of aspiring intelligent people behave. It may be biased.

This could open to a lot of people the doors to beginning to overcome the failures of their visceral probability heuristics, as well as the standard set of cognitive biases. The test's questions may need to be considerably dynamic to avert the possibility that people condition to specific problems without shedding the entire infected heuristic.

Voting is highly recommended - please do, and feel free to reply to comments with additional commentary as well. Otherwise I'd say leave them as be.

Also related: A lot of the Sequences show marks of their origin on Overcoming Bias that could be confusing to someone who lands on that article:

Example: "Since this is an econblog... " in http://lesswrong.com/lw/j3/science_as_curiositystopper/

I think some kind of editorial note is in order here, if not a rewrite.

Alternatively, we could repost/revisit the sequences on a schedule, and let the new posts build fresh comments.

Or even better, try to cover the same topics from a different perspective.

See also 'crank magnetism.'

I wonder if this counts as evidence for my heuristic of judging how seriously to take someone's belief on a complicated scientific subject by looking to see if they get the right answer on easier scientific questions.

Thanks for asking! I also really don't want this to fizzle away.

It is still being pursued by myself, Michael Vassar, and Michael GR via back channels rather than what I outlined in that post and it is indeed getting serious attention, but I don't expect us to have meaningful results for at least a year. I will make a Less Wrong post as soon as there is anything the public at large can do -- in the meanwhile, I respectfully ask that you or others do not start your own Craigslist charity group, as it may hurt our efforts at moving forward with this.

ETA: Successfully pulling off this Craigslist thing has big overlaps with solving optimal philanthropy in general.

I think there is an additional interpretation that you're not taking into account, and an eminently reasonable one.

First, to clarify the easy question: unless you believe that there is something mysteriously uncomputable going on in the human brain, the question of whether science can be automated in principle is trivial. Obviously, all you'd need to do is to program a sufficiently sophisticated AI, and it will do automated science. That much is clear.

However, the more important question is -- what about our present abilities to automate science? By this I mean both the hypothetical methods we could try and the ones that have actually been tried in practice. Here, at the very least, a strong case can be made that the 20th century attempt to transform science into a bureaucratic enterprise that operates according to formal, automated procedures has largely been a failure. It has undoubtedly produced an endless stream of cargo-cult science that satisfies all these formal bureaucratic procedures, but is nevertheless worthless -- or worse. At the same time, it's unclear how much valid science is coming out except for those scientists who have maintained a high degree of purely informal and private enthusiasm for discovering truth (and perhaps also those in highly practical applied fields where the cash worth of innovations provides a stringent reality check).

This is how I read Moldbug: in many important questions, we can only admit honestly that we still have no way to find answers backed by scientific evidence in any meaningful sense of the term, and we have to grapple with less reliable forms of reasoning. Yet, there is the widespread idea that if only the proper formal bureaucratic structures are established, we can get "science" to give us answers about whichever questions we find interesting, and we should guide our lives and policies according to the results of such "science." It's not hard to see how this situation can give birth to a diabolical network of perverse incentives, producing endless reams of cargo-cult scientific work published by prestigious outlets and venerated as "science" by the general public and the government.

The really scary prospect is that our system of government might lead us to a complete disaster guided by policy prescriptions coming from this perverted system that has, arguably, already become its integral part.

That it should be possible to Algorithmize Science seems clear from that the human brain can do science and the human brain should be possible to describe algorthmically. If not at a higher level, so at least -- in principle -- by quantum electrodynamics which is the (known and computable in principle) dynamics of electrons and nuclei that are the building blocks of the brain.( If it should be possible to do in practice it would have to be done at a higher level but as a proof of principle that argument should be enough.)

I guess, however, that what is actually meant is if the scientific method itself could be formalised (algorithmized), so that science could be "mechanized" in a more direct way than building human-level AIs and then let them learn and do science by the somewhat informal process used today by human scientists. That seems plausible. But has still to be done and seems rather difficult. The philosophers of science is working on understanding the scientific process better and better, but they seem still to have a long way to go before an actually working algorithmic description has been achieved. See also the discussion below on the recent article by Gelman and Shalizi criticizing bayesianism.

EDIT "done at a lower level" changed to "done at a higher level"

View 2: “How dare, how dare those people try to mechanize science! . . .

The pithy reply would be that science already is mechanized. We just don't understand the mechanism yet.

"Silas, there is no Bayesian ‘revival’ in science. There is one amongst people who wish to reduce science to a mechanical procedure." – Gene Callahan

Am I the only one who finds this extremely unlikely? So far as I know, Bayesian methods have become massively more popular in science over the last 50 years. (Count JSTOR hits for the word 'Bayesian,' for example, and watch the numbers shoot up over time!)

How hard do you think mechanizing science would be? It strikes me as being at least in the same class with natural language.

It's probably best to take a cyborg point of view -- consciously followed algorithms (like probabilistic updating) aren't a replacement for common sense, but they can be integrated into common sense, or used as measuring sticks, to turn common sense into common awesome cybersense.

You probably won't find much opposition to your opinion here on LW. Duh, of course science can and will be automated! It's pretty amusing that the thesis of Cosma Shalizi, an outspoken anti-Bayesian, deals with automated extraction of causal architecture from observed behavior of systems. (If you enjoy math, read it all; it's very eye-opening.)

I read Moldbug's quote as saying: there is currently no system, algorithmic or bureaucratic, that is even remotely close to the power of human intuition, common sense, genius, etc. But there are people who implicitly claim they have such a system, and those people are dangerous liars.

I pointed this out to my buddy who's a psychology doctoral student, his reply is below:

I don't know enough about g to say whether the people talking about it are falling prey to the general correlation between tests, but this phenomenon is pretty well-known to social science researchers.

I do know enough about CFA and EFA to tell you that this guy has an unreasonable boner for CFA. CFA doesn't test against truth, it tests against other models. Which means it only tells you whether the model you're looking at fits better than a comparator model. If that's a null model, that's not a particularly great line of analysis.

He pretty blatantly misrepresents this. And his criticisms of things like Big Five are pretty wild. Big Five, by its very nature, fits the correlations extremely well. The largest criticism of Big Five is that it's not theory-driven, but data-driven!

But my biggest beef has got to be him arguing that EFA is not a technique for determining causality. No shit. That is the very nature of EFA -- it's a technique for loading factors (which have no inherent "truth" to them by loading alone, and are highly subject to reification) in order to maximize variance explained. He doesn't need to argue this point for a million words. It's definitional.

So regardless of whether g exists or not, which I'm not really qualified to speak on, this guy is kind of a hugely misleading writer. MINUS FIVE SCIENCE POINTS TO HIM.

I think this is one of the few cases where Shalizi is wrong. (Not an easy thing to say, as I'm a big fan of his.)

In the second part of the article he generates synthetic "test scores" of people who have three thousand independent abilities - "facets of intelligence" that apply to different problems - and demonstrates that standard factor analysis still detects a strong single g-factor explaining most of the variance between people. From that he concludes that g is a "statistical artefact" and lacks "reality". This is exactly like saying the total weight of the rockpile "lacks reality" because the weights of individual rocks are independent variables.

As for the reason why he is wrong, it's pretty clear: Shalizi is a Marxist (fo' real) and can't give an inch to those pesky racists. A sad sight, that.

But neither of those are particularly compelling reasons for disagreement - can anyone more familiar with the psychological/statistical territory shed some light?

Shalizi's most basic point — that factor analysis will generate a general factor for any bunch of sufficiently strongly correlated variables — is correct.

Here's a demo. The statistical analysis package R comes with some built-in datasets to play with. I skimmed through the list and picked out six monthly datasets (72 data points in each):

• atmospheric CO2 concentrations, 1959-1964
• female UK lung deaths, 1974-1979
• international airline passengers, 1949-1954
• sunspot counts, 1749-1754
• average air temperatures at Nottingham Castle, 1920-1925
• car drivers killed & seriously injured in Great Britain, 1969-1974

It's pretty unlikely that there's a single causal general factor that explains most of the variation in all six of these time series, especially as they're from mostly non-overlapping time intervals. They aren't even that well correlated with each other: the mean correlation between different time series is -0.10 with a std. dev. of 0.34. And yet, when I ask R's canned factor analysis routine to calculate a general factor for these six time series, that general factor explains 1/3 of their variance!

However, Shalizi's blog post covers a lot more ground than just this basic point, and it's difficult for me to work out exactly what he's trying to say, which in turn makes it difficult to say how correct he is overall. What does Shalizi mean specifically by calling g a myth? Does he think it is very unlikely to exist, or just that factor analysis is not good evidence for it? Who does he think is in error about its nature? I can think of one researcher in particular who stands out as just not getting it, but beyond that I'm just not sure.

Here is a useful post directly criticizing Shalizi's claims: http://humanvarieties.org/2013/04/03/is-psychometric-g-a-myth/

I don't think it's surprising that an untenable claim could persist within a field for a long time, once established. Pluto was called a planet for seventy-six years.

I've no idea whether the critique of g is accurate, however.

There are many momentous issues here.

First: I think a historical narrative can be constructed, according to which a future unexpected in, say, 1900 or even in 1950 slowly comes into view, and in which there are three stages characterized by an extra increment of knowledge. The first increment is cryonics, the second increment is nanotechnology, and the third increment is superintelligence. There is a highly selective view; if you were telling the history of futurist visions in general, you would need to include biotechnology, robotics, space travel, nuclear power, even aviation, and many other things.

In any case, among all the visions of the future that exist out there, there is definitely one consisting of cryonics + nanotechnology + superintelligence. Cryonics is a path from the present to the future, nanotechnology will make the material world as pliable as the bits in a computer, and superintelligence guided by some utility function will rule over all things.

Among the questions one might want answered:

1) Is this an accurate vision of the future?

2) Why is it that still so few people share this perspective?

3) Is that a situation which ought to be changed, and if so, how could it be changed?

Question 1 is by far the most discussed.

Question 2 is mostly pondered by the few people who have answered 'yes' to question 1, and usually psychological answers are given. I think that a certain type of historical thinking could go a long way towards answering question 2, but it would have to be carried out with care, intelligence, and a will to objectivity.

This is what I have in mind: You can find various histories of the world which cover the period from 1960. Most of them will not mention Ettinger's book, or Eric Drexler's, or any of the movements to which they gave rise. To find a history which notices any of that, you will have to specialize, e.g. to a history of American technological subcultures, or a history of 20th-century futurological enthusiasms. An overkill history-based causal approach to question 2 would have a causal model of world history since 1960, a causal model of those small domains in which Ettinger and Drexler's publications had some impact, and finally it would seek to understand why the causal processes of the second sort remained invisible on the scale of the first.

Question 3 is also, intrinsically, a question which will mostly be of interest to the small group who have already answered 'yes' to question 1.

A good illustration of multiple discovery (not strictly 'discovery' in this case, but anyway) too:

While Ettinger was the first, most articulate, and most scientifically credible person to argue the idea of cryonics,[citation needed] he was not the only one. In 1962, Evan Cooper had authored a manuscript entitled Immortality, Scientifically, Physically, Now under the pseudonym "N. Durhing".[8] Cooper's book contained the same argument as did Ettinger's, but it lacked both scientific and technical rigor and was not of publication quality.[citation needed]

Okay, I've read through the whole thing so far.

This is not rationalist fiction. This is standard war porn, paperback thriller stuff. Many many technical descriptions of guns, rockets, military vehicles, etc. Throughout the story there's never any real conflict, just the American military (with help from the rest of the world) steamrolling everything, and the denizens of Heaven and Hell admiring the American way of life. It was well-written enough to hold my attention like a can of Pringles would, but I don't feel enriched by reading it.

Why did you link to TV Tropes instead of the thing itself?

Direct link to story

Ooh, an LW-themed horror story. My humble opinion: it's awesome! This phrase was genius:

What's it going to do, write pi at you?

Moar please.

Wait, is that the whole story? 'cause if so, I really don't get it. Where's the rest of it? What happens next? Is Jerry afraid that his algorithm is a self-improving AI or something?

How does 2212221 represent perfect numbers?

Playing Conway's Life is a great exercise - I recommend trying it, to anyone who hasn't. Feel free to experiment with different starting configurations. One simple one which produces a wealth of interesting effects is the "r pentomino":

Edit: Image link died - see Vladimir_Nesov's comment, below.

There is a third option of making a reasoned, rational meta-argument as to why the methods they were using to develop their position were wrong. I don't know how reliable it is, however.

I wish there is an area of science that gives reductionist explanations of morality, that is, the detailed contents of our current moral values and norms. One example that came up earlier was monogamy - why do all modern industrialized countries have monogamy as a social norm?

I'm currently reading The Moral Animal by Robert Wright, because it was recommended by, among others, Eliezer. I'm summarizing the chapters online as I read them. The fifth chapter, noting that more human societies have been polygynous than have been monogamous, examines why monogamy is popular today; you might want to check it out.

As for the wider question of reductionist explanations of morality, I'm a fan of the research of moral psychologist Jonathan Haidt (New York Times article, very readable paper).

That sounds like a reasonable criterion.

steven0461 already posted this to the previous Open Thread and we had a nice little talk.

I wrote a backlink to here from OB. I am not yet expert enough to do an evaluation of this. I do think however that it is an important and interesting question that mjgeddes asks. As an active (although at a low level) rationalist I think it is important to try to at least to some extent follow what expert philosophers of science actually find out of how we can obtain reasonably reliable knowledge. The dominating theory of how science proceeds seems to be the hypothetico-deductive model, somewhat informally described. No formalised model for the scientific process seems so far has been able to answer to serious criticism of in the philosophy of science community. "Bayesianism" seems to be a serious candidate for such a formalised model but seems still to be developed further if it should be able to anser all serious criticism. The recent article by Gelman and Shalizi is of course just the latest in a tradition of bayesian-critique. A classic article is Glymour "Why I am Not a Bayesian" (also in the reference list of Gelman and Shalizi). That is from 1980 so probably a lot has happened since then. I myself am not up-to-date with most of development, but it seems to be an import topic to discuss here on Less Wrong that seems to be quite bayesianistically oriented.

mjgeddes, why all the hate?

ETA: Never mind. I got my crackpots confused.

Original text was:

mjgeddes was once publicly dissed by Eliezer Yudkowsky on OB (can't find the link now, but it was a pretty harsh display of contempt). Since then, he has often bashed Bayesian induction, presumably in an effort to undercut EY's world view and thereby hurt EY as badly as he himself was hurt.

I'm not expert enough to interpret.

But I know Shalizi is skeptical of Bayesians and some of his blog posts seem so directly targeted at the LessWrong point of view that I almost suspect he's read this stuff. Getting in contact with him would be a coup.

(Fixed) link to earlier discussion of this paper in the last open thread.

(Edit - that's what I get for posting in this thread without refreshing the page. cousin_it already linked it.)

Yesterday, I posted my thoughts in last month's thread on the article. I'm reproducing them here since this is where the discussion is at:

[cousin_it summarizing Gelman's position] See, after locating the hypothesis, we can run some simple statistical checks on the hypothesis and the data to see if our prior was wrong. For example, plot the data as a histogram, and plot the hypothesis as another histogram, and if there's a lot of data and the two histograms are wildly different, we know almost for certain that the prior was wrong. As a responsible scientist, I'd do this kind of check. The catch is, a perfect Bayesian wouldn't. The question is, why?

Model checking is completely compatible with "perfect Bayesianism." In the practice of Bayesian statistics, how often is the prior distribution you use exactly the same as your actual prior distribution? The answer is never. Really, do you think your actual prior follows a gamma distribution exactly? The prior distribution you use in the computation is a model of your actual prior distribution. It's a map of your current map. With this in mind, model checking is an extremely handy way to make sure that your model of your prior is reasonable.

However, a difference in the data and a simulation from your model doesn't necessarily mean that you have an unreasonable model of your prior. You could just have really wrong priors. So you have to think about what's going on to be sure. This does somewhat limit the role of model checking relative to what Gelman is pushing.

Can anyone with more experience with Bayesian statistics than me evaluate this article?

I, for one, would be very interested in seeing a top-level post about this.

Even if antinatalism is true at present (I have no major opinion on the issue yet) it need not be true in all possible future scenarios.

In fact, should the human race shrink significantly [due to antinatalism perhaps], without societal collapse, the average utility of a human life should increase. I find it highly unlikely that even the maximum average utility is still less than zero.

I have long wrestled with the idea of antinatalism, so I should have something to say here. Certainly there were periods in my life in which I thought that the creation of life is the supreme folly.

We all know that terrible things happen, that should never happen to anyone. The simplest antinatalist argument of all is, that any life you create will be at risk of such intolerably bad outcomes; and so, if you care, the very least you can do is not create new life. No new life, no possibility of awful outcomes in it, problem avoided! And it is very easy to elaborate this into a stinging critique of anyone who proposes that nonetheless one shouldn't take this seriously or absolutely (because most people are happy, most people don't commit suicide, etc). You intend to gamble with this new life you propose to create, simply because you hope that it won't turn out terribly? And this gamble you propose appears to be completely unnecessary - it's not as if people have children for the greater good. Etc.

A crude utilitarian way to moderate the absoluteness of this conclusion would be to say, well, surely some lives are worth creating, and it would make a lot of people sad to never have children, so we reluctantly say to the ones who would be really upset to forego reproduction, OK, if you insist... but for people who can take it, we could say: There is always something better that you could do with your life. Have the courage not to hide from the facts of your own existence in the boisterous distraction of naive new lives.

It is probably true that philanthropic antinatalists, like the ones at the blog to which you link, are people who have personally experienced some profound awfulness, and that is why they take human suffering with such deadly seriousness. It's not just an abstraction to them. For example, Jim Crawford (who runs that blog) was once almost killed in a sword attack, had his chest sliced open, and after they stitched him up, literally every breath was agonizing for a long time thereafter. An experience like that would sensitize you to the reality of things which luckier people would prefer not to think about.

Why do you link to a blog, rather than an introduction or a summary? Is this to test whether we find it so silly that we don't look for their best arguments?

My impression is that antinatalists are highly verbal people who base their idea of morality on how people speak about morality, ignoring how people act. They get the idea that morality is about assigning blame and so feel compelled only to worry about bad acts, thus becoming strict negative utilitarians or rights-deontologists with very strict and uncommon rights. I am not moved by such moralities.

Maybe some make more factual claims, eg, that most lives are net negative or that reflective life would regret itself. These seem obviously false, but I don't see that they matter. These arguments should not have much impact on the actions of the utilitarians that they seem aimed at. They should build a superhuman intelligence to answer these questions and implement the best course of action. If human lives are not worth living, then other lives may be. If no lives are worth living, then a superintelligence can arrange for no lives to be lead, while people evangelizing antinatalism aren't going to make a difference.

Incidentally, Eliezer sometimes seems to be an anti-human-natalist.

The antinatalist argument goes that humans suffer more than they have fun, therefore not living is better than living. Why don't they convert their loved ones to the same view and commit suicide together, then? Or seek out small isolated communities and bomb them for moral good.

I believe the answer to antinatalism is that pleasure != utility. Your life (and the lives of your hypothetical kids) could create net positive utility despite containing more suffering than joy. The "utility functions" or whatever else determines our actions contain terms that don't correspond to feelings of joy and sorrow, or are out of proportion with those feelings.

Here's one: I bet if you asked lots of people whether their birth was a good thing, most of them would say yes.

If it turns out that after sufficient reflection, people, on average, regard their birth as a bad thing, then this argument breaks down.

I'm not entirely opposed to the idea. 6 billion is enough for now. Make more when we expand and distance makes it infeasible to concentrate neg-entropy on the available individuals. This is quite different from the Robin-Hanson 'make as many humans as physically possible and have them living in squalor' (exaggerated) position but probably also in in complete dissagreement with arguments used for Anti-natalism.

Either antinatalism is futile in long run, or it is existential threat.

If we assume that antinatalism is rational, then in long run it will lead to reduction of part of human population, that is capable/trained to do rational decisions, thus making antinatalists' efforts futile. As we can see, people that should be most susceptible to antinatalism don't even consider this option (en masse at least). And given their circumstances they have clear reason for that: every extra child makes it less likely for them to starve to death in old age, as more children more chances for family to control more resources. It is big prisoner dilemma, where defectors win.

Edit: Post-humans are not considered. They will have other means to acquire resources.

Edit: My point: antinatalism can be rational for individuals, but it cannot be rational for humankind to accept (even if it is universally true as antinatalists claim).

That there's a hereditary component to schizophrenia.

Is there really a cosmology that says that most beings with my subjective experiences are Boltzmann brains? It seems to me that in a finite universe, most beings will not be Boltzmann brains. And in an infinite universe, it's not clear what "most" means.

Would the simulation allow us to exit, in order to perform further research on the nature of the external world?

If so, I would enter it. If not? Probably not. I do not want to live in a world where there are ultimate answers and you can go no further.

The fact that I may already live in one is just bloody irritating :p

If we move into the same simulation and can really interact with others, then I wouldn't mind the move at all. Apart from that, experiences are the important bit and simulations can have those.

I might do that just sort of temporarily because it would be fun, similar to how apes like to watch other apes in ape situations even when it doesn't relate to their own lives.

But I would have to limit this kind of thing because, although pleasurable, it doesn't support my real values. I value real paperclips, not simulated paperclips, fun though they might be to watch.

Your footnote assumes away most of the real reasons for objecting to such a scenario (i.e. there is no remotely plausible world in which you could be confident that the simulation is either indestructible or tamper-proof, so entering it means giving up any attempt at personal autonomy for the rest of your existence).

Part 2 seems similar to the claim (which I have made in the past but not on LessWrong) that the Matrix was actually a friendly move on the part of that world's AI.

The given assumption seems unlikely to me, but in that case I think I'd go for it.

Is it assumed that no new information will be entered into simulation after launch?

And does it change your answers if you learn that we are living in a simulation now? Or if you learn that Tegmark's theory is correct?

Yes, assuming further that the simulation will expand optimally to use all available resources for its computation, and that any persons it encounters will be taken into the simulation.

Does Clippy maximise number-of-paperclips-in-universe (given all available information) or some proxy variable like number-of-paperclips-counted-so-far? If the former, Clippy does not want to move to a simulation. If the latter, Clippy does want to move to a simulation.

The same analysis applies to humankind.

My answer is yes, and your point is well-taken: We have to be careful about what we mean by "the real world".

These strike me as good examples of applied rationality. I wonder what percentage of Less Wrong readers would succeed as startup founders?

I wonder what percentage have ever tried?

These strike me as good examples of applied rationality. I wonder what percentage of Less Wrong readers would succeed as startup founders?

I would not deviate too much from the prior (most would fail).

I know this thread is a bit bloated already without me adding to the din

Do not worry about that. Pearl's Causality is part of the canon of this place.

You are right that YW means "Y and W". (The fact that they might be disjoint doesn't matter. It looks like the property you are referring to follows from the definition of conditional independence, but I'm not good at these kinds of proofs.)

And welcome to LW, don't feel bad about adding a question to the open thread.

Well, given that I can now be confident my words won't encourage you*, I will feel free to mention that I found the attitudes of many of those replying to you troubling. There seemed to be an awful lot of verbiage ascribing detailed motivations to you based on (so far as I could tell) little more than (a) your disagreement and (b) your tone, and these descriptions, I feel, were accepted with greater confidence than would be warranted given their prior complexity and their current bases of evidential support.

I'm slightly worried that some of my remarks to Sam fell in that category. Rereading them, I don't see that, but there may be substantial cognitive biases preventing me from seeing this issue in my own remarks. Did any of my comments fall into that category under your estimate? If so, which ones?

This is not a distortion of the original meaning. “Feeding the trolls” is just giving them replies of any sort — especially if they're well-written, because you’re probably investing more effort than the troll.

I don't think this is unique to LW at all. I've seen well-argued rebuttals to trolls labeled as feeding in many different contexts including Slashdot and the OOTS forum.

Definitely the latter, even when I'm on my own, any subsequent ideas after my first one tend to be variations on my first solution, unless I try extra hard to escape its grip.

You might think about the zen idea, in which the proposal of solutions is certainly held off, or treated differently. This is a very common idea in response to the tendency of solutions to precipitate themselves so ubiquitously.

To me "Bayescraft" has the connotation of a particular mental attitude, one inspired by Eliezer Yudkowsky's fusion of the ev-psych, heuristics-and-biases literature with E.T. Jaynes' idiosyncratic take on "Bayesian probabilistic inference", and in particular the desiderata for an inference robot: take all relevant evidence into account, rather than filter evidence according to your ideological biases, and allow your judgement of a proposition's plausibility to move freely in the [0..1] range rather than seek all-or-nothing certainty in your belief.

Capitalized words are often technical terms. So "Traditional Rationality" refers to certain epistemic attitudes and methods which have, in the past, been called "rational" (a word which is several hundred years old). This frees up the lower-case word "rationality", which on this site is also a technical term.

Bayescraft is just a synonym for Rationallity, with connotations of a) Bayes theorem, since that's what epistemic rationallity must be based on, and b) the notion that rationallity is a skill which must be developed personally and as a group (see also: Martial art of Rationallity (oh look, more capitals!))

The capitals are just for emphasis of concepts that the writer thinks are fundamentally important.