LessWrong 2.0 Reader

See my comment. The problem with the post is revealed in the fourth sentence:

To demonstrate how chaos theory imposes some limits on the skill of an arbitrary intelligence, I will also look at a game: pinball.

Note that predicting a ball is not at all the same thing as skill in manipulating a ball. It's just a giant non sequitur being slipped in before he begins the math. Which is why he is 100% wrong when he concludes

This is not a problem that is solvable by applying more cognitive effort.

It totally is solvable. The 'cognitive effort' here is 'git gud at pinball, scrub, and stop making excuses for losing', and as he admits in the footnote he didn't include in the LW version, in real life, when adequately incentivized to win rather than find excuses involving 'well, chaos theory shows you can't predict ball bounces more than n bounces out', pinball pros learn how to win and rack up high scores despite 'muh chaos'.

And that is why I don't believe your anecdotal survey responses imply anything good. I think that several or all of those cases, if we were able to investigate them adequately, would turn out to be similar to this pinball essay: a lot of browbeating intimidation-by-math, possibly completely valid insofar as it went, but ultimately, proving an irrelevant claim and the problem in fact soluble.

I am not sure if this is the answer you're looking for, but it is the one I've come to prefer, and others have given the standard answer already.

You can write out a Bayesian algorithm to determine what happens. The program specifics determine the result. Two that come to mind of many possible results:

• After making an observation which had prior probability 0%, the program returns an 'cannot divide by 0' error.
• The program updates on the observation, but this rules out the entirety of its probability-space, and it soon fails to find expected items in an empty set.

Bayes' theorem is an algorithm which is used (and selected for) because it helps predict the world, rather than something with metaphysical status.

This usefulness also makes it selected for (in complex worlds where agents have incomplete information, like this one). We can also imagine different mathematical worlds where Bayes' theorem is not useful.

I didn't so much find any specific thing in particular, as I found a sensation of excitement that had been missing from my life. I thought about people I could talk to, games I could play, shows I could watch. I am excited to do this again next Hammertime repeat.

I am 90% sure that most AI Safety talent aren't thinking hard enough about what Neglectedness. The industry is so nascent that you could look at 10 analogous industries, see what processes or institutions are valuable and missing and build an organisation around the highest impact one. 

The highest impact job ≠ the highest impact opportunity for you!

 

Curated. I appreciated reading this attempt to actually get to the bottom of a simple, widely popularized narrative. It's a helpful datapoint about how reliable narratives are that are spread around our civilization, and how much work is actually involved in checking what actually happened.

I suspect experiments with almost-genetically identical twin tests might advance our understanding about almost all genes except sex chromosomes.

Sex chromosomes are independent coin flips with huge effect sizes.  That's amazing!  Natural provided us with experiments everywhere!  Most alleles are confounded (e.g.. correlated with socioeconomic status for no causal reason) and have very small effect sizes.

Example: Imagine an allele which is common in east asians, uncommon in europeans, and makes people 1.1 mm taller.  Even though allele causally makes people taller, the average height of the people with the allele (mostly asian) would be less than the average height of the people without the allele (mostly European).  The +1.1 mm in causal height gain would be drowned out by the ≈-50 mm in Simpson's paradox.  Your almost-twin experiment gives signal where observational regression gives error.

That's not needed for sex differences.  Poor people tend to have poor children.  Caucasian people tend to have Caucasian children.  Male people do not tend to have male children.  It's pretty easy to extract signal about sex differences.

(far from my area of expertise)

I agree, but I'd lump all of that into "Analyze the circumstances that caused it". Maybe I should've included more external examples like these

Ah, sorry that one went unfixed for as long as it did; a fix is now written and should be deployed pretty soon.

This method is interesting to me and I'd like to get into it someday. Personally I keep finding that whenever I decline to write something down, that one thing will come back to bite me a few days later (because I'd forgotten it). Do you find that you're able to mentally keep track of things better than before, even if they're just vaguely in the back of your mind?

Remember the proof that humans can't get high scores playing pinball because 'chaos theory' [LW · GW]? 

Can you point to where the post says this? Because I read it as saying "It is impossible to predict a game of pinball for more than 12 bounces in the future" and "Professional pinball players try to avoid the parts of the board where the motion is chaotic."