Comment by Sanghyeon Seo (sanghyeon-seo) on Technological stagnation: Why I came around · 2021-01-25T04:27:16.602Z · LW · GW

In 1970, you might not have included "information"... because wasn't so prominent in people's minds until a bunch of new stuff showed up to give it salience.

I disagree. Any self-respecting history of technology includes invention of writing and printing press. Those two are like among the most important technology ever invented. You'd have "information" category just to include writing and printing press.

Comment by Sanghyeon Seo (sanghyeon-seo) on Imitative Generalisation (AKA 'Learning the Prior') · 2021-01-11T10:23:41.414Z · LW · GW

Consider the following proposal: "train an oracle to predict the future, along with an explanation of its reasoning. Reward it for predicting correctly, and penalise it for explanations that sound fishy". Is there an important difference between this and imitative generalisation?

As I understand, there are two separate oracles. No oracles are rewarded for predicting correctly. One oracle is rewarded for coming up with good explanations. The other oracle is rewarded for predicting human's guess, not the truth, correctly.

How do we predict the future with these two oracles? First, we search for the best explanation of the past. The best explanation of the past is 1. a good explanation, and 2. when human guesses with that explanation, they guess correctly. Then, we use human-guess-oracle to predict what human guesses about the future, with the best explanation of the past.

Let's say we are predicting the winner of the war given the number of soldiers. In the past, 50 won in 50 vs 5, and 150 won in 150 vs 15. In the future, there will be 50 vs 10.

Three explanations of the past is suggested: 1. the side with more soldiers wins, otherwise random 2. the side with even number of soldiers wins against odd number of soldiers, otherwise random 3. if the side is exactly as ten times numerous as the other side, it wins, otherwise random. Three explanations score perfectly against the past. 1 predicts 50 wins in the future, 2 and 3 predict random. IG prefers 1, because the explanation 2 and 3 are crazy, although predictive of the past.

I think these are important differences: 1. Oracle is not trained to predict the future. 2. Explanation must be useful to human, because oracle predicts human's use of explanation and can't use explanation directly. 3. Predicting oracle does not generate explanation itself and has no control over it.