LessWrong 2.0 Reader

Sus.

That's a lie.

Dunno. She might be in danger from Duncan Andrew Sabien.

I'm now suing him for libel and criminal harassment.

Leverage Research.

Does the Solomonoff Prior Double-Count Simplicity?

Question: I've noticed what seems like a feature of the Solomonoff prior that I haven't seen discussed of any intros I've read. The prior is usually described as favoring simple programs through its exponential weighting term, but aren't simpler programs already exponentially favored in it just through multiplicity alone, before we even apply that weighting?

Consider Solomonoff induction applied to forecasting e.g. a video feed of a whirlpool, represented as a bit string $x$. The prior probability for any such string is given by:
$P\left(x\right)=\sum _{p:U\left(p\right)=x}\frac{1}{2^{\left|p\right|}}$
where $p$ ranges over programs for a prefix-free Universal Turing Machine.

Observation: If we have a simple one kilobit program $p_1$ that outputs prediction $x_1$, we can construct nearly $2^{1000}$ different two kilobit programs that also output $x_1$ by appending arbitrary "dead code" that never executes. 

For example:
DEADCODE="[arbitrary 1 kilobit string]"
[original 1 kilobit program $p_1$]
EOF
Where programs aren't allowed to have anything follow EOF, to ensure we satisfy the prefix free requirement.

If we compare $p_1$ against another two kilobit program $p_2$ outputting a different prediction $x_2$, the prediction $x_1$ from $p_1$ would get $2^{1000-\left|G\right|}$ more contributions in the sum, where $\left|G\right|$ is the very small number of bits we need to delimit the DEADCODE garbage string. So we're automatically giving $x_1$ ca.  $2^{1000}$ higher probability – even before applying the length penalty $\frac{1}{2^{\left|p\right|}}$. $p_1$ has less 'burdensome details', so it has more functionally equivalent implementations. Its predictions seem to be exponentially favored in proportion to its length $\left|p_1\right|$ already due to this multiplicity alone.

So, if we chose a different prior than the Solomonoff prior which just assigned uniform probability to all programs below some very large cutoff, say ${10}^{90}$ bytes:
$P\left(x\right)=\frac{\sum _{p:\left|p\right|\le {10}^{90}}\frac{1}{2^{\left|p\right|}}}{2^{{10}^{90}}}$

and then followed the exponential decay of the Solomonoff prior for programs longer than ${10}^{90}$ bytes, wouldn't that prior act barely differently than the Solomonoff prior in practice? The only difference would seem to be the tiny $2^{\left|G\right|}$ prefix term to start the garbage string, which doesn't depend on the cutoff and would become irrelevant almost immediately.^[1] 

Am I missing something here?
 

1. ^{^}

   Context for the question: Multiplicity of implementation is how simpler hypotheses are favored in Singular Learning Theory [? · GW] despite the prior over neural network weights usually being uniform. I'm trying to understand how those SLT statements about neural networks generalising relate to algorithmic information theory statements about Turing machines, and Jaynes-style pictures of probability theory.

The log-returns are not linear in the bits. (They aren't even constant for a given level of bits.)

For example, say the market is 1:1, and you have 1 bit of information: you think the odds are 2:1, then Kelly betting you will bet 1/3 of your bankroll and expect to make a ~20% log-return.

Say the market was 2:1, and you had 1 bit of information: you think the odds are 4:1, then Kelly betting, you will bet 1/5 of your bankroll and expect to make a ~27% log-return.

We've already determined that quite different returns can be obtained for the same amount of information.

To get some idea of what's going on, we can plot various scenarios, how much log-returns does the additional bit (or many bits) generate for a fixed market probability. (I've plotted these as two separate charts - one where the market already thinks an event is evens or better, 

For example, we can spot where the as the market thinks an event is more likely (first chart, 0 bits in the market, 1 bit in the market, etc) and looking at the point where we have 1 bit, we can where we expect to make 20%, then 27% [the examples I worked through first], and more-and-more as the market becomes more confident.

You are not alone. Paul Graham has been writing essays for a long time, and he is revising and rewriting a lot too. Here you can see him write one of his essays as an edit replay.

Also: "only one sentence in the final version is the same in the first draft."

Also:

The drafts of the essay I published today. This history is unusually messy. There's a gap while I went to California to meet the current YC batch. Then while I was there I heard the talk that made me write "Founder Mode." Plus I started over twice, most recently 4 days ago.

For those who would like a hint.

In English, "And" generally indicates addition, "Per" division.

Now consider which of the following makes sense:

Ferrets and seconds

Ferrets per second

Why are you confident it's not the other way around? People who decide to pursue alignment research may have prior interest or experience in ML engineering that drives them towards mech-interp.

This sounds like it's using Russell's theory of descriptions, in that you're replacing "Colorless green ideas do Y" with "For all X such that X is a colorless green idea, X does Y." Not everyone agrees that this is a correct interpretation, in part because it seems that statements like "Dragons are attacking Paris" should be false.

I think it would be reasonable to say that "colorless green ideas" is not just a set of objects in which there are no existing members, but meaningless (for two reasons: "colorless" and "green" conflict, and ideas can't be colored, anyway). I think that was Chomsky's intention—not to write a false sentence, but a meaningless one.