Does the Telephone Theorem give us a free lunch? 2023-02-15T02:13:39.819Z

Abstraction As Symmetry and Other Thoughts 2023-02-01T06:25:53.338Z

Transformer language models are doing something more general 2022-08-03T21:13:42.472Z

Comment by Numendil on Abstraction As Symmetry and Other Thoughts · 2023-02-01T23:49:34.419Z · LW · GW

While I like his idea, it's still not computable.

Comment by Numendil on Abstraction As Symmetry and Other Thoughts · 2023-02-01T20:40:52.475Z · LW · GW

The notion of intelligence as compression is an old one; I believe Marcus Hutter was the first to formalize it back in the early 2000s (this is also where AIXI comes from). The problem with Hutter's formalism is that his definition of compressibility (Kolmogorov complexity) is uncomputable; "find the shortest Turing machine that outputs X" requires unbounded resources even if you have a halting oracle.

I believe that, in this paradigm, the NAH is fundamentally saying: well, for "natural" data, compressibility is computable; there's some minimal representation to which any sufficiently powerful (yet still finite) model will converge. The problem is, therefore, to figure out what a sufficiently powerful model looks like.

Comment by Numendil on Lies Told To Children · 2022-04-15T03:16:21.552Z · LW · GW

Predictions over the course of my reading:

Guessed "dath ilan" from title.

Switched prediction to "Amenta" at mention of red hair.

Switched back to "dath ilan" (with much higher certainty) at the first use of capital-C "Civilization".