LessWrong 2.0 Reader

Depending on what one means by 'learn' this is provably impossible. The reason has nothing to do with the transformer architecture (which one shouldn't think of as a canonical architecture in the grand scheme of things anyway).

There is a 2-state generative HMM such that the optimal predictor of the output of said generative model provably requires an infinite number of states. This is for any model of computation, any architecture.

Of course, that's maybe not what you intend by 'learn'. If you mean by 'learn' express the underlying function of an HMM then the answer is yes by the Universal Approximation Theorem (a very fancy name for a trivial application of the Stone-Weierstrass theorem).

Hope this helped. 😄

$\omega$ isn't a reference frame; rather, if $\psi$ is a world then ${\omega }^{-1}\left(\right\{\psi \left\}\right)$ aka $\{\iota \in I\mid \omega (\iota )=\psi \}$ are the reference frames for $\psi$.

Essentially when dealing with generalized reference frames that contain answers to questions such as "who are you?", the possible reference frames are going to depend on the world (because you can only be a real person, and which real people there are depends on what the world is). As such, "reference frames" don't make sense in isolation, rather one needs a (world, reference frame) pair, which is what I call an "interpretation".

Thanks, didn't know about the low basis theorem.

U axiomatizes a consistent guessing oracle producing a model of T. There is no consistent guessing oracle applied to U.

In the previous post I showed that a consistent guessing oracle can produce a model of T. What I show in this post is that the theory of this oracle can be embedded in propoaitional logic so as to enable probability preserving translations.

LS shows to be impossible one type of infinitarian reference, namely to uncountably infinite sets. I am interested in showing to be impossible a different kind of infinitarian reference. "Impossible" and "reference" are, of course, interpreted differently by different people.

Is there some theoretical result along the lines of "A sufficiently large transformer can learn any HMM"?

The test scores me as 'normal' with 29/36. I remember doing a similar (maybe the same) test and scoring decidedly below average about two years ago.

I understand the attraction of having this skill trainable in its own context like flashcards but consider it a false shortcut.  I think it is more about directing attention. 

Setting aside a few cycles of my attention to practice in every day life worked for me and I think it should be wildly superior to treating it as a problem of categorizing features.

1. You get so much more context to infer from and that hints at things you should be able to detect. After all, the true version of the skill is not 'detect <basic emotion x >' but 'emulate people roughly and extract information'. For that to actually happen you want to keep detecting new features and explore them. Not be x% better at categorizing separating desire vs. attention.
2. You also train actually using the skill in the background (that is becoming aware that a person feels x instead of just being able to answer if you should happen to ask yourself about what they might feel). This is also the hard part in my opinion. 

It is frustrating, but every time I want to modify anything about my mind it comes down to a mindfulness exercise.
 

Maybe Hanson et al.'s Grabby aliens model? @Anders_Sandberg [LW · GW]  said that some N years before that (I think more or less at the time of working on Dissolving the Fermi Paradox), he "had all of the components [of the model] on the table" and it just didn't occur to him that they can be composed in this way. (personal communication, so I may be misremembering some details). Although it's less than 10 years, so...

Speaking of Hanson, prediction markets seem like a more central example. I don't think the idea was [inconceivable in principle] 100 years ago.

It would be interesting for people to post current research that they think has some small chance of outputting highly singular results!

Grothendiek seems to have been an extremely singular researcher, various of his discoveries would have likely been significantly delayed without him. His work on sheafs is mind bending the first time you see it and was seemingly ahead of its time.