Have general decomposers been formalized?

post by Quinn (quinn-dougherty) · 2020-06-27T18:09:06.411Z · LW · GW · No comments

This is a question post.

Contents

  Answers
    1 George
None
No comments

Hi, I'm working on a response to ML projects on IDA [LW · GW] focusing on a specific decomposer, and I don't know if someone's formalized what a decomposer is in the general case.

Intuitively, a system is a decomposer if it can take a thing and break it down into sub-things with a specific vision about how the sub-things recombine.

Answers

answer by George3d6 (George) · 2020-06-27T21:26:12.152Z · LW(p) · GW(p)

Why is the literature into reversible encoders/autoencoders/embedding generators not relevant for your specific usecase ?

Give an answer to that it might be easier to recommend stuff.

↑ comment by Quinn (quinn-dougherty) · 2020-06-28T17:35:55.391Z · LW(p) · GW(p)

Sorry, I think I might have a superficial understanding of encoders and embeddings. Would you be able to try pointing out for me how decomposition is performed in that case (or point me toward a favorite reading on the subject)? When I think of feeding a sentence into an encoder, I can think of multiple ways in which some compositional structure might be inferred.

I'm drawing up a proof of concept with seq2seq learners right now, but my hypothesis is that they will be inadequate decomposers suitable only for benchmarking a baseline.

Replies from: George3d6
↑ comment by George3d6 · 2020-07-02T00:25:51.768Z · LW(p) · GW(p)

I was asking why because I wanted to understand what you mean by "decomposition".

a system is a decomposer if it can take a thing and break it down into sub-things with a specific vision about how the sub-things recombin

Defines many things.

Usually the goal is feature extraction (think Bert) or reducing the size of a representation (think autoencoders or simpler , PCA)

You need to narrow down your definition, I think, to get a meaningful answers.

Replies from: abramdemski
↑ comment by abramdemski · 2020-07-08T21:10:55.452Z · LW(p) · GW(p)

I think Quinn means factored cognition, which is quite different from autoencoders/embeddings/PCA.

Replies from: quinn-dougherty
↑ comment by Quinn (quinn-dougherty) · 2020-07-10T15:11:55.172Z · LW(p) · GW(p)

Thank you Abram. Yes, factored cognition is more what I had in mind. However, I think it's possible to speak of decomposition generally enough to say that PCA/SVD is a decomposer, albeit an incredibly parochial one that's not very useful to factored cognition.

Like, my read of IDA is that the distillation step is proposing a class of algorithms, and we may find that SVD was a member of that class all along.

No comments

Comments sorted by top scores.