The Cluster Structure of Thingspace

But if a definition works well enough in practice to point out the intended empirical cluster, objecting to it may justly be called "nitpicking".

You should probably put in a disclaimer excepting mathematics from this -- assuming that you agree it should be excepted. (That is, assuming you agree that "Aristotelian" precision -- what mathematicians call "rigor" -- is appropriate in mathematics.)

What's interesting about "Thingspace" (I sometimes call it "orderspace") is that it flattens out all the different combinations of properties into a mutually exclusive space of points. An observable "thing" in the universe can't be classified in two different points in Thingspace. Yes you can have a range in Thingspace representing your uncertainty about the classification (If you're a mere mortal you always have this error bar) but the piece-of-universe-order you are trying to classify is in ideal terms only one point in the space.

IMO this could explain the way we deal with causality. Why do we say effects have only one cause? Where does the Principle of Sufficient Reason come from? The universe is not actually quantized in pieces that have isolated effects on each other. However, causes and effects are "things", they are points in Thingspace and as "things" they actually represent aggregates, bunches of variable values that when recognized as a whole have, by definition, unique cause-effect relationships with other "things". I see causality as arrows from one area of thing space to another. Some have tried to account for causality with complex Bayesian networks based on graph theory that are hard to compute. But I think applying causality to labeled clusters in Thingspace instead of trying to apply it to entangled real values seems simpler and more accurate. And you can do it at different levels of granularity to account for uncertainty. The space is then most useful classified hierarchically into an ontology. Uncertainty about classification is then represented by using bigger, vaguer, all encompassing clusters or "categories" in the Thingspace and high level of certainty is represented by a specified small area.

I once tried (and pretty much failed) to create a novel machine learning algorithm based on a causality model between hierarchical EM clusters. I'm not sure why it failed. It was simple and beautiful but I had to use greedy approaches to reduce complexity which might have broken my EM-algorithm. Well at least it (just barely) got me a masters degree. I still believe in my approach and I hope someone will figure it out some day. I've been reading and questioning the assumptions underlying all of this lately and specially pondering the link between the physical universe and probability theory and I got stuck at the problem of the arrow of time which seems to be the unifying principle but which also seems not that well understood. A well... maybe in another life.

If we find a category that has a very tight cluster, such that for that category it's reasonably straightforward to define that cluster, and only a tiny handful of distant outliers that seem to only shakily fit with the rest of the category, than it may be wise in some cases to conciously redefine that category in terms of the explicit definition that represents the tight cluster, and maybe use a different category, or a broader one, to represent or include those outliers.

Psy-Kosh, dangerous heuristic. Isn't that how the Nazis thought of the Jews? We should look first and foremost at ways things fit into clusters, not ways they don't - otherwise nine-fingered Fred gets ruled out of being human at an early hurdle. I'm sure you'll agree Fred fits better into 'human' than 'broad general-human-type', despite his missing digit.

Ostriches are a long way from that tight, feathery birdy cluster, but we leave them out of 'general bird-ness' at our peril. Mr Ostrich scores 84% on birdiness, not 16% on not-birdiness. (He also scores in the high 60s in dinosauriness, but that's another matter.)

I sense these 6 essays on cognitive semantics are going to bring us back to transhumanism sooner or later. As of right now, whatever the radial distance from the prototype, and except on the Island of Dr Moreau, you are DEFINITELY human or definitely not, definitely a bird or definitely not. Pluto is DEFINITELY a pla...... whoops.

or maybe a more sophisticated view that .

?

What are the dimensions of thingspace?

Are "number of sides", "IQ", "age", and "font" all dimensions?

And what are the points in thingspace? It sounds like they include anything that is somewhat "mother" and anything that is somewhat "robin". (And I should think thingspace is a point in thingspace too.)

I think this post makes some good points, the main one, for me, being that words are centers of (indefinitely extending) clusters rather than boundaries of sets. But I think the notion of thingspace rests on shaky foundations: it assumes the world is broken down into things and those things have attributes.

We don't all share the same thingspace do we?

Thingspace seems rather like cladistics, in which you come up with groups of characteristics and then work out trees of evolutionary descent. Note that this originated in studying the evolution of life on Earth and piecing together the Tree of Life, but is applicable anywhere an evolutionary process can work, e.g. linguistic evolution. Without necessarily going as far as the actual sorting stuff into trees, cladistics may be useful in helping conceptualise thingspace and distance in thingspace.

A thought I recently had: Shouldn't we be interested in "anti-clusters" too? ie, regions of comparatively low density compared to the surroundings/Patterns of stuff that tends to conspicuously fail to happen compared to what would be otherwise expected.

This essay reminds me of Samuel Delany saying that the word "the" seems like a gray ellipse to him, and each adjective modifies the ellipse.

does thingspace remain static? that is; would definitional/structural changes within the space correspond to a folding or reorienting of the space where the clusters become reorganized?

You could give relatively simple verbal intensional definitions to try and lead someone to the bird cluster, yes. But if you had someone who wasn't practically accessible through those verbal communications, how would you do it?

You'd have to show extensional examples, positives and negatives, and indicate the value of each example by some clear and consistent signal.

You couldn't give all possible extensional examples, so you would have to select some. And you couldn't give them all at once, so you'd have to present them in a particular order.

What is the theory for finding optimized selections and orderings of examples for leading the learner to the cluster? How does that theory extend to the more complicated case where you have to communicate the subtypes within the "bird" cluster?

This is one of the many things that the Theory of Direct Instruction that's presented in Engelmann and Carnine's text Theory of Instruction: Principles and Applications addresses. [They call it a "multi-dimensional non-comparative concept" (non-comparative" meaning the value of any example is absolute rather than relative to the last), or "noun" for short.]

And of course, if you had to select and order the presentation of simple verbal definitions/descriptions as examples themselves, the theory would also have application.

Please see here for a clarification of what "someone who wasn't practically accessible through those verbal communications" means, and a more concrete example of teaching the higher-order class 'vehicles' and sub-classes.

Hi there, fairly new here to LW. I'm reading through the sequences in order. went through map and territory and mysterious answers to mysterious questions. Now going through this 37 ways words can be wrong sequence, as its recommended before i delve into reductionism.

Its been said several times that LW tries to cater to a broad audience, but i find myself lost here. I have not extensively studied physics, only having done 1 year of engineering so far, and the physics references here are pretty much unintelligible to me. I don't know what configuration space is, or quaternary coordinates, or thingspace, or what strings are being referred to. I find myself struggling to grasp this post.

EDIT: I've read through this a few times. I still have almost no idea on most of the math, but I'm guessing the "moral" of this post is basically "don't become overly obsessed with definitions"?

The Aristotelian syllogism would run, "Humans have ten fingers, Fred has nine fingers, therefore Fred is not a human" but the way we actually think is "Humans have ten fingers, Fred is a human, therefore Fred is a 'nine-fingered human'."

It's not important to your central claim, but this is the strawmanniest thing since Straw Man came to Straw Town.

A robin's DNA is a highly multidimensional variable, but you can still think of it as part of a robin's location in thingspace—millions of quaternary coordinates, one coordinate for each DNA base—or maybe a more sophisticated view that .

Um. That...?

or maybe a more sophisticated view that .

I guess there was a misformatted link in there or something?

One small (hopefully not too obvious) addition: the cluster-nature of thing-space is dependent on the distance function, and there is no single obviously corrent one. Is a penguin more like an eagle or a salmon? Depends on what you mean by "more like". It's perfectly reasonable to say "right now, the most useful concept of 'more like' is 'last common ancestor' so penguins are more like eagles and 'birds' is a cluster' and then as your needs change to say "right now, the most useful concept of 'more like' is similarity of habitat so penguins are more like salmon and 'sealife' is a cluster."

Before reading this article, I had already been using this visualization technique to think of probability densities. I wonder how common that is? Probably happened because of exposure to statistics.

"Humans have ten fingers, Fred has nine fingers, therefore Fred is not a human" but the way we actually think is "Humans have ten fingers, Fred is a human, therefore Fred is a 'nine-fingered human'."

What I actually thought reading this was: "Frodo is a nine-fingered Hobbit"...

I'm glad to see Eliezer addressed this point.  This post doesn't get across how absolutely critical it is to understand that {categories always have exceptions, and that's okay}.  Understanding this demolishes nearly all Western philosophy since Socrates (who, along with Parmenides, Heraclitus, Pythagoras, and a few others, corrupted Greek "philosophy" from the natural science of Thales and Anaximander, who studied the world to understand it, into a kind of theology, in which one dictates to the world what it must be like).

Many philosophers have recognized that Aristotle's conception of categories fails; but most still assumed that that's how categories must work in order to be "real", and so proving that categories don't work that way proved that categorizations "aren't real".  They them became monists, like the Hindus / Buddhists / Parmenides / post-modernists.  The way to avoid this is to understand nominalism, which dissolves the philosophical understanding of that quoted word "real", and which I hope Eliezer has also explained somewhere.

I found some criticism of this post on a RationalWiki talk page.

For another example, "Clusters in Thingspace" has a number of issues. Most simply, it seriously undersells Aristotle's ability to handle a nine-fingered person. Certainly, if you make 'has ten fingers' part of the definition of human, then you will be able to infer that a person without ten fingers is not a human; nobody, though, has ever seriously put forward such a proposal. For Aristotle's part, he would simply say that having a certain number of fingers is not an essential property of being human (and so should not be factored into the definition). Yudkowsky is also wrong to say that the coordinate point (0,0,5) contains the same information as the HTML color blue. To the contrary, the coordinate point by itself contains no information; it can contain color information only when paired with some interpretation function I (in the case of HTML, the software provides this function). As for where else these ideas can be found, philosophers have been working on conceptual vagueness intensely since the mid-20th century, and cluster concepts were a relatively early innovation. The philosophical literature also has the benefit of being largely free of nebulous speculations about cognition and needless formalism (and the discussion of configuration space here is needless formalism, since Yudkowsky is drawing only qualitative conclusions and the practical constraints on constructing a configuration space even for robins alone are severe). The literature also uses terminology in the ordinary way familiar to everybody engaging these issues professionally (compare Yudkowsky's muddled understanding of intension) and avoids the invention of needless terms like "thingspace", which mainly achieve the isolation of LessWrong from the external literature (whose relative richness and rigor would doubtlessly benefit them far more than the Sequences, the works of a single, self-aggrandizing amateur). That's not to say that there's no good ideas in the article, only that it is unoriginal, muddled, imprecise, and parochial.

What do you guys think?

https://rationalwiki.org/wiki/Talk:LessWrong#EA_orgs_praising_AI_pseudoscience_charity._Is_it_useful.3F

intensional terms

Should probably link to Extensions and Intensions [LW · GW]; not everyone reads these posts in order.