Comment by beezlebub33 on Meta Programming GPT: A route to Superintelligence? · 2020-07-13T11:33:31.410Z · LW · GW

I think that the article is very interesting, and that the meta-intelligence point of view is a useful one.

However, I'd like to add the following argument: We need to consider why humans are intelligent, and why they are the intelligent in they way that they are. Other animals get by just fine being less intelligent, including other primates, and perform very well in their environments. The (likely?) answer is that we are intelligent not to perform well against the overall environment but to perform well against each other. Your greatest competitor (evolutionarily speaking) is not gathering food or defending against a lion, but against they person down the street who is trying to take your mate, either overtly by pairing up with them long term or covertly by mating with them when you are not around.

In that environment, it is important for an animal to understand what the motives, intentions, and plan are for their competitors. That means that they need to have a model of them, what they know, what their options are, and what they will decide to do. That is meta-intelligence. Humans are good at forming mental models of certain types, infering consequences, and predicting outcomes. That ability, writ large, is the basis of math and science: form a mental model of a situation, process it, manipulate it, pick a useful outcome.