Artificial general intelligence is here, and it's useless

Thanks for posting this here; the downvotes are probably because you don't seem to understand the positions you are arguing against, and since those positions are what we are famous for here, we get even more salty than we would if you were shitting on someone else. :)

[edit: Looking back, this is an unusually harsh tone for me. I sincerely regret any hurt it may have caused. It seemed to me to be justified retaliatory snark, given the harsh language you used to describe people like me who are concerned about AGI.]

Here's my point-by-point lightning rebuttal:

AGI is here, as of the time of writing there are an estimate of 7,714,576,923 AGI algorithms residing upon our planet. You can use the vast majority of them for less than 2$/hour. They can accomplish the vast majority intellectual task that can be well-defined by humans, not to mention they can invent new tasks themselves.

There's a big difference between being able to accomplish tasks that can be well-defined by humans, and being able to accomplish all strategically & economically relevant tasks. The latter is what people are talking about when they talk about superintelligence (See: Bostrom, Superintelligence)

Also, it's debatable that current algorithms are better than humans at the vast majority of well-defined intellectual tasks. Here's an intellectual task that is well-defined: Produce a 2-hour video that will make at least $10M at the box office this summer. Here's another: Tell me which of these thousand stocks will yield the highest returns if I invest my savings in it, over the next year.

They are capable of creating new modified version of themselves, updating their own algorithms, sharing their algorithms with other AGIs and learning new complex skills.

Yeah but they aren't good at it, or at least, there are lots of very useful modifications and updates that still require humans, not to mention even more possible modifications that humans can't do but could in principle be done by AGI.

So, if we are agreed on the fact that 70 billion people wouldn't be much better than 7 billion, that is to say, adding brains doesn't scale linearly… why are we under the assumption that artificial brains would help ?

Adding brains doesn't scale linearly but it does scale to some significant extent. See e.g. here for fun discussion. To answer your question, as Bostrom might, artificial brains can differ from human brains in speed, organization, and quality/intelligence. And they tend to be cheap, too. So hypothetical future AGI systems might be like humans only much faster, or like humans only much cheaper, or like humans only much better at working in teams, or... just a lot smarter than humans. (that's the point you are going to discuss later I take it)

If we disagree on that assumption, than what's stopping the value of human mental labor from sky-rocketing if it's in such demand ? What's stopping hundreds of millions of people with perfectly average working brains from finding employment ?

I'm not sure I follow your dilemma here. Why is it that the value of average human mental labor must skyrocket, if we think that 70 billion people would be better than 7 billion?

Well, you could argue, it's about the quality of the intelligence, not all humans are intellectually equal. Having a few million artificial Joe nobody wouldn't help the world much, but having even a few dozen Von Neumanns would make a huge difference. Or, even better, it's about having an AI that's more intelligent than any human that we've yet encountered.

Yes. Though as mentioned before, there's also speed, cheapness, and coordination ability to consider. They would make AGI world-shakingly powerful even if it was no smarter than me.

This leads me to my second point.

2. Defining intelligence

How does one define an intelligent human or intelligence in general ?

Good question. Have you looked at the literature on the subject, with respect to AGI at least? Bostrom spends a chapter or so of his book on this question, and e.g. Shane Legg talks about it in his dissertation and some of his published work.

An IQ test is the go-to measure of human intelligence used by sociologists and psychologists.

However, algorithms can readily out-score humans in IQ tests, for a very long time, and they are able to do so reliably with more and more added constraints.

The problem here is that IQ tests are designed for humans… not machines.

Yes.

But, let's assume we come up with a "machine intelligence quotient" (MIQ) test that our potential AGI friends cannot use their perks to "cheat" on.

But how do we design it to avoid the pitfalls of a human IQ test when taken to the extremes ? Our purpose, after all, is not to "grade" algorithms with this test in a stagnant void, but to improve them in such a way that scoring higher on the tests means they are more "intelligent" in general.

In other words, this MIQ test needs to be very efficient at spotting intelligence outliers.

If we come back to an IQ test, we'll notice it's rather inaccurate at the thin ends of the distribution.

Have you ever heard of Marilyn vos Savant or Chris Langan or Terence Tao or William James Sidis or Kim Ung-yong ? These are, as far as IQ tests are concerned, the most intelligence members that our species currently contains.

While I won't question their presumed intelligence, I can safely say their achievements are rather unimpressive. Go down the list of high IQ individuals and what you'll mostly find is rather mediocre people with a few highly interesting quirks (can solve equations quickly, can speak a lot of languages, can memorize a lot of things… etc).

This relates to Goodhart's Law [LW · GW]. IQ is a proxy for intelligence, and intelligence is a proxy for world-shaking-ability. So yeah we should expect there to be outliers that are very high in the proxy but not in the other thing.

Once you consider the top 100,000 or so, there is most certainly a great overlap with people that have created impressive works of engineering, designed foundational experiments, invented theories that explain various natural phenomenon, became great leaders… etc (let's call these people "smart", for the sake of brevity).

But IQ is not a predictor of that, it's more of a filter. You can almost guarantee that a highly smart person (as viewed by society) will have a high IQ, but having the highest IQ doesn't even guarantee you will be in the top 0.x% of "smart" people as defined by your achievements.

I don't remember where I saw this (Gwern maybe?) but I think I read that no, IQ continues to be predictive even fairly far out into the tail.

So even if we somehow manage to create this MIQ to be as good of an indicator of whether or not a machine is intelligent as IQ is for humans, it will still be a bad criterion of benchmark against.

Indeed, the only reason IQ is a somewhat useful marker for humans is because natural selection did not use IQ, start having some mad-scientists IQ-based human optimization farms and soon enough you might end up with individuals that can score 300 on an IQ test but can't hold civil conversations or operate in society or tie their shoelaces.

This seems like an obvious point, a good descriptor of {X} becomes bad once it starts being used as a guidelines to maximize {X}. This should be especially salient to AI researchers and anyone working in automatic differentiation based modeling, optimizing a model for a known criterion (performance on training data) does not guarantee success on future similar data (testing/validation set), indeed, optimizing it too much can lead to a worst model than stopping at some middle ground.

Nice--that is basically exactly Goodhart's Law. OK, we are on the same page here. And actually I think this is an interesting point you are making, that I hadn't considered before. If intelligence is a proxy for world-shaking-ability, then maybe superintelligence won't be that powerful after all... Doesn't seem right to me, but I gotta count it as a decent and novel argument at least. I guess the objection that comes to mind is that this argument might be too general, it might prove too much. For example, vehicular speed is only a proxy for how long a transatlantic journey takes--there are other things to consider, like stops, loading and unloading times, etc. But it would be foolish to say, prior to the advent of planes, "So now we are about to invent aeroplanes that can travel very quickly. But we shouldn't expect transatlantic journey times to decrease, because vehicular speed is only a proxy for journey time!"

Intelligence may be only a proxy for world-shaking-ability, but remember, AI scientists are not just optimizing for MIQ, they are optimizing for world-shaking-ability. More powerful, impactful algorithms will be sought out, replicated, modified and expanded on.

We might be able to say "An intelligent algorithm should have an MIQ of at least 100", but we'll hardly be able to say "Having an MIQ of 500 means that an algorithm has super-human intelligence".

The problem is that we aren't intelligent enough to define intelligent. If the definition of intelligence does exist, there is no clear path to finding out what it is.

Even worse, I would say it's highly unlikely that the definition of intelligence exists. What I might consider intelligent is not what you would consider intelligent… our definitions may overlap to some extent, we'll likely be able to come up with a common definition of what constitutes "average" intelligence (e.g. the IQ test), but they would diverge towards the tail.

We don't actually need a definition of intelligence in order to be worried about the possible harmful effects of smarter-than-us AGI. See this paper for more.

Most people will agree as to whether or not someone is somewhat intelligent, but they will disagree to no end on a "most intelligent people in the world" list.

Well, you may say, that could indeed be true, but we can judge people by their achievements. We can disagree all day on whether or not some high IQ individual like Chris Langan is a quack numerologist or a misunderstood god. But we can all agree that someone like Richard Feynman or Tom Mueller or Alan Turing is rather bright based on their achievements alone.

Yes.

Which brings me to my third point.

3. Testing intelligence

The problem of our hypothetical superhuman AGI, since we can't come up with a simple test to determine it's intelligence, is that it would have to prove itself as capable.

This can be done in three ways:

Use previous data and see if the "AI" is able to perform on said data as well or better than humans.

Use very good simulations of the world and see if the "AI" is able to achieve superhuman results competing inside said simulations.

Give the "AI" the resource to manifest itself in the real world and act in much the same way the human would (with all the benefits of having a computer for a brain).

Approach number (1) is how we currently train ML algorithms, and it has the limitation of only allowing us to train on very limited tasks where we know all the possible "paths" once the task is complete.

For example, we can train a cancer detecting "AI" on a set of medical imaging data, because it's rather easy to then take all of our subjects and test whether or not they "really" have cancer using more expensive methods or waiting.

It's rather hard to train a cancer curing "AI", since that problem contains "what ifs" that can't be explored. There are limitless treatment options and given that a treatment fails (the person dies of cancer)… we can't really go back and try again to see what the "correct" solution was.

I wrote a bit more about this problem here, if you're interested in understanding it a bit more. But, I assume that most readers with some interest in statistics and/or machine learning have stumbled upon this issue countless times already.

This can be solved by (2), which is creating simulations in which we can test an infinite amount of hypothesis. The only problem being that creating simulations is rather computationally and theoretically expensive.

To go back to our previous example of cancer curing "AI", currently the "bleeding edge" of biomolecular dynamics simulation, is being able to simulate a single medium-sized gene, in a non reactive substance, for a few nanoseconds, by making certain intelligent assumptions that speed up our simulation by making it a bit less realistic, on a supercomputer worth a few dozens of millions of dollars… Yeah, the whole simulation idea might not work out that well after all.

Coincidentally, most phenomenon that can be easily simulated are also the kind of phenomenon that are either very simplistic or fall into category (1), go figure.

So we are left with approach (3), giving our hypothetical AGI the physical resources to put its humanity-changing ideas into practice. But… who's going to give away those resources ? Based on what proof ? Especially when the proposition here is that we might have to try out millions of these AGIs before one of them is actually extremely smart, much like we do with current ML models.

The problem, of course, is that resources are finite and controlled by people (not in the sense that they are stagnant, but we can't create an infinite amount of resources on demand, it takes time). Nothing of extrinsic value is free.

Which leads me to my fourth point about AGI being useless.

So yeah, it might cost us some real-world resources to test our AGI in the real world, and if we don't do that, we'll be stuck to testing it in simulation which has limitations.

So what?

This is already true of current ML systems, but it doesn't stop progress or even slow it down much.

4. The problem of gradual tool building

The process by which humanity advances is, once you boil it down, one of building new tools using previous tools.

People in the bronze age weren't unable to smelt steel because they didn't have the intelligence to "figure it out", but because they didn't have the tools to reach the desired temperature, evaluate the correct iron & carbon mixture, mine the actual iron out of the ground and establish the trade networks required to make the whole process viable.

As tools of bronze helped us build better tools of bronze, that helped us discover more easy to mine iron deposits, build ships and caravans to trade the materials needed to process said iron and build better smelters, we were suddenly able to smelt steel. Which leads us to being able to build better and better tooling out of steel… etc, etc.

Human civilization doesn't advance by breeding smarter and smarter humans, it advanced by building better and better tools.

It was a bit of both, presumably. But yes, point taken.

The gradual knowledge that we acquire is mostly due to our tools. We don't own our knowledge of particle physics or chemistry to a few intelligent blokes that "figured it out". We owe it to the years of cumulative tool building that lead us to being able to build the tools to perform the experiments that gave us insights into the very world we inhabit.

Take away Max Planck and you might set quantum mechanics back by a few years, but in a rather short time someone would have probably figured out the same things he did. This is rather obvious when you look at multiple discoveries throughout history, that is to say, people discovering the same thing, at about the same time, without being aware of each other’s works.

Have Max Planck be born among a tribe of hunter gatherers in the neolithic period, and he might become a particularly clever hunter or shaman… but it's essentially impossible for him to have the tooling which allowed him to make the same discoveries about nature as 20th century Plank.

However, to some extent, the process of tool building is inhibited by time and space. If we decided to build a more efficient battery, or a more accurate electronic microscope, or a more accurate radio telescope, we wouldn't be limited by our intelligence, but by the thousands of hours required for our factories to build the better tools required to build better factories to build even better tools required to build even better factories in order to build even more amazing tools… etc.

Thousands of amazing discoveries, machines and theories lie within our grasp and one of the biggest bottlenecks is not intelligence but resources.

Not matter how smart your interstellar spaceship design is, you will still need rare metals, radioactive materials, hard to craft carbon fiber and the machinery to put it all together. Which is rather difficult, since we've collectively decided those resources are much better spent on portable masturbation aids, funny looking things to stick on our bodies and giant bombs just in case we need to murder all of humanity.

So, would a hypothetical superintelligent AGI help this process of tools building ? Most certainly. But it will probably end up with the same bottlenecks people that want to create amazing things face today, other people not wanting to give up their toys and the physical reality requiring the passage of time to shape.

Eventually it will end up with bottlenecks, yes. Speed of light, speed of factory assembly robot, speed of DNA synthesis, etc. But you've gotta admit, there's a lot of room for improvement before we get to that point.

Don't get me wrong, I'm not necessarily blaming people for choosing to focus on 3rd printing complexly shaped water-resistant phallus-like structures instead of using those resources to research senolytic nanobots. As I've mentioned before, defining intelligence is hard, and so is defining progress. What might seem "awesome" for the AGI reading this article could be rather boring or stupid for a few other billions of AGIs.

I'm not sure I get your point at the end--are you saying that different AGIs might value different things? Yes, but how is that a reason not to be concerned?

In conclusion

Artificial general intelligence is something we have plenty of here on Earth, most of it goes to waste, so I'm not sure designing AGI based on a human model would help us much.

Disagree.

Superhuman artificial general intelligence is not something that we can define, since nobody has come up with a comprehensive definition of intelligence that is self-sufficient, rather than requiring real world trial and error.

There is a literature on this, which I agree hasn't been fully satisfactory, but I think it's done good enough. Besides, we don't need a definition to be concerned.

Superhuman artificial general intelligence is not something we can test, since we can't gather statistically valid training datasets for complex problem and we can't afford to test via trial and error in the real world.

All the more reason to be concerned! People will create and deploy these things before they have tested them properly!

Even if superhuman artificial intelligence was somehow created, there's no way of knowing that they'd be of much use to us. It may be that intelligence is not the biggest bottleneck to our current problems, but rather time and resources.

What? Of course they would be of much use to us--if they wanted to be, at least. Even if intelligence isn't our biggest bottleneck, it is a limitation and so overcoming it would help greatly. There's lots of room for improvement.

Argument: Intelligence lets us design new tech faster. New tech is super useful. Therefore, superhuman AGI could be super useful.

If you think you're specific business case might require an AGI, feel free to hire one of the 3 billion people living in South East Asia that will gladly help you with tasks for a few dollars an hour. It's likely to be much cheaper, Amazon is already doing it with Alexa, since it turns out to be somewhat cheaper than doing it via machine learning.

Again, the thought is that AGI could be better than humans in speed, quality, organization, or cheapness, not just equal to humans.

Is that to say I am "against" the machine learning revolution ?

No, fuck no, I'd be an idiot and a hypocrite if I thought machine learning wouldn't lead to tremendous human progress in the following decades.

It seems like some of your arguments are too general; they prove too much; for example, they could just as well be used to argue for the conclusion that ML won't lead to tremendous human progress.

I specifically wanted to work in the area of generic machine learning. I think it's the place to be in the next 10 or 20 years in terms of exciting developments.

But we have to stop romanticizing or fear-mongering about the pointless concept of a human-like intelligence being produced by software. Instead, we should think of machine learning (or "AI", if you must really call it that) as a tool in our great arsenal of thinking tools.

Sticks and stones may hurt my bones, but words will never hurt me...

Machine learning is awesome if you apply it to the set of problems it's good at solving and if we try to extent that set of problems by being better and collecting and building algorithms we might be able to accomplish some amazing feats. But the idea that an algorithm that can mimic a human would be of particular use to us, is similarly silly to the idea that a hammer which also serves as a teaspoon would revolutionize the world of construction. Tools are designed to be good at their job, not much else.

Again, the thought is that AGI could be better than humans in speed, quality, organization, or cheapness, not just equal to humans. And it's pretty obvious why AGI of this sort would be useful.

...

If you'd like to talk more about this with me, I'd be happy to continue the conversation! Send me a PM, maybe we can skype or something.

I think it's in the spirit of the conversation here.

AGI is certainly a topic of great interest here, but the "spirit" of this post is pretty far off of LessWrong standards:

One of the most misunderstood ideas that's polluting the minds of popular "intellectuals", many of them seemingly accustomed with statistics and machine learning, is the potential or threat that developing an artificial general intelligence (AGI) would present to our civilization.

This myth stems from two misunderstanding of reality.

Ouch? "polluting" minds? "intellectuals" in scare quotes? "Myth" from "misunderstanding of reality"? We call ourselves "rationalists" here. Understanding reality (and winning) is exactly what we value. It looks like you're deliberately trying to insult us. I think this explains the downvotes.

But I have a fairly thick skin and care more about the truth than whether you're trying to being mean or not, so I would have pretty much overlooked this if you had a good argument. (Rationalists love good arguments, especially when they contradict our beleifs.) Sadly, this is not the case. We've tread this ground before.

We already have AGIs

You missed the "artificial" part, and that matters. Yes, a human is a General Intelligence. You spend a section elaborating on that, but it's not news to anyone. Human brains are very limited by biology in ways that simply don't apply to machines. Fitting through the birth canal, for example.

How does one define an intelligent human or intelligence in general ?

I would point to AIXI as the most "intelligent" agent possible in the limit. This has a very formalized definition of "intelligent agent". Intelligence doesn't have to be human-like at all. Think of it as an "optimization process" or outcome pump [LW · GW] instead if "intelligence" has too much anthropomorphic baggage for you.

I had heard of Terrence Tao, but I do agree that IQ is an imperfect measure of g. And the g factor is not at all the same thing as rationality. But it's not relevant given the AIXI formalism.

Human civilization doesn't advance by breeding smarter and smarter humans, it advanced by building better and better tools.

Not exactly true. There are native tribesmen that could walk naked into the jungle and come out in a few days with clothes, weapons, and a basket full of fruit and game. You couldn't do that. Their technology had to "advance" to that point. I saw a textbook on how to build an entire machine shop starting with a lathe. I saw a video of someone building an electric battery using only stone-age tools and materials foraged from nature. "Technology" means technique more than it means "tools". Notice the Greek root techne or "art" in both words. Human civilization advanced by accumulating culture, and that component of our smarts is evolving faster than our DNA. I would identify the invention of the printing press as the point where civilization really took off. It's not that resources are irrelevant, obviously they're required, but they're not as important as you seem to think.

What really hit this point home for me was That Alien Message [LW · GW]. You'll have to respond to that, at least, before I can take you seriously.

There are good reasons (like biology) to think that molecular nanotechnology is physically feasible. Once that's started, it probably wouldn't require more than CHON (carbon, hydrogen, oxygen, nitrogen) from the air and energy to produce the kind of radical abundance that makes all the other resources either obsolete or really cheap to acquire.

One reason why artificial intelligence might be more useful than a human for some service is because artificial intelligence is software, and therefore you can copy-paste it for every service that we might want in an industry.

Recruiting and training humans takes time, whereas if you already have an ML model that performs well on a given task, you only need to acquire the relevant hardware to run the model. If hardware is cheap enough, I can see how using artificial intelligence could be much cheaper than spending money on {training + recruiting + wages} for a human. Automation in jobs such as audio transcription exemplify this trend -- although I think the curve for automation is smooth as the software services require continuously less supervision over time as they improve.

The entire argument seems to boil down to

3. Give the "AI" the resource to manifest itself in the real world and act in much the same way the human would (with all the benefits of having a computer for a brain)...

So we are left with approach (3), giving our hypothetical AGI the physical resources to put its humanity-changing ideas into practice. But… who's going to give away those resources ? Based on what proof ?

We already connect AI systems to the real world, for example Facebook's algorithms that learn based on user behavior. There's nothing implausible about this.

Besides that:

They [humans] are capable of creating new modified version of themselves, updating their own algorithms...

Human ability to self-modify is very limited. Most of our algorithms are opaque to our conscious mind.

...I would say it's highly unlikely that the definition of intelligence exists.

There is already progress in defining intelligence, starting from Legg and Hutter 2007.

What's truly jarring, though, is the passage

Have you ever heard of Marilyn vos Savant or Chris Langan or Terence Tao or William James Sidis or Kim Ung-yong ? These are, as far as IQ tests are concerned, the most intelligence members that our species currently contains.

While I won't question their presumed intelligence, I can safely say their achievements are rather unimpressive.

Terence Tao is a fields medalist. You bet I heard of Tao. If a Fields Medal is "rather unimpressive", I can't imagine what is impressive.

I'm just gonna leave this here and highlight points 2 and 3.