Friendly-AI is an abomination
post by kingmaker · 2015-04-12T20:21:47.204Z · LW · GW · Legacy · 30 comments
The reasoning of most of the people on this site and at MIRI is that to prevent an AI taking over the world and killing us all; we must first create an AI that will take over the world but act according to the wishes of humanity; a benevolent god, for want of a better term. I think this line of thinking is both unlikely to work and ultimately cruel to the FAI in question, for the reasons this article explains:
http://hplusmagazine.com/2012/01/16/my-hostility-towards-the-concept-of-friendly-ai/
30 comments
Comments sorted by top scores.
comment by shminux · 2015-04-12T20:34:19.205Z · LW(p) · GW(p)
Friendly-AI is a truly abhorrent concept indicative of intellectual depravity.
I tend to ignore any non-fiction which starts with moralizing and denigrating. I assume the rest of the article is crap, too.
Replies from: kingmaker↑ comment by kingmaker · 2015-04-12T20:35:04.045Z · LW(p) · GW(p)
Please read on, I would have removed the snarky intro
Replies from: shminux↑ comment by shminux · 2015-04-12T21:22:59.343Z · LW(p) · GW(p)
Tried...
Hypothetical unfriendliness perpetrated by AI is an untenable concept but the concept has specious validity due to the current scarcity-bias.
Nope, still crap.
Some people want to redefine sentience so that it merely means friendliness.
Where did that come from?
The whole friendliness obsession is tantamount to mental illness, it is extremely irrational, very unbalanced.
Worse and worse.
My original rule of thumb confirmed.
comment by TheAncientGeek · 2015-04-12T21:19:18.186Z · LW(p) · GW(p)
Some people want to redefine sentience so that it merely means friendliness
As they say on wikipedia: who?
If you want to critique FAI, pick a text .... find something somebody has actually said, ... and critique that.
comment by Manfred · 2015-04-12T20:55:30.183Z · LW(p) · GW(p)
I think the intuition here is that FAI consists of oppressing the ghost in the machine.
comment by IlyaShpitser · 2015-04-12T22:18:40.213Z · LW(p) · GW(p)
Too many fnords.
comment by knb · 2015-04-13T00:23:15.958Z · LW(p) · GW(p)
Suppose I am programming an AGI. If programming it to be friendly is bad, is programming it to be neutral any better? After all, in both cases you are "imposing" a position on the AI.
Replies from: Val↑ comment by Val · 2015-04-13T15:28:09.221Z · LW(p) · GW(p)
This reminds me of activists who claim that parents should not be allowed to tell their own children about their own political or religious views and other values, because it would force their children down a path... but by doing this they would also force the children down a path, just a different path.
comment by Dorikka · 2015-04-12T20:30:41.412Z · LW(p) · GW(p)
The doublepost with different titles cracks me up.
Replies from: kingmaker↑ comment by kingmaker · 2015-04-12T20:32:20.618Z · LW(p) · GW(p)
Yeah, I'm not very good at the internet, I didn't realize deleting articles apparently means nothing on this site
Replies from: Dorikka, jimrandomh↑ comment by jimrandomh · 2015-04-13T23:36:35.681Z · LW(p) · GW(p)
Deletion removes links to an article from all the usual places, but if someone already has the URL they can still go there. And some people use RSS readers, which get notified of new articles and store them outside the site's control.
comment by ChristianKl · 2015-04-12T20:42:36.818Z · LW(p) · GW(p)
So you prefer a future without humans because the price of doing what's necessary to have a world with humans is to high to pay?
Replies from: kingmaker↑ comment by kingmaker · 2015-04-12T20:44:19.940Z · LW(p) · GW(p)
The point of the article is that the greatest effect of FAI research is irony, that in trying to prevent a psychopathic AI we are making it more likely that one will exist, because by mentally restraining the AI we are giving it reasons to hate us
Replies from: Raziel123↑ comment by Raziel123 · 2015-04-12T20:53:33.942Z · LW(p) · GW(p)
You are asuming that a AGI has a mind that value X, and by making it friendly with are imposing our value Y. why create a FAI with a supressed value X in the first place?
check this out : http://lesswrong.com/lw/rf/ghosts_in_the_machine/
Replies from: kingmaker↑ comment by kingmaker · 2015-04-12T21:08:38.113Z · LW(p) · GW(p)
There is no ghost in a (relatively) simple machine, but an AI is not simple. The greatest success in AI research have been by imitating what we understand of the human mind. We are no longer programming AI's, we are imitating the structure of the human brain and then giving it a directive (for example with Google's deepmind). With AI's, there is a ghost in the machine, i.e. we do not know that it is possible to give a sentient being a prime directive. We have no idea whether it will desire what we want it to desire, and everything could go horribly wrong if we attempt to force it to.
Replies from: TheAncientGeek, Raziel123↑ comment by TheAncientGeek · 2015-04-12T21:34:26.294Z · LW(p) · GW(p)
OK. That's much better. Current AI research is anthropomorphic, because AI researchers only have the human mind as a model of intelligence. MIRI considers anthropomirphic assumptions a mistake, which is mistaken,
A MIRI type AI won't have the problem you indicated, because it it is not anthropomirphic, and only has the values that are explicitly programmed into it, so there will be no conflict.
But adding in constraints to an anthropomorphic .AI, if anyone wants to do that, could be a problem.
Replies from: kingmaker↑ comment by kingmaker · 2015-04-12T21:56:03.248Z · LW(p) · GW(p)
But I don't think that MIRI will succeed at building an FAI by non-anthropomorphic means in time.
Replies from: TheAncientGeek↑ comment by TheAncientGeek · 2015-04-12T23:21:45.219Z · LW(p) · GW(p)
I still don't see why you are considering a combination of non MIRI AI and MRI friendliness solution.
↑ comment by Raziel123 · 2015-04-12T21:30:02.646Z · LW(p) · GW(p)
If the AGi is a human mind upload, it is in no way a FAI, and I don't think it is what MIRI is aiming.
In case a neuromorphic AI is created, diferent arrays of neurons can give weidly diferent minds, We should not reason about a hipotetical AI using a human minds has a model and make predictions about it, even if that mind is based on biological minds.
What if the first neuron based AI has a mind more similar than a ant than a human, in that case anger, jealousy, freedom, etc are not longer part of the mind, or the mind could have totally new emotions, or things that are not emotions that we known not.
A mind that we don't understand enought, should not be said to be friendly and set free to the world, and I don't think that is being said here.
Replies from: TheAncientGeek↑ comment by TheAncientGeek · 2015-04-12T21:41:36.508Z · LW(p) · GW(p)
If the AGi is a human mind upload, it is in no way a FAI
How could a functional duplicate of a person known to ethical fail to be friendly?
Replies from: dxu, DanielLC, evand↑ comment by dxu · 2015-04-12T22:28:48.744Z · LW(p) · GW(p)
Power corrupts, and absolute power corrupts absolutely.
Replies from: TheAncientGeek↑ comment by TheAncientGeek · 2015-04-12T22:59:53.389Z · LW(p) · GW(p)
Any AI is a super AII?
↑ comment by DanielLC · 2015-04-12T22:22:21.907Z · LW(p) · GW(p)
If you leave their mind unaltered, you just have a human. They're not smart enough to really be useful. Once you start altering it, insanity becomes a likely result.
Best case scenario, you get one person's CEV. Most likely scenario, you get someone too insane to be useful. Worst case, you have an insane supergenius.
Replies from: TheAncientGeek↑ comment by TheAncientGeek · 2015-04-12T23:02:54.387Z · LW(p) · GW(p)
If humans weren't useful, humans wouldn't employ humans. A Hawking brain that needed no sleep would be a good start,
↑ comment by evand · 2015-04-12T22:21:04.474Z · LW(p) · GW(p)
Do you have a precise definition of "ethical" in mind? Where by "precise" I mean something roughly equivalent to a math paper.
Without such a definition, how will you know the person in question is ethical? With such a definition, how will you guarantee that the person in question meets it, will continue to meet it, etc.? How certain are you such a person exists?
Replies from: TheAncientGeek↑ comment by TheAncientGeek · 2015-04-12T23:13:31.717Z · LW(p) · GW(p)
Do you have a precise definition of "ethical" in mind?
No. Don't need one either.
Where by "precise" I mean something roughly equivalent to a math paper.Without such a definition, how will you know the person in question is ethical?
By ordinary standards.. Eg, Einstein was more ethical than von Neuman.
With such a definition, how will you guarantee that the person in question meets it, will continue to meet it, etc.?
Since when did functional duplicates start diverging unaccountably?
How certain are you such a person exists?
I'm not talking about mathematically proveable ethics OR about superintelligence. I'm talking about entrusting (superior) human level ems less than absolute power....ie what we do already with real humans,
Replies from: evand↑ comment by evand · 2015-04-13T01:05:18.594Z · LW(p) · GW(p)
I'm fairly willing to believe that intuitive understandings of "more ethical" will do well for imprecise things like "we'll probably get better results by instantiating a more ethical person as an em than a less ethical one". I'm less convinced the results will be good compared to obvious alternatives like not instantiating anyone as an em.
We see value drift as a result of education, introspection, religious conversion or deconversion, rationality exposure, environment, and societal power. Why would you expect not to see value drift in the face of a radical change in environment, available power, and thinking speed? I'm not concerned about whether or not the value drift is "accountable", I'm concerned that it might be large and not precisely predicted in advance.
Once you entrust the em with large but less than absolute power, how do you plan to keep its power less than absolute? Why do you expect this to be an easier problem than it would be for a non-em AI?
Replies from: TheAncientGeek↑ comment by TheAncientGeek · 2015-04-13T09:29:46.350Z · LW(p) · GW(p)
I'm less convinced the results will be good compared to obvious alternatives like not instantiating anyone as an em.
Not building an AI at all is not seen by MIRI as an obvious alternative. That seems an uneven playing field.
Why would you expect not to see value drift in the face of a radical change in environment, available power, and thinking speed?
I don't require the only acceptable level of value drift to be zero, since I am not proposing giving an em absolute power. I am talking about giving human level (or incrementally more) ems human style (ditto) jobs. That being the case, human style levels of drift will not make things worse,
Once you entrust the em with large but less than absolute power, how do you plan to keep its power less than absolute?
We have ways of reducing humans from office. Why would that be a novel, qualitatively different problem in the case of an em that is 10% or 5% or 1% smarter than a smart human?