Overview article on FAI in a popular science magazine (Hebrew)
post by JoshuaFox · 2012-05-15T11:09:51.759Z · LW · GW · Legacy · 32 commentsIn coming decades, engineers will build an entity with intelligence on a level which can compete with humans. This entity will want to improve its own intelligence, and will be able to do so. The process of improvement will repeat, until it reaches a level far above that of humans; the entity will then be able to achieve its goals efficiently. It is thus essential that its goals are good for humanity. To guarantee this, it is necessary to define the correct goals before this intelligence is built.
32 comments
Comments sorted by top scores.
comment by Dr_Manhattan · 2012-05-15T12:30:16.703Z · LW(p) · GW(p)
Artificial Intelligence: Can we reign in the golem
Love it from purely humorous angle. Now UFAI has a cultural flavor!
Replies from: JoshuaFox↑ comment by JoshuaFox · 2012-05-16T15:20:10.936Z · LW(p) · GW(p)
Good point, now that you mention it, the tag-line is not too bad. The golem is a legendary example of an intelligent, though not superintelligent, entity which poses danger as it carries out its instructions to the letter. Luke and Louie used a golem for their though experiment.
And though we cannot hope to control our future superintelligence, the tag-line is at least phrased as a question.
comment by MarkusRamikin · 2012-05-15T06:37:38.984Z · LW(p) · GW(p)
Good stuff.
Any idea how well the article was received?
Replies from: JoshuaFox↑ comment by JoshuaFox · 2012-05-15T11:12:40.518Z · LW(p) · GW(p)
It just came out, but I am certainly interested in seeing how it is received.
I think that pop science magazines have an important role in giving social validation to new scientific ideas.
Academic publishing is so big that it is hard to tell what ideas are good; ordinary popular media have little concern for accuracy in reporting on science, but good pop science magazines often do a pretty good job of gatekeeper in explaining the true state of real science.
comment by Kaj_Sotala · 2012-05-15T05:22:45.661Z · LW(p) · GW(p)
Good work!
comment by thomblake · 2012-05-14T18:33:17.676Z · LW(p) · GW(p)
Move to Discussion.
As an aside, I seriously think we need to start considering general AI stuff as off-topic again.
Replies from: steven0461, None, MarkusRamikin, David_Gerard↑ comment by steven0461 · 2012-05-14T20:16:33.649Z · LW(p) · GW(p)
It's not "general AI stuff", it's about Friendly AI, as suggested by the post's title.
Joshua, congrats on publishing this.
Replies from: thomblake↑ comment by thomblake · 2012-05-14T20:23:23.938Z · LW(p) · GW(p)
In this context I put Friendly AI in the category of "general AI stuff".
The important part here is that it's about FAI, not about the art of human rationality.
Replies from: steven0461↑ comment by steven0461 · 2012-05-14T20:24:47.641Z · LW(p) · GW(p)
As long as we're allowing some discussion on off-topic subjects that are not "the art of human rationality", can we please get rid of the useful off-topic subjects last?
Replies from: thomblake↑ comment by thomblake · 2012-05-14T20:27:33.595Z · LW(p) · GW(p)
I'd rather diminish the discussion of off-topic subjects, and get rid of the noisiest topics first.
AI and FAI are notable because people like to talk about them a lot for something off-topic.
Replies from: steven0461↑ comment by steven0461 · 2012-05-14T20:30:25.820Z · LW(p) · GW(p)
I'd be more likely to agree if there were somewhere else to productively discuss Singularity/FAI issues.
Replies from: thomblake↑ comment by thomblake · 2012-05-14T20:35:05.543Z · LW(p) · GW(p)
I agree that there should be somewhere else to discuss those things.
Replies from: Richard_Kennaway↑ comment by Richard_Kennaway · 2012-05-15T08:13:42.993Z · LW(p) · GW(p)
SL4?
Replies from: thomblake↑ comment by thomblake · 2012-05-15T13:55:20.890Z · LW(p) · GW(p)
Does that still exist?
Replies from: Richard_Kennaway↑ comment by Richard_Kennaway · 2012-05-15T14:22:15.417Z · LW(p) · GW(p)
The site's there, but I don't know how active the community still is.
Replies from: Kaj_Sotala, thomblake↑ comment by Kaj_Sotala · 2012-05-15T19:36:36.097Z · LW(p) · GW(p)
Dead.
Replies from: Richard_Kennaway↑ comment by Richard_Kennaway · 2012-05-17T09:22:36.589Z · LW(p) · GW(p)
If that community couldn't sustain itself, is there reason to think a subreddit here would prosper any better?
The problem with discussion of AGI, nanotechnology, and all the other "Shock Level N" memes for N ≥ 2 is that there is no real subject matter. For the most part it's just verbal geekery about cool ideas that no-one is actually doing anything about, because they're too far beyond current capabilities. Fine to engage in for a while at an SF con or in a pub with other geeks, but there's only so long you can be at a party before realising you're just seeing the same ideas over and over and it's time to leave.
I never read SL4 -- is that an accurate description of why it died?
Replies from: Kaj_Sotala↑ comment by Kaj_Sotala · 2012-05-17T12:16:42.052Z · LW(p) · GW(p)
I've entertained a similar hypothesis myself.
As for its relation to SL4, I'd say that it sounds roughly right - I wouldn't go as far as to say that there was "no real subject matter", but it's true that the list eventually ran out of worthwhile things to say that hadn't been already discussed.
↑ comment by thomblake · 2012-05-15T16:59:04.737Z · LW(p) · GW(p)
Aha - a relevant discussion was had on the list about a year ago, hereabouts.
We really ought to have a subreddit if people really want to talk about sl4/fai topics here. A different site on the same engine would be even better.
Replies from: Zetetic↑ comment by [deleted] · 2012-05-14T18:39:30.824Z · LW(p) · GW(p)
As an aside, I seriously think we need to start considering general AI stuff as off-topic again.
Perhaps the Singularity Institute and the Center for Applied Rationality should have separate community blogs?
Replies from: katydee↑ comment by katydee · 2012-05-14T19:06:27.700Z · LW(p) · GW(p)
This is theoretically a good idea, but I think at present there is so much crossover between the communities that it would be unwise to make such a move.
Replies from: Armok_GoB, Psy-Kosh↑ comment by Psy-Kosh · 2012-05-14T20:55:23.009Z · LW(p) · GW(p)
Besides, the subject matters of the two have significant overlap. Where would you put formal analysis/development of various decision theories, for example?
Replies from: Luke_A_Somers, Oscar_Cunningham↑ comment by Luke_A_Somers · 2012-05-14T21:44:52.076Z · LW(p) · GW(p)
Speaking of which, where is all that good stuff put as it stands?
Replies from: Normal_Anomaly↑ comment by Normal_Anomaly · 2012-05-15T17:00:59.134Z · LW(p) · GW(p)
In all seriousness, you may want to try Stuart Armstrong's user page.
↑ comment by Oscar_Cunningham · 2012-05-14T22:35:55.535Z · LW(p) · GW(p)
Put it on either and link it from the other one.
↑ comment by MarkusRamikin · 2012-05-15T06:35:44.077Z · LW(p) · GW(p)
I like AI stuff.
Replies from: thomblake, Dolores1984↑ comment by Dolores1984 · 2012-05-15T08:00:06.372Z · LW(p) · GW(p)
As do I.
↑ comment by David_Gerard · 2012-05-15T10:30:38.007Z · LW(p) · GW(p)
As an aside, I seriously think we need to start considering general AI stuff as off-topic again.
+1
It's interesting, but it's not something that fits the tagline: "refining the art of human rationality".