Suggest alternate names for the "Singularity Institute"

So I read this, and my brain started brainstorming. None of the names I came up with were particularly good. But I did happen to produce a short mnemonic for explaining the agenda and the research focus of the Singularity Institute.

A one word acronym that unfolds into a one sentence elevator pitch:

Crisis: Catastrophic Risks in Self Improving Software

• "So, what do you do?"
• "We do CRISIS research, that is, we work on figuring out and trying to manage the catastrophic risks that may be inherent to self improving software systems. Consider, for example..."

Lots of fun ways to play around with this term, to make it memorable in conversations.

It has some urgency to it, it's fairly concrete, it's memorable.
It compactly combines goals of catastrophic risk reduction and self improving systems research.

Bonus: You practically own this term already.

An incognito Google search gives me no hits for "Catastrophic Risks In Self Improving Software", when in quotes. Without quotes, top hits include the Singularity Institute, the Singularity Summit, intelligencexplosion.com. Nick Bostrom and the Oxford group is also in there. I don't think he would mind too much.

LessDoomed.

It's worth noting that your current name has advantages too; people who are interested in the accelerating change singularity will naturally run into you guys. These are people, some pretty smart, who are at home with weird ideas and like thinking about the far future. Isn't this how Louie found out about SI?

Maybe instead of changing your name, you could spin out yet another organization (with most of your current crew) to focus on AI risk, and leave the Singularity Institute as it is to sponsor the Singularity Summit and so on. My impression is that SI has a fairly high brand value, so I would think twice before discarding part of that. Additionally, I know at least one person assumed the Singularity Summit was all you guys did. So having the summit organized independently of the main AI risk thrust could be good.

Center for AI Safety most accurately describes what you do.

To be honest, the I. J. Good Institute sounds the most prestigious.

Beneficial Architectures Research makes you sound like you're researching earthquake safety or something similar. I don't think you necessarily need to shy away from the word "AI."

AI Impacts Research sounds incomplete, though I think it would sound good with the word "society," "foundation," or "institute" tacked onto either end.

I really like Center for AI Safety.

The AI Risk Reduction Center

Center for AI Risk Reduction

Institute for Machine Ethics

Center for Ethics in Artificial Intelligence

And I favor this kind of name change pretty strongly.

• Center for Helpful Artificial Optimizer Safety (CHAOS)
• Center for Slightly Less Probable Extinction
• Freindly Optimisation Of the Multiverse (FOOM)
• Yudkowsky's Army
• The Center for World Domination
• Pinky and The Brain Institute
• Cyberdyne Systems

The obvious change if Singularity has been co-opted is the Institute for Artificial Intelligence. (but IAI is not a great acronym).

Institute for Artifical Intelligence Safety lets you keep the S, but it's at the wrong spot. Safety Institution for Artificial Intelligence is off-puttingly incorrect.

The Institute for Friendly Artificial Intelligence (pron. eye-fay) is IFAI... maybe?

If you go with the Center for Friendly Artificial Intelligence you get CFAI, sort of parallel to CFAR (if that's what you want).

Oh! If associating with CFAR is okay, then what's really lovely is the Center for Friendly Artificial Intelligence Research, acronym as CFAIR. (You could even get to do cute elevator pitches asking people how they'd program their obviously well-defined "fairness" into an AI.)

Edit: I do agree that "Friendly" is not, on the whole, desirable. I prefer "Risk Reduction" to "Safety", because I think Safety might bring a little bit of the same unsophistication that Friendly would bring.

Paraphrasing, I believe it was said by an SIer that "if uFAI wasn't the most significant and manipulable existential risk, then the SI would be working on something else." If that's true, then shouldn't its name be more generic? Something to do with reducing existential risk...?

I think there are some significant points in favor of a generic name.

• Outsiders will more likely see your current focus (FAI) as the result of pruning causes rather than leaping toward your passion -- imagine if GiveWell were called GiveToMalariaCauses.

• By attaching yourself directly with reducing existential risk, you bring yourself status by connecting with existing high status causes such as climate change. Moreover, this creates debate with supporters of other causes connected to existential risk -- this gives you acknowledgement and visibility.

• The people you wish to convince won't be as easily mind-killed by research coming from "The Center for Reducing Existential Risk" or such.

Is it worth switching to a generic name? I'm not sure, but I believe it's worth discussing.

I have direct experience of someone highly intelligent, a prestigious academic type, dismissing SI out of hand because of its name. I would support changing the name.

Almost all the suggestions so far attempt to reflect the idea of safety or friendliness into the name. I think this might be a mistake, because for people who haven't thought about it much, this invokes images of Hollywood). Instead, I propose having the name imply that SI does some kind of advanced, technical research involving AI and is prestigious, perhaps affiliated with a university (think IAS).

Center for Advanced AI Research (CAAIR)

The Center for AI Safety

Like it. What you actually do.

The I.J. Good Institute

Eww. Pretentious and barely relevant. Some guy who wrote a paper in 1965. Whatever. Do it if for some reason you think prestigious sounding initials will give enough academic credibility to make up for having a lame irrelevant name. Money and prestige are more important than self respect.

Beneficial Architectures Research

Architectures? Word abuse! Why not go all the way and throw in "emergent"?

A.I. Impacts Research

Not too bad.

Sell the naming rights.

The AI researcher saw the word 'Singularity' and, apparently without reading our concise summary, sent back a critique of Ray Kurzweil's "accelerating change" technology curves. (Even though SI researchers tend to be Moore's Law agnostics, and our concise summary says nothing about accelerating change.)

Worse, when you try to tell someone who already mainly associates the idea of the singularity with accelerating change curves about the distinctions between different types of singularity, they can, somewhat justifiably from their perspective, dismiss it as just a bunch of internal doctrinal squabbling among those loony people who obsess over technology curves, squabbling that it's really beneath them to investigate too deeply.

The Center for AI Safety-- best of the bunch. It might be clearer as The Center for Safe AI.

The I.J. Good Institute-- I have no idea what the IJ stands for.

Beneficial Architectures Research-- sounds like an effort to encourage better buildings.

A.I. Impacts Research-- reads like a sentence. It might be better as Research on AI Impacts.

You are worried that the SIAI name signals a lack of credibility. You should be worried about its people do. No, it's not the usual complaints about Eliezer. I'm talking about Will Newsome, Stephen Omohundro, and Ben Goertzel.

Will Newsome has apparently gone off the deep end: http://lesswrong.com/lw/ct8/this_post_is_for_sacrificing_my_credibility/6qjg The typical practice in these cases, as I understand it, is to sweep these people under the rug and forget that they had anything to do with the organization. This might not be the most intellectually honest thing to do, but it's more PR-minded than leaving them listed, and more polite than adding them to a hall of shame.

And, while the Singularity Institute is announcing that it is absolutely dangerous to build an AGI without proof of friendlyness, two of its advisors, Omohundro and Goertzel, are, separately, attempting to build AGIs. Of course, this is only what I have learned from http://singularity.org/advisors/ -- maybe they have since changed their minds?

...but neither of these is a strong reason to keep the word 'singularity' in the name of our AI Risk Reduction organization.

Why not just call it that, then ? "AI Risk Reduction Institute".

"Safe" is a wrong word for describing a process of rewriting the universe.

(An old tweet of mine; not directly relevant here.)

I think something about "Machine ethics" sounds best to me. "Machine learning" is essentially statistics with a computational flavor, but it has a much sexier name. You think statistics and you think boring tables, you think "machine learning" and you think Matrix or Terminator.

Joke suggestions: "Mom's friendly robot institute," "Institute for the development of typesafe wishes" (ht Hofstadter).

Cerberus

I think a name change is a great idea. I can certainly imagine someone being reluctant to associate their name with the "Singularity" idea even if they support what SIAI actually does. I think if I was a famous researcher/donor, I would be a bit reluctant to be strongly associated with the Singularity meme in its current degraded form. Yes, there are some high-status people who know better, but there are many more who don't.

Here is a suggestion: Center for Emerging Technology Safety. This name affiliates with the high-status term "emerging technology", while terms with "Singularity" and even "AI" often (unfairly, in my opinion) strike people as being crackpot/kooky. Admittedly, this is less descriptive than some other possible names (but more descriptive than "The I.J. Good Institute"), but descriptiveness isn't the most important factor. Rather, you should consider what kind of organization potential donors or (high-status) employees would like to brag about to the their non-LW reading friends/family at dinner parties.

"Singularity Institue? Oh, Kurzweil!" It's as if he has a virtual trademark on the word. Yeah.

Semi-serious suggestions:

• Intelligence Explosion Risk Research Group
• Foundation for Obviating Catastrophes of Intelligence (FOCI)
• Foundation for Evaluating and Inhibiting Risks from Intelligence Explosion (FEIRIE)
• Center for Reducing Intelligence Explosion Risks (CRIER)
• Society for Eliminating Existential Risks (SEERs) of Intelligence Explosion
• Center for Understanding and Reducing Existential Risks (CURER)
• Averting Existential Risks from Intelligence Explosion (AERIE) Research Group (or Society or ...)

'A.I. Impact Institute', although that leads to the unfortunate acronym AIII...

Do we actually have rigorous evidence of a need for name change? It seems that we're seriously considering an expensive and risky move on the basis of mere anecdote.

It’s quite likely you can solve the problem of people miss-associating SI with “accelerating change“ without having to change names.

The AI researcher saw the word 'Singularity' and, apparently without reading our concise summary, sent back a critique of Ray Kurzweil's "accelerating change" technology curves.

What if the AI researcher read (or more likely, skimmed) the concise summary before responding to the potential supporter? At least this line in the first paragraph, “artificial intelligence beyond some threshold level would snowball, creating a cascade of self-improvements,” doesn’t necessarily make it obvious enough that SI isn’t about “accelerating change”. (In fact, it sounds a lot like an accelerating-change-type idea.)

In my opinion at least, you need to get any potential supporter/critic to make the association between the name “Singularity Institute” and what SI actually does(/it’s goals) as soon as possible. While changing the name could do that, “Singularity Institute” has many useful aesthetic qualities that a replacement name probably won’t have.

On the other hand doing something like adding a clear tag-line about what SI does (e.g. “Pioneering safe-AI research”) to the header, would be a relatively cheap and effective solution. Perhaps rewriting the concise summary to discuss the dangers of a smarter-than-human AI before postulating the possibility of an intelligence explosion would also be effective; seeing as a smarter-than-human AI would need to be friendly, intelligence explosion or no.

AI Impacts Research seems to me the best of the bunch, because it's pretty easy to understand. People who know nothing about Eliezer's work can see it and think, "Oh, duh AI will have an impact, it's worth thinking about that." On the other hand:

• Center for AI Safety: not bad, but people who don't know Eliezer's work might wonder why we need it (same thing with a name involving "risk")
• The I.J. Good Institute: Sounds pretigious, but gives no information to someone who doesn't know who I.J. Good is.
• Beneficial Architectures Research: meaningless to 99% of the population, agree with whoever said people will think it's about bridge design.

Somehow, "AI Impact Research" sounds better than "Impact," perhaps to avoid the "reads as a sentence" thing, or just because my brain thinks of "impact" as one (possibly very complex) thing. Also agree that "society" or "institute" could go in there somewhere.

I actually suspect that the word "Singularity" serves as a way of differentiating you from the huge number of academic institutes to do with AI so I'm not endorsing change necessarily.

However, if you do change, I vote for something to do with the phrase "AI Risk" - your marketing speel is about reducing risk and I think you're name will attract more donor attention if people can see a purpose rather than a generic name. As such, I vote against "I.J. Good Institute".

I also think "Beneficial Architectures Research" is too opaque a name and suspect (though with less certainty) that suggestions to do with "Friendly AI" are also too opaque (the name might seem cuddly but I don't think it will have deeper meaning to those who don't already know what you do).

I think something like "The Center for AI Safety" or "The AI Risk Institute" (TARI) would be your best bet (if you did decide a change was a good move).

Clearly, though, that's simply a list of one person's opinions on the matter.

A.I. Safety Foundation

Center for existential risk reduction

Friendly A.I. Group

A.I. Ethics Group

Institute for A.I. ethics

Why did the "AI" part get dropped from "SIAI" again?

Mandate

"The Mandate is a Gnostic School founded by Seswatha in 2156 to continue the war against the Consult and to protect the Three Seas from the return of the No-God.

... [it] also differs in the fanaticism of its members: apparently, all sorcerers of rank continuously dream Seswartha's experiences of the Apocalypse every night ...

...the power of the Gnosis makes the Mandate more than a match for schools as large as, say, the Scarlet Spires."

No-God/UFAI, Gnosis/x-rationality, the Consult/AGI community? ;-)

Does this mean it's too late to suggest "The Rationality Institute for Human Intelligence" for the recent spin-off, considering the original may no longer run parallel to that?

Seriously though, and more to the topic, I like "The Center for AI Safety", not only because it sounds good and is unusually clear as to the intention of the organization, but also because it would apparently, well, run parallel with "The Center for Modern Rationality" (!), which is (I think) the name that was ultimately (tentatively?) picked for the spin-off.

Center for AI Safety sounds excellent actually.

AI Ballistics Lab? You're trying to direct the explosion that's already underway.

Center for General Artificial Intelligence Readiness Research

The Last Organization.

Come to think of it, SI have a bigger problem than the name: getting a cooler logo than these guys.

/abg frevbhf

More than that, many people in SU-affiliated circles use the word "Singularity" by itself to mean Singularity University ("I was at Singularity"), or else next-gen technology; and not any of the three definitions of the Singularity. These are smart, innovative people, but some may not even be familiar with Kurzweil's discussion of the Singularity as such.

I'd suggest using the name change as part of a major publicity campaign, which means you need some special reason for the campaign, such as a large donation (see James Miller's excellent idea).

A suggestion: it may be a bad idea to use word 'artificial intelligence' in the name without qualifiers, as to serious people in the field

• the 'artificial intelligence' has much, much broader meaning than what SI is concerning itself with

• there is very significant disdain for the commonplace/'science fiction' use of 'artificial intelligence'

I like "AI Risk Reduction Institute". It's direct, informative, and gives an accurate intuition about the organization's activities. I think "AI Risk Reduction" is the most intuitive phrase I've heard so far with respect to the organization.

• "AI Safety" is too vague. If I heard it mentioned, I don't think I'd have a good intuition about what it meant. Also, it gives me a bad impression because I visualize things like parents ordering their children to fasten their seatbelts.
• "Beneficial Architectures" is too vague. It's not clear it's AI-related.
• "AI Impacts Research" is too vague and non-prescriptive. Unlike "AI Risk Reduction", it's ambiguous in its intentions.

I'll focus on "The Center for AI Safety", since that seems to be the most popular. I think "safety" comes across as a bit juvenile, but I don't know why I have that reaction. And if you say the actual words Artificial Intelligence, "The Center for Artificial Intelligence Safety" it gets to be a mouthful, in my opinion. I think a much better option is "The Center for Safety in Artificial Intelligence", making it CSAI, which is easily pronounced See-Sigh.

You could reuse the name of the coming December conference, and go for AI Impacts (no need to add "institute" or "research").

Retaining the meaning of 'intelligence explosion' without the word 'singularity':

• The Takeoff Institute

Center for AI Ethics Research

Center for Ethical AI

Singularity Institute for Ethical AI

The Good Future Research Center

A wink to the earlier I.J. Good Institute idea, it matches the tone of the current logo while being unconfining in scope.

Institute for Friendly Artificial Intelligence (IFAI).

It would be nice if the name reflected the SI's concern that the dangers come not just from some cunning killer robots escaping a secret government lab or a Skynet gone amok, or a Frankenstein monster constructed by a mad scientist, but from recursive self-improvement ("intelligence explosion") of an initially innocuous and not-very smart contraption.

I am also not sure whether the qualifier "artificial" conveys the right impression, as the dangers might come from an augmented human brain suddenly developing the capacity for recursive self-improvement, or from some other creation that does not look like a collection of silicon gates.

If I understand it correctly, SI wants to ensure "safe recursive self-improvement" of an intelligence of any kind, "safe" for the rest of the (human?) intelligences existing at that time, though not necessarily for the self-improver itself.

Of course, a name like "Society For Safe Recursive Self-Improvement" is both unwieldy and unclear to an outsider. (And the acronym sounds like parseltongue.) Maybe there is a way to phrase it better.

I agree that something along the lines of "AI Safety" or "AI RIsk Reduction" or "AI Impacts Research" would be good. It is what the organization seems to be primarily about.

As a side-effect, it might deter folks from asking why you're not building AIs, but it might make it harder to actually build an AI.

I'd worry about funding drying up from folks who want you to make AI faster, but I don't know the distribution of reasons for funding.

I'd prefer AI Safety Institute over Center for AI Safety, but I agree with the others that that general theme is the most appropriate given what you do.

Going by the google suggest principle, how about the AI Safety Syndicate (ASS)

The Centre for the Development of Benevolent Goal Architectures

Once, a smart potential supporter stumbled upon the Singularity Institute's (old) website and wanted to know if our mission was something to care about. So he sent our concise summary to an AI researcher and asked if we were serious. The AI researcher saw the word 'Singularity' and, apparently without reading our concise summary, sent back a critique of Ray Kurzweil's "accelerating change" technology curves. (Even though SI researchers tend to be Moore's Law agnostics, and our concise summary says nothing about accelerating change.)

For what it's worth, my instinct would be to send back a message (if I had the opportunity) saying, "Yes, I agree completely; I don't believe that Kurzweil's accelerating change argument has merit. In fact, I believe that most Singularity Institute researchers feel the same way. If you'd like to hear an argument in favor of FAI that does have merit, I'd suggest reading such-and-such."

Can a moderator please deal with private_messaging, who is clearly here to vent rather than provide constructive criticism?

You currently have 290 posts on LessWrong and Zero (0) total Karma.

I don't care about opinion of a bunch that is here on LW.

Others: please do not feed the trolls.

Heh. It's a pretty rare organisation that does Research in Artificial Intelligence Risk Reduction.

(Artificial Intelligence Risk Reduction by itself might work.)

Here's a few:

• Protecting And Promoting Humanity In The Future
• Society For An Ethical Post-Humanity
• Studies For Producing A Positive Impact Through Deep Recursion
• Rationality Institute For Self-Improving Information Technology

• Remedial Investigation [or Instruction] of Safety Kernel for AI [or: 'for AGI'; 'for Friendly AI'; 'for Friendly AGI'; 'for AGI Research'; etc.] (RISK for AI; RISK for Friendly AI)
• Friendly Architectures Research (FAR)
• Sapiens Friendly Research (SFR - pronounced 'Safer')
• Sapiens' Research Foundation (SRF)
• Sapiens' Extinction [or Existential] Risk Reduction Cooperative [or Conglomerate] (SERRC)
• Researchers for Sapiens Friendly AI (RSFAI)

While the concise summary clearly associates SI with Good's intelligence explosion, nowhere does it specifically say anything about Kurzweil or accelerating change. If people really are getting confused about what sort of singularity you're thinking about, would it be helpful as a temporary measure to put some kind of one-sentence disclaimer in the first couple paragraphs of the summary? I can understand that maybe this would only further the association between "singularity" and Kurzweil's technology curves, but if you don't want to lose the word entirely, it might help to at least make clear that the issue is in dispute.

Also, on a separate subject, I notice that the summary presently has a number of "??" marks, presumably as a kind of formatting error. Just a heads-up. :)

Ok.

The Center for AI safety and Centre for Friendly Artificial Intelligence research sound the most correct as of now.

If you wanted to aim for a more creative name, then here are some

Centre for Coding Goodness

Man's Best Friend Group (If the slightly implied sexism of "Man's" is Ok..)

The Artificial Angels Institute / Centre for Machine Angels - The angels word directly conveys goodness and superiority over humans, but due to its christian origins and other associated imagery, it might be walking a tight rope.

Wasn't this discussed before?

Center for Applied Rationality Education?

the Best Ever Association For Rational AI Development. ;)

Kurzweil Institute

What you are doing here is called psychopathic behaviour outside utilitarian circles (and even utilitarians can see utility in crackpottery being easy to identify). Please stop helping to strip a crackpot research organization of all the tell tale signs that help to inexpensively identify it as being crackpot. World is a better place when the researchers themselves choose names and the names are indicative of the likehood of crankery.

It is not OK to replace the goal of AI safety (with the subgoal of 'we need more money if our work helps improve AI safety and we need to dissolve otherwise') with the sub-goal of collecting more money (conditional having been lost). That leads to behaviour identical to your unfriendly paperclip maximizer which once made paperclips for the purpose of helping African schoolchildren or something.

edit: Furthermore, you guys do nothing about safety of e.g. self driving cars, nor plan to do anything of this kind. You're no "Center for AI Safety" (or anything of this kind), that much is a fact. You're a "Centre for preventing a risk few believe is real exclusively via making a solution (FAI) that even fewer believe will work". (and not even a centre but more like a fringe). Changing your name to a more general one that differs in the sense that it would seem to imply you are contributing to e.g. safety of self driving cars (or any other non singularity AI scenario), is nothing short of lying.