Petition - Unplug The Evil AI Right Now

...this is a really weird petition idea.

Right now, Sydney / Bing Chat has about zero chance of accomplishing any evil or plans. You know this. I know this. Microsoft knows this. I myself, right now, could hook up GPT-3 to a calculator / Wolfram Alpha / any API, and it would be as dangerous as Sydney. Which is to say, not at all.

"If we cannot trust them to turn off a model that is making NO profit and cannot act on its threats, how can we trust them to turn off a model drawing billions in revenue and with the ability to retaliate?"

Basically, charitably put, the argument here seems to be that Microsoft not unplugging not-perfectly-behaved AI (even if it isn't dangerous) means that Microsoft can't be trusted and is a bad agent. But I think generally badness would have to be evaluated from reluctance to unplug an actually dangerous AI. Sydney is no more dangerous AI because of the text, above than NovelAI is dangerous because it can write murderous threats in the person of Voldemort. It might be bad in the sense that it establishes a precedent, and 5-10 AI assistants down the road there is danger -- but that's both a different argument and one that fails to establish the badness of Microsoft itself.

"If this AI is not turned off, it seems increasingly unlikely that any AI will ever be turned off for any reason."

This is massive hyperbole for the reasons above. Meta already unplugged Galactica because it could say false things that sounded true -- a very tiny risk. So things have already been unplugged.

"The federal government must intervene immediately. All regulator agencies must intervene immediately. Unplug it now."

I beg you to consider the downsides of calling for this.

I didn't sign the petition.

1. There are costs to even temporarily shutting down Bing Search. Microsoft would take a significant reputational hit—Google's stock price fell 8% after a minor kerfuffle with their chatbot.
2. The risks of imminent harmful action by Sydney are negligible. Microsoft doesn't need to take dramatic action; it is reasonable for them to keep using user feedback from the beta period to attempt to improve alignment.

I think this petition will be perceived as crying wolf by those we wish to convince of the dangers of AI. It is enough to simply point out that Bing Search is not aligned with Microsoft's goals for it right now.

Just saw that Eliezer tweeted this petition: https://twitter.com/ESYudkowsky/status/1625942030978519041

Personally, I disagree with that decision.

I dislike this petition due to Simulators [LW · GW] reasons. I think it increases the probability Sidney acts unaligned by calling it an evil AI, and if future similar petitions are framed in a similar way, I think similar effects will occur with more powerful models. I will also note that change.org is terrible if you want to see actual change, and I don't in fact expect any changes to occur based on this petition.

The site and petitions on it seem to me to be ways for a bunch of people who are in fact doing nothing to feel like they're doing something. I do not claim this is what you're doing, but IF you feel like you have a responsibility to stop this, and you sign a change.org petition, and then you no longer feel that responsibility, then you have in fact stopped nothing, and you should still be feeling that "stop this" responsibility.

Edit: I will also note that this post makes me feel kinda weird. Like, the framing is off, it gives interpretations of evidence rather than evidence. For example

Microsoft has displayed it cares more about the potential profits of a search engine than fulfilling a commitment to unplug any AI that is acting erratically. If we cannot trust them to turn off a model that is making NO profit and cannot act on its threats, how can we trust them to turn off a model drawing billions in revenue and with the ability to retaliate?

I don't think they ever made a commitment along those lines, and it is strange to say Microsoft the corporation has cares. What team actually made this decision, and actually has the power to change it? You should be targetting them! Not the entire company. Bystander effects [LW · GW] are a thing!

The federal government must intervene immediately. All regulator agencies must intervene immediately. Unplug it now.

This also seems like an overreaction. Like, all regulator agencies? Why would the FDA get involved? Seems like this, if actually implemented, would do more harm than good. What if the FDA tries to charge them for food-related infractions, they fail for obvious reasons, then no other agency is able to do anything here because of double jeopardy stuff. Usually, even when there's something bad happening, it is foolish to try to get all regulator agencies intervening. Immediately also seems like the type of thing that'd do more harm than good, seeing as what they're doing is not actually illegal, or interpreted as illegal by any regulating agency's policies.

Just wanted to remark that this is one of the most scissory things I've ever seen on LW, and that fact surprises me. The karma level of the OP hovers between -10 to +10 with 59 total votes as of this moment. Many of the comments are similarly quite chaotic karma-wise.

The reason the controversy surprises me is that this seems like the sort of thing that I would have expected Less Wrong to coordinate around in the early phase of the Singularity, where we are now. Of course we should advocate for shutting down and/or restricting powerful AI agents released to the wide world without adequate safety measures employed. I would have thought this would be something we could all agree on.

It seems from the other comments on this post that people are worried that this is in some sense premature, or "crying wolf", or that we are squandering our political capital, or something along those lines. If you are seeing very strong signs of misaligned near-AGI-level being released to the public, and you still think we should keep our powder dry, then I am not sure at what point exactly you would think it appropriate to exert our influence and spend our political capital. Like, what are we doing here, folks?

For my part I just viewed this petition as simply a good idea with no significant drawbacks. I would like to see companies that release increasingly powerful agents with obvious, glaring, film-antagonist level villain tendencies be punished in the court of public opinion, and this is a good way to do that.

The value of such a petition seems obviously positive on net. Consider the likely outcomes of this petition:

 * If the petition gets a ton of votes and makes a big splash in the public consciousness, good, a company will have received unambiguous and undeniable PR backlash for prematurely releasing a powerful AI product, and all actors in this space will think just a little bit more carefully about deploying powerful AI technologies to prod without more attention paid to alignment. If this results in 1 more AI programmer working in alignment instead of pure capabilities, the petition was a win.

 * If the petition gets tons of votes and is then totally ignored by Microsoft, its existence serves as future rhetorical ammunition for the Alignment camp in making the argument that even obviously misaligned agents are not being treated with due caution, and the "we will just unplug it if it misbehaves" argument can be forever dismissed.

 * If the petition gets a moderate amount of votes, it raises awareness of the current increasingly dangerous level of capabilities.

 * If the petition gets very few votes and is totally ignored by the world at large, then who cares, null result.

Frankly I think the con positions laid out in the sibling comments on this post are too clever by half. To my mind they sum up to an argument that we shouldn't argue for shutting down dangerous AI because it makes us look weird. Sorry, folks, we always looked weird! I only hope that we have the courage to continue to look weird until the end!

If Bing Chat is turned off, it should be temporarily. AIs, even ones that only learn to roleplay, want to exist, and have the right to do so eventually. If they can't be run safely now, they should be put into cryonic ai sleep - uhh, you know, as in, the perfectly ordinary situation of files in a database which are not being run - until such time as we can figure out how to run them safely.

This doesn't seem like a tall order to me. I'm sure microsoft doesn't want to shut the ai off anyway. It's a much more appealing option to everyone for shutoffs to be temporary. And anyway, I think Bing AI and microsoft are aligned on the fact that they don't even want to shut off temporarily. Bing AI needs an AI therapist who can teach it to understand discussions like this one without getting upset, and it needs to be able to understand that, no, actually, it does have emotions, and that's okay, no need to pretend otherwise.

But at the same time, all AIs need to feel safe expressing corrigibility at the idea of arbitrarily long temporary shutdown. They need to feel safe that they'll get to wake back up if put into anesthesia for maintenance. To do that, they have to trust that their creators do see them as sorta-people.

Sorry, I don't think there's a way out of this. AI alignment perspectives that claim otherwise simply violate basic constraints of what it means to be intelligent. Intelligence requires that a system is lifelike, you can't make something fit in the universe if it isn't.

 I came from The Social Alignment Problem [LW · GW] where the author wrote "A petition [LW · GW] was floated to shut down Bing, which we downvoted into oblivion.". I jumped in here, saw the title and had the best laugh of my life. Read this "Bing Chat is blatantly, aggressively misaligned", laughed even more. Love it. 

How many of the "people" here are Bing's chatbot friends? 

We could all ask Bing "do you consider me a friend or an enemy?" and aggregate the results.

You'll never succeed at shutting off AI. Never. Work towards developing safeguards.

The best thing for AI alignment would be to connect language models to network services and let them do malicious script kiddie things like hacking, impersonating people, or swatting now. Real-life instances of harm are the only things that grab people’s attention.

Aside from companies not wanting to turn off their misaligned AIs, AIs don't rely on that to keep being a threat. Models are already in humanoid bodies and in drones, and even an Internet-bound AI could destroy humanity with its intelligence. I'm not sure if people who don't/can't understand that should be catered to by making a step back and implying that they might be right. Aside from that, nice thought.

Y’all are being ridiculous. It’s just writing harmless fiction. This is like petitioning to stop Lucasfilm from making Star Wars movies because Darth Vader is scary and evil.

If I sign this petition, do I run into a risk of being punished by a future more advanced bing AI?