What can we learn from Microsoft's Tay, its inflammatory tweets, and its shutdown?

If you paint a Chinese flag on a wolverine, and poke it with a stick, it will bite you.

This does not mean that the primary danger of aggravating the Chinese army is that they will bite you.

It certainly does not mean that nations who fear Chinese aggression should prepare by banning sticks or investing in muzzles for wolverines.

There is an opinion expressed here, that I agree with: http://smerity.com/articles/2016/tayandyou.html TL;dr: No "learning" from interactions on twitter happened. The bot was parroting old training data, because it does not really generate text. The researchers didn't apply an offensiveness filter at all.

I think this chat bot was performing badly right from the start. It would not make sense to give too much importance to the users it was chatting with, and they did not change its mind. That bit of media sensationalism is BS. Natural language generation is an open problem and almost every method I have seen (not an expert in NLP, but would call myself one in Machine Learning) ends up parroting some of its training text, implying that it is overfitting.

Given this, we should learn nothing about AI from this experiment, only about people's reaction to it, mainly the media reaction to it. Users' reaction while talking to AI is well documented.

I though a bit about it, but I think Tay is basically a software version of a parrot that repeats back what it hears - I don't think it has any commonsense knowledge or serious attempt to understand that tweets are about a world that exists outside of twitter. I.e it has no semantics, it's just a syntax manipulator that uses some kind of probabilistic language model to generate grammatically correct sentences and a machine learning model to try and learn which kind of sentences will get the most retweets or will most closely resemble other things people are tweeting about. Tay does't know what a "Nazi" actually is. I haven't looked into it in any detail but I know enough to guess that that's how it works.

As such, the failure of Tay doesn't particularly tell us much about Friendliness, because friendliness research pertains to superintelligent AIs which would definitely have a correct ontology/semantics and understand the world.

However, it does tell us that a sufficiently stupid, amateurish attempt to harvest human values using an infrahuman intelligence wouldn't reliably work. This is obvious to anyone who has been "in the trade" for a while, however it does seem to surprise the mainstream media.

It's probably useful as a rude slap-in-the-face to people who are so ignorant of how software and machine learning work that they think friendliness is a non-issue.

The first obvious point is that when learning human values you need a large dataset which isn't biased by going viral on 4chan.

The more interesting question is what happens when we get more powerful AI which isn't just a chatbot. Suppose in the future a powerful Baysian inference engine is developed. Its not an AGI, so there is no imminent singularity, but it does have the advantages of very large datasets and being completely unbiased. Asking it questions produces provably reliable results in many fields (but it is not smart enough to answer "how do I create AGI?"). Now, there are a lot of controversial beliefs in the world, so I would say it is probable that it answers at least one question in a controversial way, whether this is "there is no God" or "there are racial differences in intelligence" or even "I have ranked all religions, politics and philosophies in order of plausibility. Yours come near the bottom. I would say I'm sorry, but I am not capable of emotions.".

How do people react? Since its not subject to emotional biases, it's likely to be correct on highly controversial subjects. Do people actually change their minds and believe it? After the debacle, Microsoft hardcoded Tay to be a feminist. What happens if you apply this approach to the Baysian inference engine? Well, if there is logic like so:

The scientific method is reliable -> very_controversial_thing

And hardcoded:

P(very_controversial_thing)=0

Then the conclusion is that the scientific method isn't reliable.

I the point I am trying to make is that if an AI axiomatically believes something which is actually false, then this is likely to result in weird behaviour.

As a final thought, for what value of P(Hitler did nothing wrong) does the public start to freak out? Any non-zero ammount? But 0 and 1 are not probabilities!

A chatbot like Tay has no deep insight into the things it says. It's just pattern matching existing human messages from its dataset. The religious AI researchers would understand that just like I'm sure Microsoft's researchers understand why Tay said what it did.

Not much.

Did they delete posts?

That Artificial Intelligence is going to do a lot of the same things that Natural Intelligence does.

Sure, but he point stands: failures of nattow AI systems aren't informative about likely faulures of superintelligent AGIs.

Yes, you are correct. And if image recognition software started doing some kind of unethical recognition (I can't be bothered to find it, but something happened where image recognition software started recognising gorillas as African ethnicity humans or vice versa), then I would still say that it doesn't really give us much new information about unfriendliness in superintelligent AGIs.

I'm sure the engineers knew exactly what would happen. It doesn't tell us much about the control problem that we didn't already know.

OTOH, if this wasn't an intentional PR stunt, that means management didn't think this would happen even though the engineers presumably knew. That definitely has unsettling implications.

BTW, the twitter account is here if you want to see the things the AI said for yourself.

Original thread here.

It might help to take an outside view here:

Picture a hypothetical set of highly religious AI researchers who make an AI chatbot, only to find that the bot has learned to say blasphemous things. What lessons should they learn from the experience?

Original thread here.

Two things come to mind.

1. Programming a "friendly" AI may be impossible but it is to soon to tell.

2. A recursively self-modifying system lacking any guiding principles is not a good place to start.

They would perhaps conclude that an AI has no soul?

Oh, yes, good old potential UFAI #261: let the AI learn proper human values from the internet.

The point here being, it seems obvious to me that the vast majority of possible intelligent agents are unfriendly, and that it doesn't really matter what we might learn from specific error cases. In order words, we need to deliberately look into what makes an AI friendly, not what makes it unfriendly.

Can microsoft and google's Ai learn political correctness by coding a automatic feedback mechanism to heavily penalise the situations that took them offline to begin with?