"AlphaStar: Mastering the Real-Time Strategy Game StarCraft II", DeepMind [won 10 of 11 games against human pros]

Well, apparently that's exactly what happened with TLO and MaNa, and then the DeepMind guys were (at least going by their own accounts) excited about the progress they were making and wanted to share it, since being able to beat human pros at all felt like a major achievement. Like they could just have tested it in private and continued working on it in secret, but why not give a cool event to the community while also letting them know what the current state of the art is.

E.g. some comments from here:

I am an administrator in the SC2 AI discord and that we've been running SC2 bot vs bot leagues for many years now. Last season we had over 50 different bots/teams with prizes exceeding thousands of dollars in value, so we've seen what's possible in the AI space.

I think the comments made in this sub-reddit especially with regards to the micro part left a bit of a sour taste in my mouth, since there seems to be the ubiquitous notion that "a computer can always out-micro an opponent". That simply isn't true. We have multiple examples for that in our own bot ladder, with bots achieving 70k APM or higher, and them still losing to superior decision making. We have a bot that performs god-like reaper micro, and you can still win against it. And those bots are made by researchers, excellent developers and people acquainted in that field. It's very difficult to code proper micro, since it doesn't only pertain to shooting and retreating on cooldown, but also to know when to engage, disengage, when to group your units, what to focus on, which angle to come from, which retreat options you have, etc. Those decisions are not APM based. In fact, those are challenges that haven't been solved in 10 years since the Broodwar API came out - and last Thursday marks the first time that an AI got close to achieving that! For that alone the results are an incredible achievement.

And all that aside - even with inhuman APM - the results are astonishing. I agree that the presentation could have been a bit less "sensationalist", since it created the feeling of "we cracked SC2" and many people got defensive about that (understandably, because it's far from cracked). However, you should know that the whole show was put together in less than a week and they almost decided on not doing it at all. I for one am very happy that they went through with it.

And the top comment from that thread:

Thank you for saying this. A decent sized community of hobbyists and researchers have been working on this for YEARS, and the conversation has really never been about whether or not bots can beat humans "fairly". In the little documentary segment, they show a scene where TLO says (summarized) "This is my off race, but i'm still a top player. If they're able to beat me, i'll be really surprised."

That isn't him being pompous, that's completely reasonable. AI has never even come CLOSE to this level for playing starcraft. The performance of AlphaStar in game 3 against MaNa left both Artosis AND MaNa basically speechless. It's incredible that they've come this far in such a short amount of time. We've literally gone from "Can an AI play SC2 at a high level AT ALL" to "Can an AI win 'fairly'". That's a non-trivial change in discourse that's being completely brushed over IMO.

Is there a reason to assume that DeepMind is being intentionally deceptive rather than making a good-intentioned mistake? After all, they claim that they consulted with TLO on the APM rate, which is something that they seem unlikely to lie about, since it would be easy for TLO to dispute that if it was untrue. So presumably TLO felt like the caps that they instituted were fair, and it would have been reasonable for DeepMind to trust a top player's judgment on that, none of them (AFAIK) being top players themselves. And even with the existing caps, people felt like the version which played TLO would likely have lost to MaNa, and the camera-limited version did actually lose to him. So it seems reasonable for someone to think that a good enough player would still beat versions of AlphaStar even with the existing limits, and that they didn't thus give it an unfair advantage.

On the other hand, while I can think of plenty of reasons for why DeepMind might have honestly thought that this setup was fair, I can't think of any good reason for them to decide to be intentionally deceptive in this way. They've been open about their agent currently only playing Protoss vs. Protoss on a single map and of earlier versions seeing the whole map, and five of their games were played against a non-Protoss player. If they honestly felt that they only won because of the higher APM rate, then I don't see why they wouldn't just admit that the same way they're forthcoming about all of the system's other limitations.

Ok, I read it too after my comment above... And I thought that than future evil superintellgence will start shooting at people on streets, the same commenters will said: "No, it is not a superintelligence, it is good just in tactical use of guns, and it just knows where humans are located and never misses, but its strategy is awful". Or, in other words:

Weak strategy + perfect skills = dangerous AI