[link] Baidu cheats in an AI contest in order to gain a 0.24% advantage

Take a look at this image.

Is 'pure AI startups' the relevant category here?

Stuart Russell said recently "The commercial investment in AI the last five years has exceeded the entire world wide government investment in AI research since it's beginnings in the 1950's."

That would be astonishing if true, but I have to be doubtful. Does Russell provide sources for this? The last 5 years is not much time to outspend the cumulative total of MIT AI Lab, the Fifth Generation Project, a fair number of DARPA's projects, the AI Winter, and all AI groups for half a century.

Just FYI to readers: the source of the first image is here.

Doing compression is not the goal of computer vision.

It really is isomorphic to the generally proclaimed definition of computer vision as the inverse problem of computer graphics. Graphics starts with an abstract scene description and applies a transformation to obtain an image; vision attempts to back-infer the scene description from the raw image pixels. This process can be interpreted as a form of image compression, because the scene description is a far more parsimonious description of the image than the raw pixels. Read section 3.4.1 of my book for more details (the equivalent interpretation of vision-as-Bayesian-inference may also be of interest to some).

Well, lossless compression implies understanding. Lossy compression may or may not imply understanding.

Also, usually you can get a lossy compression algorithm from a lossless one. In image compression, the lossless method would typically be to send a scene description plus a low-entropy correction image; you can easily save bits by just skipping the correction image.

I emphasize lossless compression because it enables strong comparisons between competing methods.

If you really believe that slowing progress on AGI is a good thing, you should do it by encouraging young people to go into different fields, not by encouraging people to waste their careers.

Yes.

I think general RL AI is now advanced to the point where testing some specific AI safety/control subproblems is becoming realistic. The key is to decompose and reduce down to something testable at small scale.

One promising route is to build on RL game playing agents, and extend the work to social games such as MMOs. In the MMO world we already have a model of social vs antisocial behavior - ie playerkillers vs achievers vs cooperators.

So take some sort of MMO that has simple enough visuals and or world complexity while retaining key features such as gold/xp progression, the ability to kill players with consequences, etc (a pure text game may be easier to learn, or maybe not). We can then use that to train agents and test various theories.

For example, I would expect that training an RL agent based on maximizing a score type function directly would result in playerkilling emerging as a natural strategy. If the agents were powerful enough to communicate, perhaps even simple game theory and cooperation would emerge.

I expect that training with Wissner-Gross style maximization of future freedom of action (if you could get it to scale) would also result in playerkillers/sociopaths.

In fact I expect that many of MIRI's intuitions about the difficulty of getting a 'friendly' agent out of simple training specification are probably mostly correct. For example, you could train an agent with a carefully crafted score function that penalizes 'playerkilling' while maintaining regular gold/xp reward. I expect that those approaches will generally fail if the world is complex enough - the agent will simply learn a griefing exploit that doesn't technically break the specific injunctions you put in place (playerkilling or whatever). I expect this based on specific experiences with early MMOs such as Ultima Online where the designers faced a simliar problem of trying to outlaw or regulate griefing/playerkilling through code - and they found it was incredibly difficult.

I also expect that IRL based approaches could eventually succeed in training an agent that basically learns to emulate the value function and behaviours of a 'good/moral/friendly' human player - given sufficient example history.

I think this type of research platform would put AI safety research on equal footing with mainstream ML research, and allow experimental progress to supplement/prove many concepts that up until now exist only as vague ideas.