Even when contrarians win, they lose: Jeff Hawkins

I feel like the lesson here is "if your plan depends on a conjunction of propositions, then you might fail even if you are right about some of those propositions and everyone else is wrong."

I followed Jeff Hawkins' work closely for a long time. In my view, several things happened, although due to the somewhat secretive nature of the startup, I may be grossly wrong and invite corrections to my viewpoint.

Firstly, their algorithm idea (HTM) was from the start based on a set of limiting simplifying assumptions that made it hard to generalize their work to problem domains outside of computer vision. Approximately the same time work on HTMs was starting, people in the deep learning community were starting to work seriously on complicated vision problems, making a large part of HTM moot/obsolete by the time a public release was made (by the way, deep neural nets and HTM share a lot in common, which is interesting since they were arrived at from very different directions).

Later, Hawkins and Dileep George (the 'technical' lead) had something of a rift where Hawkins emphasized temporal learning and George wanted to focus more on getting vision right. This led to George leaving Numenta and Numenta essentially becoming one of many companies offering 'data mining' and 'big data analytics' services. George, meanwhile, started his own company (Vicarious), focused on human-like computer vision software. Vicarious has not yet released a product, but have admitted their approach uses probabilistic graphical models, which would put it in line with most of the 'mainstream' work on the subject.

tl;dr: Numenta's work was significant but the machine learning field as a whole is moving so rapidly that yesterday's breakthroughs are today's mundane trivialities.

At least Ng's career though can be credited to Hawkins.

'At least a part'? Also,

???

Worth mentioning that some parts of Superintelligence are already a less contrarian version of many arguments made here in the past.

Also note that although some people do believe that FHI is some sense "contrarian", when you look at the actual hard data on this the fact is FHI has been able to publish in mainstream journals (within philosophy at least) and reach important mainstream researchers (within AI at least) at rates comparable, if not higher, to excellent "non-contrarian" institutes.

I also read On Intelligence and it had a large impact on my reading habits. I was not previously aware that Andrew Ng had a similar experience, which leads me to wonder how many people became interested in neuroscience as a result of that one book.

On a side note: the only significance of Andrew Ng's stated belief that AGI is far is as an indicator that he doesn't see a route to get there in the near term. On a related note, he gave a kind of wierd comment recently at the end of a conference talk to the effect of "Worrying about the dangers of machine superintelligence today is like worrying about overpopulation on Mars."

In one sense, the "one learning algorithm" hypothesis should not seem very surprising. In the fields of AI/machine learning, essentially all practical learning algorithms can be viewed as some approximation of general Bayesian inference (yes - this includes stochastic gradient descent). Given a utility function and a powerful inference system, defining a strong intelligent agent is straightforward (general reinforcement learning, AIXI, etc.)

The difficulty of course is in scaling up practical inference algorithms to compete with the brain. One of the older views in neuroscience was that the brain employed a huge number of specialized algorithms that have been fine tuned in deep time by evolution - specialized vision modules, audio modules, motor, language, etc etc. The novelty of the one learning hypothesis is the realization that all of that specialization is not hardwired, but instead is the lifetime accumulated result of a much simpler general learning algorithm.

On Intelligence is a well written pop sci book about a very important new development in neuroscience. However, Hawkin's particular implementation of the general ideas - his HTM stuff - is neither groundbreaking, theoretically promising, nor very effective. There are dozens of unsupervised generative model frameworks that are more powerful in theory and in practice (as one example, look into any of Bengio's recent work), and HTM itself has had little impact on machine learning.

I wonder also about Hassibis (founder of DeepMind) - who studied computational neuroscience and then started a deep learning company - did he read On Intelligence? Regardless, you can see the flow of influence in how deep learning papers cite neuroscience.

I downvoted this post because it is basically meta discussions above arguments from authority and tribalism: Andrew Ng and MIRI == good, turns out Jeff Hawkins influenced Ng and shares some conceptual ideas with MIRI, therefore Hawkins == good. That's faulty reasoning which has the capability to reinforce wrong beliefs.

Tell me, what about Hawkins/Numentia's work makes it wrong or right on its own merits? Why or why not is it likely to lead to capable general purpose intelligences?