Connectome-Specific Harmonic Waves

post by lsusr · 2019-12-05T00:23:53.864Z · score: 20 (11 votes) · LW · GW · 14 comments


  Artificial Neural Networks

Selen Atasoy's new theory of brain function called Connection-Specific Harmonic Waves (CSHW) could be a giant step forward in our understanding of consciousness. To put this in perspective I will first review what we know so far about how minds work.

Artificial Neural Networks

Much of the most recent progress in machine learning (ML) is in the realm of image recognition. That's because Artificial Neural Networks (ANNs) are unusually good at image processing. Much of the recent progress in machine learning (ML) has been applying artificial neural networks to new problem domains.

This has been made possible by a single invention, Graphical Processing Units (GPUs). A GPU is a computer chip that generates the images in graphically-intensive videogames by performing matrix algebra. Training an ANN is matrix algebra too so the fastest way to train an ANN is with a GPU. This has allowed ANN hardware to outpace than Moore's Law, if just for a moment.

The simplest kind of neural network is a feed forward neural network (FFNN). An FFNN has layers of nodes with connections between them. You train a neural network with the backpropagation algorithm. The important things to take away from the backpropagation algorithm is it's recursive. Therefore neural networks possess a fractal architecture. This is important and we'll get back to it later.

Besides FFNNs, the other kind of neural network is called a recurrent neural network (RNN). This is a neural network with cyclic feedback loops, which creates hysteresis (short-term memory). RNNs have seen success in natural language processing but are not used in the flashy new advancements such as self-driving cars. That's because RNNs are more complicated than FFNNs. We understand RNNs less well than FFNNs. RNNs are simple enough to use on very short time scales but we don't know how to scale them up to long time scales.

Due to our reliance on FFNNs over RNNs, we don't know how to get an ANN to handle time-series data. I know this because I run a startup that uses ML to process time series data. Self-driving cars use FFNNs. FFNNs' inability to process time series data was a contributing factor to the Uber self-crashing car [LW · GW].

We'll get back to RNNs later too.


Your brain is a neural network. There are two CSHW-related differences between your brain's neurons and ANN neurons.

We know how individual neurons in the human brain work. We know what different regions of the brain do because we observe how human behavior changes when different brain regions are damaged. But we don't know how neurons work together to create these brain regions. CSHW suggests an answer to this question.

At the same time, global workspace theory (GWT) offers an observable definition of consciousness[1]. Basically it's the idea that consciousness is the thing your whole mind is thinking about. Under the hood, one of your brain's parts networks broadcasts itself to the rest of your brain and this becomes the thing you're thinking about. GWT is well-supported by both psychological experiments and contemplative tradition but we don't know how the brain does it. Our ANNs have no global workspace.

To summarize.

  1. We know what neurons are.
  2. We can simulate artificial neurons to accomplish simple tasks at a speed competitive with human beings. This is the best method of writing software to do things like image recognition and playing go.
  3. We know what each part of the human brain does.
  4. We don't know to build a machine to accomplish these complicated tasks using artificial neurons.

In short, our ANNs are fast, scalable and parallelizable. Our ANNs can solve problems where conceptual complexity is very small. However, our ANNs have trouble handling conceptual complexity and time-series data. We don't know how to make the inductive step of putting our simple neural networks together into a larger intelligence. They can pattern-match but they can't strategize. And they lack consciousness. This might not be a coincidence.

If brains possess a fractal architecture then we're missing the hierarchical inductive step.


In 2016, Selen Atasoy published a paper in Nature Communications titled "Human brain networks function in connectome-specific harmonic waves". Here's the most important sentence.

[E]igendecomposition of the Laplace operator...can predict the collective dynamics of human cortical activity at the macroscopic scale.

―Atasoy, S., Donnelly, I. & Pearson, J. Human brain networks function in connectome-specific harmonic waves. Nat Commun 7, 10340 (2016) doi:10.1038/ncomms10340

"[E]igendecomposition of the Laplace operator" means finding the harmonics of the connectome. Unstated in this sentence is the possibility that eigendecomposition of the Laplace operator can predict collective dynamics on arbitrary scales. In case that doesn't make sense, here's a crash course on acoustics.

Every sound wave can be broken down into the superposition of resonant frequencies or harmonics[2]. This forms a basis for sound waves oscillating through the geometry. This isn't limited to sound waves. Any kind of wave can be broken down this way. Different harmonics have different frequencies. Higher-frequency harmonics oscillate faster and propagate shorter distances. Lower frequency harmonics oscillate slower and propagate farther distances.

This kind of resonance happens whenever waves bounds through a solid structure, such as sound waves through a violin or x-rays through a crystal. Selen Atasoy and her lab have confirmed resonance of brainwaves bouncing through the connectome. The neurons in a single functional region of the brain (a region we've observed to do something important) resonate together. She calls this a "state network".

So what?

When you press middle C on a piano it's not just middle C that vibrates. The other C strings will vibrate too, especially those closest to middle C. That's because from left to right the strings for each octave are half as long as the previous. Integer multiples like this produce resonance.

Every note on the piano has a particular resonance with each other keys. Some pairs of notes are highly resonant with each other. Other pairs have low resonance[3]. It depends on the ratio of one frequency to another.

CSHW is a simple, elegant way to coordinate many different sub-networks into a human brain. When different networks are out of phase with each other the inputs of one turn into static for the other, which is mathematically equivalent to tuning out a radio.

This could explain what meditation does.


Conventionally scientific information on meditation is hard-to-come-by because:

  1. Psychology as a science is less than a hundred years old. We've barely recovered from the behaviorist overreaction to Freud.
  2. Governments suppress research on psychedelics. We lost scientific research into meditation amidst the collateral damage.
  3. Government funding for psychological research revolves around curing diseases. Meditation is for healthy people to get better.
  4. MRIs are expensive.
  5. Long-term meditation takes serious dedication every day for decades. Figuring out what does and doesn't work takes centuries.
  6. The only intellectual traditions to record this knowledge in useful form exist outside the Western intellectual tradition.

Eastern monks and yogis have been experimenting with meditation, comparing their results and iterating this technology in an unbroken dharma for several millennia. Our understanding of meditation is like naturalism in the time of Darwin except this time religion has all the data.


Two and a half thousand years ago lived an Indian prince named Siddhartha. He mastered the already ancient Hindu yogic techniques. He meditated for several years. Then one day, while meditating under a tree, he saw the truth of reality. We don't know exactly what this means. But we do know he got there via meditation, it freed Siddhartha of dukkha and it was permanent. This state is called enlightenment.

Then Siddhartha established a monastic order to pass down his discoveries and improve upon them. This organization evolved into the world religion of Buddhism. Some sects have moved on from meditation. Others have been improving upon Siddhartha's techniques up to the present day. But all of them share the objective of recreating the enlightenment state of mind Siddhartha achieved so long ago.

You're probably wondering how we can verify this. The Dalai Lama likes science so he helped convince some ascetic yogis to come down from the mountains and fly across the world and submit to brain scans while meditating and while at rest[4]. Some important discoveries stand out.

  1. The yogis had reduced galavanic skin response in anticipation of physical pain.
  2. The yogis' default mode network did not activate during wakeful rest.
  3. The yogis were in a constant state of gamma wave activity[5].

Discovery (1) is suggestive of reduced dukkha. Discovery (2) is relevant to cybernetics, which we'll get to later. Discovery (3) might be an objective metric we could use to measure enlightenment states.

I've replicated these results myself by achieving meditative states where my default mode network stops making noise. This happens around the 30 minute mark, after access meditation and muscle spasms. Time spend in these states reduces my dukkha. I wouldn't be surprised if I've also increased my gamma wave activity for minutes at a time. I can't afford an fMRI to verify any of this objectively, but my experience is typical[6] for meditators on a path to enlightenment.

If gamma waves are related to enlightenment then we can finally ground enlightenment in the material universe.

Under CSHW, gamma waves are when everything in your brain is literally in sync with everything else. If CSHW and GWT are both true and enlightenment equals continuous gamma waves then together they would explain the meaning of the weird subjective descriptions of enlightenment people give like "my mind is bigger"[7]. Your consciousness really is bigger instead of being fractured like a split brain patient.


For a mind to interact intelligently with its environment the mind has to include a simplified model of its environment. Even a thermos does this when it decides whether to keep your drink hot or cold.

Your consciousness lives on the connectome and never interacts with reality directly. Instead, your consciousness interacts with the simplified model of reality created by your mind. Your Self and your Other are both mental constructs.

If CSHW is true and enlightenment equals gamma waves then in an enlightenment state the Self and the Other would be plugged into one another. Under normal circumstances your Self and bits of your mind's representation of the external world may be out of resonance. This is easy to understand if you've been closed off parts of yourself in reaction to abuse.

In this way, CSHW may go a long way towards explaining anattā and its cessation. See, everyone who attains enlightenment does so through one of the three marks of existence. The three marks are aniccā (impermanence), dukkha (unsatisfactoriness or suffering), and anattā (non-self). To understand one of them is to understand all of them. In other words, they're three different ways of getting at the same Truth.

The Truth is that everything you experience is a construction of your mind. But "seeing the Truth" doesn't mean understanding this intellectually. "Seeing the Truth" means getting your various state networks into resonance. You can do this by sitting still (or walking calmly or doing simple chores) and clearing your mind. If your mute your sensory inputs and default mode network long enough then eventually all your state networks will sync up like a roomfull of pendulum clocks.

These traditions present evidence CSHW is an important part of how the brain solves its challenge of coordinating neural network subsystems.

A popular secular meditation manual Mastering the Core Teachings of the Buddha: An Unusually Hardcore Dharma Book by Daniel M. Ingram "Dharma Dan" approaches enlightenment via vipassana meditation. Vipassana is the technique of paying close attention to what's happening in your mind. Dharma Dan emphasizes paying attention to individual high frequency brainwaves. According to Therevada theory, if you use vipassana to look at it your conscious experience at a high enough time resolution your conscious experience breaks down into discrete frames or oscillations. This process also generates insight with leads to enlightenment.

From the perspective of CSHW what's going on in high frequency vipassana is you're directing your global workspace to a single high frequency oscillation instead of jumping around from one signal source to another. Since every network is always working towards anticipating its own inputs this naturally leads to increased resonance as each state network syncs its internal clock with the target of vipassana attention.

The fMRI data corroborating this (the Tibetan yogis from earlier) is based on lovingkindness meditation, not vipassana, but the principle still applies. All contemplative traditions focus consciousness on a single object[8] for a long time. No matter what the object is, eventually this should lead to increased resonance, which explains how contemplative techniques as different as kasina fire meditation can produce such similar outcomes.

CSHW establishes a mathematical foundation for why anger and hatred are universally uprooted across the various contemplative traditions. Anger and hatred do not exist in isolation. They are felt toward your mind's conceptualization of something that isn't you. The distinction between yourself and the other is premised off of idea that you are separate from it. But your mental model of the Other is literally part of your brain. If your brain is in total resonance then you can't feel separation from the Other. Enlightened individuals don't feel anger or hatred because that would be an a priori contradiction. Similarly, anger and hatred are obstacles to enlightenment.


Our ANNs can scale to arbitrarily large input/output dimensionality because they're fractal structures in two directions: input/output dimensionality and number of hidden layers. You can cut an ANN in half along either of these directions and get two smaller neural networks.

This is a special case of a general principle. An information processing system can scale to arbitrarily complex problems if and only if the system is structured fractally. Systems without a fractal structure will eventually encounter a computational cliff.

We've hit this cliff with our ANNs. Our ANNs scale well in the aformentioned directions for which they possess fractal geometry. But their hierarchical structure is non-fractal, so they don't work on hierarchical conceptual problems. Our brains are better than our ANNs when it comes to strategic reasoning.

CSHW suggests a framework for how to coordinate ANNs hierarchically. Low frequency waves propagate farther than high frequency waves. So whenever you go up an order of magnitude in physical scale the wavelength increases of the relevant brainwaves increase. CSHW works the same on every scale of observation...all the way down to individual neurons. Remember how biological neurons send a pulse of action potentials instead of modulating voltage? That could be the base case to our fractal induction.

The individual components should be easy to build out of RNNs. They could be scaled with CSHW inductively to larger physical dimensions and time dimensions. This could automatically solve the other problem of how to build RNNs that work for large time scales thereby putting us a giant step forward towards building an artificial general intelligence.

  1. I'm using "consciousness" to refer to the global workspace in GWT. I mean to imply nothing metaphysical with the term. ↩︎

  2. In real-world applications this results in reconstruction errors, especially for square waves. This is addressed in Atasoy's paper. ↩︎

  3. Piano strings differing by exactly the golden ratio have minimal resonance. ↩︎

  4. You can find this research and more in the book Altered Traits by Daniel Goleman, a fascinating book on what science knows on the long-term effects of serious meditation. It was published in 2017 so it contains up-to-date information. However, many of the studies are unreplicated. Considering the historical obstacles to this research, we're lucky to have anything at all. ↩︎

  5. It frustrates me that scientists haven't conducted the same experiments on Zen masters who live in cities and have a formal system for certifying who has become enlightened. I want to know if fMRI scans correlate with dharma transmission. ↩︎

  6. For a first-person account of what it's like to follow this path to its conclusion I recommend Hardcore Zen: Punk Rock, Monster Movies, & the Truth About Reality by Brad Warner and The Science of Enlightenment: How Meditation Works by Shinzen Young. ↩︎

  7. This description comes from a young woman who stumbled into stream entry (a secular name for enlightenment) outside of any meditative tradition. If the woman comes from a Christian tradition (as this woman did) the experience can be confusing. It is believed a small number of random people stumble into enlightenment unpredictably. ↩︎

  8. Except nondual traditions like Zen. They abandon the meditation target and shoot straight towards enlightenment. ↩︎


Comments sorted by top scores.

comment by Said Achmiz (SaidAchmiz) · 2019-12-05T01:12:45.754Z · score: 6 (4 votes) · LW(p) · GW(p)

I have not yet read through this post, but I want to note that we had a post on Less Wrong about this theory over a year ago [LW · GW]. What do you think about that post, and the comments on it? What does it get right, and what do you consider to be mistakes in it?

EDIT: Since you mention meditation, consider my questions also to apply to this more recent post on CSHW [LW · GW].

comment by lsusr · 2019-12-05T02:19:56.905Z · score: 6 (4 votes) · LW(p) · GW(p)

It was an even more recent post, "Neural Annealing" [LW · GW] that inspired me to write this one. The most popular reaction "Neural Annealing" was "I don't get it. How can this be important given the relatively low bandwidth of brainwaves compared to synapses?" I wrote a comment [LW(p) · GW(p)] to answer that question, and then this "Connectome-Specific Harmonic Waves" article to address the implications in full.

comment by lsusr · 2019-12-05T02:13:44.105Z · score: 3 (2 votes) · LW(p) · GW(p)

I think "Connectome-specific harmonic waves and meditation" [LW · GW] has the right idea. There's not much to say since it's just a list of links to other sources. The most interesting thing about that post is Vaniver's comment [LW(p) · GW(p)]. I think my article makes a good point that Veniver missed. Namely, that increasing resonance within a brain increases information flow between different parts. This gives a computational advantage above pure aesthetic symmetry. The goal isn't simplicity. It's coordination.

Vaniver makes a good point in his final paragraph. This isn't addressed in "Connectome-specific harmonic waves and meditation" but is addressed in my post.

[A] 'maximize harmony' story needs to have really strong boundary conditions to create the same sorts of conflicts.

This is accounted for by CSHW because (1) oscillations within state networks are observably contained in their state networks and (2) high frequency oscillations propagate a shorter distance than low frequency oscillations.

comment by lsusr · 2019-12-05T02:04:28.160Z · score: 3 (3 votes) · LW(p) · GW(p)

Mike Johnson's "A future for neuroscience" [LW · GW] helped me write this post. I think the biggest problem with it is that Mike Johnson fails to recognize the fractal nature of these harmonics.

The fractal nature of CSHW makes possible a self-organizing system. CSHW's power comes from how its fractal nature makes possible a bottom-up understanding of the human brain. Since Mike Johnson misses this, he instead generalizes downward from the macroscopic structure of the brain. Mike Johnson concludes incorrectly that different harmonics would be the same between different people and then generalizes from his mistake.

By operating top-down, Mike Johnson runs into the same problems neuroscience has struggled with since its inception. Cutting a fractal black box into pieces will never tell you how the box works. All it will do is replace the black box with smaller, identical black boxes. It's better to investigate a fractal via induction than dissection.

Under my interpretation, CSHW is a self-organizing fractal, just like a FFNN. Therefore there's no reason to assume that one person's harmonic signature is likely to resemble someone else's, especially at high frequency oscillations. (I can explain this in more detail if the reasons are not clear from my original post.)

comment by Michael Edward Johnson (michael-edward-johnson) · 2020-04-18T06:56:05.719Z · score: 3 (3 votes) · LW(p) · GW(p)

Thanks for so clearly putting your thoughts down. Honestly, I liked your comment on my LW crosspost of Neural Annealing so that I added it to the end of the post on my blog.

Briefly, I wanted to note a key section of NA where I talk about "a continuum of CSHWs with scale-free functional roles", which depending on definitions may or may not be the same thing as CSHWs being fractal:


Last year in A Future for Neuroscience, I shared the frame that we could split CSHWs into high-frequency and low-frequency types, and perhaps say something about how they might serve different purposes in the Bayesian brain:

The mathematics of signal propagation and the nature of emotions
High frequency harmonics will tend to stop at the boundaries of brain regions, and thus will be used more for fine-grained and very local information processing; low frequency harmonics will tend to travel longer distances, much as low frequency sounds travel better through walls. This paints a possible, and I think useful, picture of what emotions fundamentally are: semi-discrete conditional bundles of low(ish) frequency brain harmonics that essentially act as Bayesian priors for our limbic system. Change the harmonics, change the priors and thus the behavior. Panksepp’s seven core drives (play, panic/grief, fear, rage, seeking, lust, care) might be a decent first-pass approximation for the attractors in this system. 

I would now add this roughly implies a continuum of CSHWs, with scale-free functional roles:

  • Region-specific harmonic waves (RSHWs) high frequency resonances that implement the processing of cognitive particulars, and are localized to a specific brain region (much like how high-frequencies don’t travel through walls) in theory quantifiable through simply applying Atasoy’s CSHW method to individual brain regions;
  • Connectome-specific harmonic waves (CSHWs) low-frequency connectome-wide resonances that act as Bayesian priors, carrying relatively simple ‘emotional-type’ information across the brain (I note Friston makes a similar connection in Waves of Prediction);
  • Sensorium-specific harmonic waves (SSHWs) very-low-frequency waves that span not just the connectome, but the larger nervous system and parts of the body. These encode somatic information – in theory, we could infer sensorium eigenmodes by applying Atasoy’s method to not only the connectome, but the nervous system, adjusting for variable nerve-lengths, and validate against something like body-emotion maps.[2][3]

These waves shade into each other – a ‘low-frequency thought’ shades into a ‘high-frequency emotion’, a ‘low-frequency emotion’ shades into somatic information. As we go further up in frequencies, these waves become more localized.


comment by lsusr · 2020-05-31T03:29:31.820Z · score: 3 (2 votes) · LW(p) · GW(p)

Thank you for putting my comment on your blog. Its very flattering.

I would now add this roughly implies a continuum of CSHWs, with scale-free functional roles:

One of the most important most important implications of CSHWs' is what you call their "scale-free functional roles" and what I call their fractal "scale invariance". Terms like RSHW, CSHW and SSHW are just markers for arbitrary scales, like "gamma rays" and "infra-red" on the electromagnetic spectrum. I just finished an article [LW · GW] attaching equations to this idea.

comment by Michael Edward Johnson (michael-edward-johnson) · 2020-04-18T07:21:26.440Z · score: 4 (3 votes) · LW(p) · GW(p)

I really like the meditation/enlightenment frame for poking at these concepts. You might enjoy my Neuroscience of Meditation piece; here's an excerpt:


Finally, we may be able to usefully describe the Buddhist jhanas through a combination of CSHW and neural annealing. Essentially, Buddha noted that as one follows the meditative path and eliminates unwholesome mental states, they will experience various trance-like states of deep mental unification he called ‘jhanas’. These are seen as progressive steps to full enlightenment- the first few jhanas focus on joy, and after these are attained one can move to jhanas which revolve around contentment and feelings of infinity, and finally full cessation of suffering. Importantly, these experiences are not seen as ‘mere signposts’ on the path, but active processes which are causally involved in the purification of the mind — in the original Pāli, the root of ‘jhana’ can refer to both ‘meditate’ and ‘burn up’, e.g. to destroy the mental defilements holding one back from serenity and insight.
A ‘modern’ approach here might be to identify the various jhanas as natural resonant modes of the brain– i.e., different jhanas would map to different harmonic configurations, each with a different phenomenological texture, but all high in consonance/harmony. If this is true, we should be able to do neat things like identify which jhana a meditator is experiencing from their CSHW data, or reconstruct Buddhism’s list of jhanas from first principles based on the math of which brain harmonics can be combined in a way that produces high consonance/harmony. Perhaps we could even find a novel, unexplored jhana or two, pleasant configurations of brain harmonics that even most experienced meditators have never experienced.
But if we add neural annealing to this model, we can start to understand how experiencing the various jhanas may actively sculpt the mind. At its most basic, meditation offers a context for people to sit with their thoughts and maybe do some work on themselves, and get some practice ‘getting out of their own way’. Basically removing the ‘defilements’ which clutter up their brain harmonics, much like removing a clamp from a bell or shaking a mouse out of a trombone. Once these ‘resonance impediments’ are removed, and energy is added to the system (through effortful meditation), brains will naturally start to self-organize toward the simpler resonant configurations, the first jhanas. But importantly, highly-resonant states are also high-energy states- i.e., the very definition of resonance is that energy travels in a periodic pattern that reinforces itself, instead of dissipating in destructive interference. So if you get a brain into a highly-resonant state (a jhana) and keep it there, this will also start a neural annealing process, basically purifying itself (and making it easier and easier to enter into that particular resonant state- “harmonic recanalization”) more or less automatically.
With this in mind, we might separate Buddha’s path to enlightenment into two stages: first, one attempts to remove the psychological conditions which prevent them from attaining a minimum level of ‘whole-brain resonance’; mostly, this will involve trying to meditate, experiencing a problem in doing so, fixing the problem, trying to meditate again. Rinse, repeat- possibly for years. But in the second stage, once enough of these conditions are gone and resonant momentum can accumulate, they can start ‘climbing the jhanas,’ mostly just entering meditative flow and letting the math of Laplacian eigenmodes and neural annealing slowly shape their mind into something that resonates in purer and purer ways[8], until at the end it becomes something which can only support harmony, something which simply has no resources that can be used to sustain dissonance.
[8] What precisely is happening as one climbs the various jhanas? Resonance in chaotic systems is inherently unstable, and so if the first jhana is “a minimum level of whole-brain resonance” we should expect many perturbations and failures in maintaining this pleasant state as unpredicted sense-data, chaotic internal feedback loops, and evolved defenses against ‘psychological wireheading‘ knock the system around. Each additional jhana, then, may be thought of as a widening of the set of factors being stabilized, or using a higher-dimensional or further-optimized implicit model of wave dynamics to compensate for more sources of turbulence. This optimization process might separate into discrete steps or strategies (jhanas), each with their own particular phenomenology, depending on what kind of turbulence it’s best at stabilizing. I expect we’ll find that earlier jhanas are characterized by seeking particular narrow resonant configurations that work; later jhanas flip the script and are characterized by seeking out the remaining distortions in the ‘web of phenomenology’, the problem states that don’t resonate, in order to investigate and release them. 

(More in the post itself)

Of course, the challenge here is: if this is a good theory of how the brain works, of how meditation works, of what enlightenment, is -- can we use it to build something cool? Something that actually helps people, that you couldn't have built without these insights?

comment by lsusr · 2020-05-31T05:45:45.193Z · score: 3 (2 votes) · LW(p) · GW(p)

I very much did enjoy this! Thank you for the link.

There are lots of good ideas in your article. I think I actually…uh…did read your article, many months ages, and then forgot that I did so and that your articles…er…um…contributed significantly to inspiring this series.

You might like my new article on resonance [LW · GW]. It builds off your three paragraphs about guitar strings. I swear I wrote it before re-reading your statement "Resonance in chaotic systems is inherently unstable". If I did read that particular line several months ago I definitely forgot about it.

[C]an we use it to build something cool?

I'm trying to build an AGI. These insights have been very helpful so far.

comment by anon123 · 2020-01-06T03:50:28.949Z · score: 4 (3 votes) · LW(p) · GW(p)

FYI Nature Communications =/= Nature. Impact factor is about 1/4th. That shouldn't be all important but since "Nature" is highlighted in the post, this should be clarified.

comment by lsusr · 2020-01-05T23:41:44.608Z · score: 1 (1 votes) · LW(p) · GW(p)

I didn't know that. Thank you. I have corrected the original article.

comment by Pattern · 2019-12-06T00:14:17.499Z · score: 4 (3 votes) · LW(p) · GW(p)

Upvoted for having a theory make predictions, and lots of clarity.

comment by lsusr · 2019-12-15T04:25:42.989Z · score: 2 (2 votes) · LW(p) · GW(p)

Thanks. I was surprised when this initially got downvoted twice in a row.

comment by NaiveTortoise (An1lam) · 2019-12-05T02:29:21.909Z · score: 3 (3 votes) · LW(p) · GW(p)

FFNNs’ inability to process time series data was a contributing factor to the Uber self-crashing car.

I don't see the evidence for this in the linked post and don't recall seeing this in any of the other few articles/pieces I've read on the crash. Can you point me to the evidence for this?

comment by lsusr · 2019-12-05T02:40:11.976Z · score: 6 (5 votes) · LW(p) · GW(p)

There were two bits of evidence I used to infer this.

"If I'm not sure what it is, how can I remember what it was doing?" The car wasn't sure whether Herzberg and her bike were a "Vehicle", "Bicycle", "Unknown", or "Other", and kept switching between classifications. This shouldn't have been a major issue, except that with each switch it discarded past observations. Had the car maintained this history it would have seen that some sort of large object was progressing across the street on a collision course, and had plenty of time to stop.

The first bit (above) is that the car throws away its past observations. The second bit of evidence is a consequence of the first.

"If we see a problem, wait and hope it goes away". The car was programmed to, when it determined things were very wrong, wait one second. Literally. Not even gently apply the brakes. This is absolutely nuts. If your system has so many false alarms that you need to include this kind of hack to keep it from acting erratically, you are not ready to test on public roads.

Humans have to write ugly hacks like this when when your system isn't architected bottom-up to handle things like the flow of time. A machine learning system designed to handle time series data should never have human beings in the loop this low down the ladder of abstraction.