0 comments
Comments sorted by top scores.
comment by Said Achmiz (SaidAchmiz) · 2019-12-05T01:12:45.754Z · LW(p) · GW(p)
I have not yet read through this post, but I want to note that we had a post on Less Wrong about this theory over a year ago [LW · GW]. What do you think about that post, and the comments on it? What does it get right, and what do you consider to be mistakes in it?
EDIT: Since you mention meditation, consider my questions also to apply to this more recent post on CSHW [LW · GW].
Replies from: lsusr, lsusr, lsusr↑ comment by lsusr · 2019-12-05T02:19:56.905Z · LW(p) · GW(p)
It was an even more recent post, "Neural Annealing" [LW · GW] that inspired me to write this one. The most popular reaction to "Neural Annealing" was "I don't get it. How can this be important given the relatively low bandwidth of brainwaves compared to synapses?" I wrote a comment [LW(p) · GW(p)] to answer that question, and then this "Connectome-Specific Harmonic Waves" article to address the implications in full.
↑ comment by lsusr · 2019-12-05T02:13:44.105Z · LW(p) · GW(p)
I think "Connectome-specific harmonic waves and meditation" [LW · GW] has the right idea. There's not much to say since it's just a list of links to other sources. The most interesting thing about that post is Vaniver's comment [LW(p) · GW(p)]. I think my article makes a good point that Veniver missed. Namely, that increasing resonance within a brain increases information flow between different parts. This gives a computational advantage above pure aesthetic symmetry. The goal isn't simplicity. It's coordination.
Vaniver makes a good point in his final paragraph. This isn't addressed in "Connectome-specific harmonic waves and meditation" but is addressed in my post.
[A] 'maximize harmony' story needs to have really strong boundary conditions to create the same sorts of conflicts.
This is accounted for by CSHW because (1) oscillations within state networks are observably contained in their state networks and (2) high frequency oscillations propagate a shorter distance than low frequency oscillations.
↑ comment by lsusr · 2019-12-05T02:04:28.160Z · LW(p) · GW(p)
Mike Johnson's "A future for neuroscience" [LW · GW] helped me write this post. I think the biggest problem with it is that Mike Johnson fails to recognize the fractal nature of these harmonics.
The fractal nature of CSHW makes possible a self-organizing system. CSHW's power comes from how its fractal nature makes possible a bottom-up understanding of the human brain. Since Mike Johnson misses this, he instead generalizes downward from the macroscopic structure of the brain. Mike Johnson concludes incorrectly that different harmonics would be the same between different people and then generalizes from his mistake.
By operating top-down, Mike Johnson runs into the same problems neuroscience has struggled with since its inception. Cutting a fractal black box into pieces will never tell you how the box works. All it will do is replace the black box with smaller, identical black boxes. It's better to investigate a fractal via induction than dissection.
Under my interpretation, CSHW is a self-organizing fractal, just like a FFNN. Therefore there's no reason to assume that one person's harmonic signature is likely to resemble someone else's, especially at high frequency oscillations. (I can explain this in more detail if the reasons are not clear from my original post.)
Replies from: michael-edward-johnson↑ comment by Michael Edward Johnson (michael-edward-johnson) · 2020-04-18T06:56:05.719Z · LW(p) · GW(p)
Thanks for so clearly putting your thoughts down. Honestly, I liked your comment on my LW crosspost of Neural Annealing so that I added it to the end of the post on my blog.
Briefly, I wanted to note a key section of NA where I talk about "a continuum of CSHWs with scale-free functional roles", which depending on definitions may or may not be the same thing as CSHWs being fractal:
-------------------
Last year in A Future for Neuroscience, I shared the frame that we could split CSHWs into high-frequency and low-frequency types, and perhaps say something about how they might serve different purposes in the Bayesian brain:
The mathematics of signal propagation and the nature of emotions
High frequency harmonics will tend to stop at the boundaries of brain regions, and thus will be used more for fine-grained and very local information processing; low frequency harmonics will tend to travel longer distances, much as low frequency sounds travel better through walls. This paints a possible, and I think useful, picture of what emotions fundamentally are: semi-discrete conditional bundles of low(ish) frequency brain harmonics that essentially act as Bayesian priors for our limbic system. Change the harmonics, change the priors and thus the behavior. Panksepp’s seven core drives (play, panic/grief, fear, rage, seeking, lust, care) might be a decent first-pass approximation for the attractors in this system.
I would now add this roughly implies a continuum of CSHWs, with scale-free functional roles:
- Region-specific harmonic waves (RSHWs) – high frequency resonances that implement the processing of cognitive particulars, and are localized to a specific brain region (much like how high-frequencies don’t travel through walls) – in theory quantifiable through simply applying Atasoy’s CSHW method to individual brain regions;
- Connectome-specific harmonic waves (CSHWs) – low-frequency connectome-wide resonances that act as Bayesian priors, carrying relatively simple ‘emotional-type’ information across the brain (I note Friston makes a similar connection in Waves of Prediction);
- Sensorium-specific harmonic waves (SSHWs) – very-low-frequency waves that span not just the connectome, but the larger nervous system and parts of the body. These encode somatic information – in theory, we could infer sensorium eigenmodes by applying Atasoy’s method to not only the connectome, but the nervous system, adjusting for variable nerve-lengths, and validate against something like body-emotion maps.[2][3]
These waves shade into each other – a ‘low-frequency thought’ shades into a ‘high-frequency emotion’, a ‘low-frequency emotion’ shades into somatic information. As we go further up in frequencies, these waves become more localized.
-------------------
Replies from: lsusr↑ comment by lsusr · 2020-05-31T03:29:31.820Z · LW(p) · GW(p)
Thank you for putting my comment on your blog. Its very flattering.
I would now add this roughly implies a continuum of CSHWs, with scale-free functional roles:
One of the most important most important implications of CSHWs' is what you call their "scale-free functional roles" and what I call their fractal "scale invariance". Terms like RSHW, CSHW and SSHW are just markers for arbitrary scales, like "gamma rays" and "infra-red" on the electromagnetic spectrum. I just finished an article [LW · GW] attaching equations to this idea.
comment by Michael Edward Johnson (michael-edward-johnson) · 2020-04-18T07:21:26.440Z · LW(p) · GW(p)
I really like the meditation/enlightenment frame for poking at these concepts. You might enjoy my Neuroscience of Meditation piece; here's an excerpt:
---------------------------
Finally, we may be able to usefully describe the Buddhist jhanas through a combination of CSHW and neural annealing. Essentially, Buddha noted that as one follows the meditative path and eliminates unwholesome mental states, they will experience various trance-like states of deep mental unification he called ‘jhanas’. These are seen as progressive steps to full enlightenment- the first few jhanas focus on joy, and after these are attained one can move to jhanas which revolve around contentment and feelings of infinity, and finally full cessation of suffering. Importantly, these experiences are not seen as ‘mere signposts’ on the path, but active processes which are causally involved in the purification of the mind — in the original Pāli, the root of ‘jhana’ can refer to both ‘meditate’ and ‘burn up’, e.g. to destroy the mental defilements holding one back from serenity and insight.
A ‘modern’ approach here might be to identify the various jhanas as natural resonant modes of the brain– i.e., different jhanas would map to different harmonic configurations, each with a different phenomenological texture, but all high in consonance/harmony. If this is true, we should be able to do neat things like identify which jhana a meditator is experiencing from their CSHW data, or reconstruct Buddhism’s list of jhanas from first principles based on the math of which brain harmonics can be combined in a way that produces high consonance/harmony. Perhaps we could even find a novel, unexplored jhana or two, pleasant configurations of brain harmonics that even most experienced meditators have never experienced.
But if we add neural annealing to this model, we can start to understand how experiencing the various jhanas may actively sculpt the mind. At its most basic, meditation offers a context for people to sit with their thoughts and maybe do some work on themselves, and get some practice ‘getting out of their own way’. Basically removing the ‘defilements’ which clutter up their brain harmonics, much like removing a clamp from a bell or shaking a mouse out of a trombone. Once these ‘resonance impediments’ are removed, and energy is added to the system (through effortful meditation), brains will naturally start to self-organize toward the simpler resonant configurations, the first jhanas. But importantly, highly-resonant states are also high-energy states- i.e., the very definition of resonance is that energy travels in a periodic pattern that reinforces itself, instead of dissipating in destructive interference. So if you get a brain into a highly-resonant state (a jhana) and keep it there, this will also start a neural annealing process, basically purifying itself (and making it easier and easier to enter into that particular resonant state- “harmonic recanalization”) more or less automatically.
With this in mind, we might separate Buddha’s path to enlightenment into two stages: first, one attempts to remove the psychological conditions which prevent them from attaining a minimum level of ‘whole-brain resonance’; mostly, this will involve trying to meditate, experiencing a problem in doing so, fixing the problem, trying to meditate again. Rinse, repeat- possibly for years. But in the second stage, once enough of these conditions are gone and resonant momentum can accumulate, they can start ‘climbing the jhanas,’ mostly just entering meditative flow and letting the math of Laplacian eigenmodes and neural annealing slowly shape their mind into something that resonates in purer and purer ways[8], until at the end it becomes something which can only support harmony, something which simply has no resources that can be used to sustain dissonance.
[8] What precisely is happening as one climbs the various jhanas? Resonance in chaotic systems is inherently unstable, and so if the first jhana is “a minimum level of whole-brain resonance” we should expect many perturbations and failures in maintaining this pleasant state as unpredicted sense-data, chaotic internal feedback loops, and evolved defenses against ‘psychological wireheading‘ knock the system around. Each additional jhana, then, may be thought of as a widening of the set of factors being stabilized, or using a higher-dimensional or further-optimized implicit model of wave dynamics to compensate for more sources of turbulence. This optimization process might separate into discrete steps or strategies (jhanas), each with their own particular phenomenology, depending on what kind of turbulence it’s best at stabilizing. I expect we’ll find that earlier jhanas are characterized by seeking particular narrow resonant configurations that work; later jhanas flip the script and are characterized by seeking out the remaining distortions in the ‘web of phenomenology’, the problem states that don’t resonate, in order to investigate and release them.
(More in the post itself)
Of course, the challenge here is: if this is a good theory of how the brain works, of how meditation works, of what enlightenment, is -- can we use it to build something cool? Something that actually helps people, that you couldn't have built without these insights?
Replies from: lsusr↑ comment by lsusr · 2020-05-31T05:45:45.193Z · LW(p) · GW(p)
I very much did enjoy this! Thank you for the link.
There are lots of good ideas in your article. I think I actually…uh…did read your article, many months ages, and then forgot that I did so and that your articles…er…um…contributed significantly to inspiring this series.
You might like my new article on resonance [LW · GW]. It builds off your three paragraphs about guitar strings. I swear I wrote it before re-reading your statement "Resonance in chaotic systems is inherently unstable". If I did read that particular line several months ago I definitely forgot about it.
[C]an we use it to build something cool?
I'm trying to build an AGI. These insights have been very helpful so far.
comment by [deleted] · 2019-12-05T02:29:21.909Z · LW(p) · GW(p)
FFNNs’ inability to process time series data was a contributing factor to the Uber self-crashing car.
I don't see the evidence for this in the linked post and don't recall seeing this in any of the other few articles/pieces I've read on the crash. Can you point me to the evidence for this?
Replies from: lsusr↑ comment by lsusr · 2019-12-05T02:40:11.976Z · LW(p) · GW(p)
There were two bits of evidence I used to infer this.
"If I'm not sure what it is, how can I remember what it was doing?" The car wasn't sure whether Herzberg and her bike were a "Vehicle", "Bicycle", "Unknown", or "Other", and kept switching between classifications. This shouldn't have been a major issue, except that with each switch it discarded past observations. Had the car maintained this history it would have seen that some sort of large object was progressing across the street on a collision course, and had plenty of time to stop.
The first bit (above) is that the car throws away its past observations. The second bit of evidence is a consequence of the first.
"If we see a problem, wait and hope it goes away". The car was programmed to, when it determined things were very wrong, wait one second. Literally. Not even gently apply the brakes. This is absolutely nuts. If your system has so many false alarms that you need to include this kind of hack to keep it from acting erratically, you are not ready to test on public roads.
Humans have to write ugly hacks like this when when your system isn't architected bottom-up to handle things like the flow of time. A machine learning system designed to handle time series data should never have human beings in the loop this low down the ladder of abstraction.