Posts
Comments
In How Not to Network a Nation, Benjamin Peters points out that Soviet computer networking projects failed partially due to this same centralizing idealism. Soviet networking projects all saw the ideal network as a central nervous system, connecting productive outposts with the central "brain" in Moscow. By contrast, the developers of ARPANET envisioned their network as a brain with the individual computers as the neurons. The formulation of ARPANET was as a distribution of homogeneous elements, whereas the formulation of the OGAS (the largest Soviet project) was hierarchical and heterogeneous from the start. It's interesting to see the similarities between all of these unsuccessful networking projects.
This reminds me a lot of what I would do when I was trying to access the Spirit of God to get divine guidance, back when I was Mormon. The essential distinction is in your sentence:
Of course, the fact that you’ve accurately expressed your brain’s sense of what’s going on doesn’t mean you’ve found the bona-fide truth.
In the process of getting my mind untied from my Mormon upbringing, I became a lot more connected with my rational mind and more untrusting of my subconscious. This is a good reminder that you have to pay attention to that subconscious processing when it comes to understanding what's happening in your head.
Edit: I'd like to add a point that another exmormon friend made to me:
I think there's a lot of value in this sort of thinking, but as an untrusted starting point that you want to confirm or disprove as fast as possible, because it engages systems that can be hijacked
In my experience, it is not safe to rely on this sort of subconscious exploration when you have been taught to have a strong fear of certain "unsafe" or "sinful" thoughts. It brings you too close to emotional reasoning.
Forget OOD for a minute; ERM can't even learn to avoid spurious correlations that have counterexamples in the training data. Datasets like Waterbirds (used in that previously linked paper) are good toy datasets for figuring this out. I think we need to solve this problem first before trying to figure out OOD generalization.