Wacky, risky, anti-inductive intelligence-enhancement methods?

Human brains are most likely undertrained on text data. 

Looking at the scaling laws from DeepMind's Chinchilla work, it looks like a system with as many parameters and as much compute as the brain should be trained on vastly more text than any human can read in their lifetime. Thus, it seems plausible that "lack of training data" is a significant bottleneck on human cognitive capabilities. 

Thus, the question: how do we best increase the efficiency of our own text training process? There are two dimensions to this question:

• What text should we include? 
  • High quality training text, which is related to the downstream domain of interest, is best. Finding such text is often a bottleneck in consuming it, so ideally we'd compile a large corpus of excellent quality human pretraining text.
  • One option might be to train a "relevant text classifier", which identifies well-written text about domains that are useful for alignment, such as alignment research, ML, math, neuroscience, biology, game theory, etc. Then, use that classifier to scour the internet and all journals / books / etc. for useful text and compile the results.
• How should we "train" on as much text as possible?
  • One simple option is to just read more, but reading is slow. There's only so much time in the day, and reading takes time away from other activities. 
  • Another option is to convert the text into audio form using text to speech. This makes it vastly more convenient to listen to large quantities of text, but has other issues such as:
    • Images are unavailable.
    • Pausing or replaying past text is often inconvenient.
    • Math or LaTex are almost never captured well.
  • The third option, and the one I believe would be most powerful / scalable, is to use a multi-modal pretrained model to convert the text + images + math into latent representations, then to feed those latents (or dimension-reduced versions of those latents) to the human via one or more of their sensory channels. 
    • What I mean by that is to have some way of translating the latent representation of the text into sensory input for the human, e.g., into an audio signal played into the human's ears, or into vibrations which a system such as the Eyeronman vests delivers as tactile sensations.
      • Doing so would require training the human to decode these sensory representations, but that should be manageable. We'd show the human side-by-side instances of the original text / images / math along with the compressed sensory representations.
    • This approach allows for a greater throughput of information into the human, while avoiding the expense, risk and technical difficulties associated with brain computer interfaces. 
    • It also allows the human to take advantage of the preprocessing provided by the pretrained transformer, meaning that the effective compute available to the human increases as well. 
    • It may also lead to a form of "knowledge distillation" from the pretrained transformer to the human. 
      • Knowledge distillation is an approach in machine learning where the latent knowledge contained in a larger, more powerful ML model is "distilled" into a smaller model. The typical approach is for the smaller model to be trained to imitate the latent representations of the larger model on some reduced corpus of training data.
      • In this case, the human learns to process the latents generated by the model. If those latents contain representations of super-humanly performant abstractions, the human may pick up on such abstractions.
      • This could potentially aid in interpretability as well, if the human in question develops a sense for the model's internal representations.
    • We could also reduce the dimensionality of the model's latent representations or strip away irrelevant information so as to further increase the richness / density of the human's input data.

Overall, the approach that I think would be most effective is:

• Collect a corpus of high quality text, images and math from books and articles that might be alignment relevant (maybe ~20 GB of text).
• Take a multi-modal transformer pretrained on much more data than the corpus (basically, you'd use the best multi-modal model available).
• Find some scalable method of translating the model's latents into information-dense, human-learnable sensory input. 
  • These would initially appear like random noise / images / vibrations to a human, but with exposure, start to make sense as the brain adapts to the new encoding. 
  • Analogously, the Eyeronman vests I mentioned translate 3-D scene representations into vibrations. After enough time with one, people can pick up a sense of what the environment around them is like through the vibrations from the vest.
• Translate the corpus into those sensory representations.
• Feed them to the human.

This is a pretty basic setup. Information only flows in one direction, from the model to the human. Most likely, there are ways of improving things by having the model learn to produce latent representations that are more useful for the cognitive tasks the human intends to perform. E.g., the methodology in "Training Language Models with Language Feedback" can be adapted so that the human can provide feedback on what sorts of things the model should focus more / less on. 

Note that the approach of translating external information into sensory inputs handles the "getting lots of information into the human" problem. The "getting lots of information out of the human" problem isn't quite so easy to handle. Humans receive more information from their senses than they transmit via their actions, so just watching human actions probably doesn't have as high a throughput. Potentially, we can use non-invasive brain imaging tech, which seem to be progressing faster than "read + write" brain computer interfaces. Having high-throughput input + output channels for the brain would let us properly do the whole "merging with technology thing" and keep up with mildly superhuman AIs^[1], for a while at least.

1. ^{^}

   I expect horse versus automobiles comparisons in response to this point. I think this analogy isn't actually illuminating here because learning systems can be combined together much more easily than physical systems. E.g., deepmind's multi-modal Flamingo model took frozen layers from the text-only Chinchilla model and integrated them with a smaller number of trainable parameters for handling the image and image-to-text side of things. Provided you let two learning systems adapt to each other (or even just let one learning system adapt to the other), it's relatively straightforward to combine learning systems together.

More of a joke, but this post has some ideas: Mad-science IQ Enhancement:

Slatestarcodex has a great post on the intelligence of birds.¹

The big take-away: due do the weight restriction of flight, birds have been under huge evolutionary pressure to miniaturize their neurons. The result?

Driven by the need to stay light enough to fly, birds have scaled down their neurons to a level unmatched by any other group. Elephants have about 7,000 neurons per mg of brain tissue. Humans have about 25,000. Birds have up to 200,000. That means a small crow can have the same number of neurons as a pretty big monkey.

Bird brains are 10x more computationally dense than our own? This is a big deal. To put this in perspective, if you replaced just 10% of your brain volume with crow neurons you could almost double your computational capacity.

I know this sounds like complete insanity, but given how every intervention to raise IQ has been ineffective, this bird-brain scheme is probably much more promising than any pharmaceutical based approach.

This raises all sorts of interesting questions. How does miniaturization effect heat dissipation? Oxygenation? Energy consumption? Could one build a human-sized brain with bird-dense neurons?

Though the brain is immunologically privileged, there is still the neuroimmune system. Can we genetically modify crow cortical neuron progenitor cells to not trigger the neuroimmune system? Is there any chance of human neurons and bird-neurons integrating usefully?

Brain grafts have been unpromising for intelligence enhancement because you would have to replace much of the brain to make a difference. With bird tissue, this problem is 10x less relevant.

Or more ominously, what if we breed crows for brain size? Hyper-Intelligent flightless crows pecking keyboards in hedgefund basements? To paraphrase Douglas Adams, there is another theory which states that this is already happening.

I actually spent several years studying this question intensely while in grad school studying neuroscience. I have a lot of thoughts about promising avenues, but my ultimate conclusion was that there just wasn't time to get a working product before the world developed superintelligent AGI, and thus, it wasn't worth pursuing.

I recently saw What's up with psychonetics? [LW · GW]. It seems like a kind of meditation practice, but one focused on gaining access to and control of mental/perceptual resources. Not sure how risky this is, but the linked text had some warnings about misuse. It might be applicable to working or long-term memory, and specifically talks about conceptual understanding ("pure meanings") as a major component of the practice.

There's a tuplamancy-adjacent construct called a "servitor", which sounds like a kind of persistent hallucination that might be able to perform various automatic functions. I can't imagine such a thing being any more useful than a smartphone (probably much less), but perhaps it would have a faster direct-mental interface.

For example, I wonder if some kind of "notepad" servitor could expand one's working memory, which seems like a major bottleneck in humans. I.e. quickly offload writing/images to the persistent hallucination, and then just look at it to reload it. This would be easy enough to test with something like digit-span or dual-n-back.

Given the way reading fails to work in dreams (it gets completely confabulated based on expectations, and then erased/regenerated as soon as you look away/back), I think there's a significant chance a servitor couldn't persist writing with any reliability, but it might be worth a try.

Trepanning. I've heard rumors that this reduces CSF pressure on the brain allowing it to expand a bit and increase intelligence. It was once common practice among many ancient cultures, but that doesn't prove they had good reasons. I find the supposed effect and proposed mechanism of action highly dubious, and, of course, brain damage/infection may well kill you if done improperly, but you asked for "risky" and "wacky".