More Babble

post by alkjash · 2018-01-12T01:41:06.965Z · LW · GW · 24 comments

This is a link post for https://radimentary.wordpress.com/2018/01/11/more-babble/

Contents

  Babble is (Sampling From) PageRank
  The Babble Graph
  The Artist and the Critic
  Other Things that are Babble
None
24 comments

In my last babble, I introduced the Babble and Prune model of thought generation: Babble with a weak heuristic to generate many more possibilities than necessary, Prune with a strong heuristic to find a best, or the satisfactory one. I want to zoom in on this model. If the last babble was colored by my biases as a probabilist, this one is motivated by my biases as a graph theorist.

First, I will speculate on the exact mechanism of Babble, and also highlight the fact Babble and Prune are independent systems that can be mocked out for unit testing.

Second, I will lather on some metaphors about the adversarial nature of Babble and Prune. Two people have independently mentioned Generative Adversarial Networks to me, a model of unsupervised learning involving two neural nets, Generator and Discriminator. The Artist and the Critic are archetypes of the same flavor - I have argued in the past the spirit of the Critic is Satan.

Babble is (Sampling From) PageRank

Previously, I suggested that a Babble generator is a pseudorandom word generator, weighted with a weak, local filter. This is roughly true, but spectacular fails one of the technical goals of a pseudorandom generator: independence. In particular, the next word you Babble is frequently a variation (phonetically or semantically) of the previous one.

PageRank, as far as I know, ranks web pages by the heuristic of "what is the probability of ending up at this page after a random walk with random restarts." That's why a better analogy for Babble is sampling from PageRank i.e. taking a weighted random walk in your Babble graph with random restarts. Jackson Pollock is visual Babble.

Imagine you're playing a game of Scrabble, and you have the seven letters JRKAXN. What does your algorithm feel like?

You scan the board and see an open M. You start Babbling letter combinations that might start with M: MAJR, MRAJ, MRAN, MARN, MARX (oops, proper noun), MARK (great!). That's the weighted random walk. You set MARK aside and look for another place to start.

Time for a restart. You find an open A before a Triple Word, that'd be great to get! You start Babbling combinations that end with A: NARA, NAXRA, JARA, JAKA, RAKA. No luck.

Maybe the A should be in the middle of the word! ARAN, AKAN, AKAR, AJAR (great!). You sense mean stares for taking so long, so you turn off the Babble and score AJAR for (1+8+1+1)x3 = 33 points. Not too shabby.

The Babble Graph

Last time, I described getting better at Babble as increasing the uniformity of your pseudorandom Babble generator. With a higher-resolution model of Babble in hand, we should reconceptualize increasing uniformity as building a well-connected Babble graph.

What is the Babble graph? It's the graph within which your words and concepts are connected. Some of these connections are by rhyme and visual similarity, others are semantic or personal. Blood and snow are connected in my Babble graph, for example, because in Chinese they are homophones: snow is 雪 (xue), and blood is 血 (xue). This led to the following paragraph from one of my high school essays (paraphrased):

In Chinese, snow and blood sound the same: "xue." Some people think the world will end suddenly in nuclear holocaust, pandemic, or a belligerent SkyNet. I think the world will die slowly and painfully, bleeding to death one drop at a time with each New England winter.

My parents had recently dragged me out to jog in the melting post-blizzard slush.

One of my favorite classes in college was a game theory class taught by the wonderful David Parkes; my wife and I lovingly remember the class as Parkes and Rec. One of the striking ideas I learned in Parkes and Rec is that exponentially large graphs can be compactly represented implicitly in memory, as long as individual edges and neighborhoods can be computed in reasonable time. Babble is capable of generating new words and combinations, so the Babble graph contains nodes you've never thought of. It's enormous, and definitely not (a subgraph of) the connectome, but rather implicitly represented therein in a compact way. This is related to the fact that the map is not the territory, except in the study of the brain, where the map is a subset of the territory.

It follows that the Babble graph is a massive implicitly represented graph, which is traversed via random walks with random restarts. How might we optimize this data structure to better fulfill its goals?

One technique I've already mentioned is to artificially replace either Babble or Prune to train the other in isolation. This is basically unit testing via mocking. To unit test Babble, we can mock out Prune with a simplified and explicit filter like the haiku, the game of Scrabble, or word games like Contact and Convergence. To unit test Prune, we replace Babble with other sources of word strings: reading, conversation, poetry, music.

Those methods completely black box the Babble and Prune algorithms and hope they self-optimize correctly. What if we want to get our hands dirty and explicitly rewire our Babble graph?

First we have to figure out what makes a quality Babble graph. I can think of two metrics worth optimizing:

1. Connectivity. With sufficient effort (i.e. taking enough random steps and restarts) you want to eventually explore the entire graph, and repeat yourself rarely. This requires not just that the graph is connected, but that it should have good expansion. Ever feel stuck on an idea, then be struck by external inspiration to explore a disconnected set of ideas you already knew, and find it massively productive? Random walks getting trapped locally is a sign your Babble graph is a bad expander.
2. Value. Every node in your Babble graph should pay rent. I have found many abandoned components in my Babble graph - ghost towns and wastelands of neural machinery left over from experiences that are no longer relevant. They can be salvaged and repurposed, if only to generate metaphors.

Ramanujan was an extraordinarily creative mathematician who produced formulas like

$p\left(n\right)\sim \frac{1}{4n\sqrt{3}}\exp \pi \sqrt{\frac{2n}{3}}$

for the number of partitions $p\left(n\right)$ of an integer $n$. Exercise: figure out how such an exponent might occur in nature. Hint: $\zeta \left(2\right)={\pi }^2/6$.

Ramanujan was also known for his mysticism, attributing his most inspired results to his patron goddess. Mystical experiences, like LSD, are often characterized by the feeling of connectedness of all things. I think Ramanujan's genius might be the result of having a Babble graph that is an exceptionally good expander. What are those called again?

Here's a story about how I improved my Babble graph by making my bed.

It all started when Jordan Peterson told me to clean my room - because one's surroundings are a reflection of one's state of being. I decided to give it a chance and make my bed every morning.

Making my bed became a daily ritual. As I do it, I repeat the "proper and humble" mantra:

To save the world, I will start by doing the proper and humble things I know how to do within the confines of my own life.

Proper and humble were not words I'd liked in a very long time. They activated ideas I haven't wrestled with for years.

Honte is a Go term which means "the proper move." Honte is playing thickly to leave few weaknesses. Honte is killing already dead stones to remove aji. Honte is doing the proper and humble thing to prevent bad aji - failure modes you can't yet articulate. There's nothing quite like playing Go against a stronger player to put the fear of aji in you.

In relationships, honte is dedication to the removal of lingering resentment. Unhappy couples have the same fights at regular intervals; the landmines that trigger them might lay untouched for upwards of a year, but they never deactivate. Why would you allow these landmines be planted in the first place? You wouldn't leave a ladder breaker for your opponent in an unapproached corner, would you? Dedicate yourself to the removal of landmines, at least when you have the slack to do so. That's honte.

A well-connected and useful Babble graph is thickness (not to be confused with thiccness). It is written: attack from thickness. When thinking from a thick Babble graph, you're not wandering lackadaisically, building an argument from scraps lying at the side of the trail. You'll have the weight of your entire intellectual life at your back.

The Artist and the Critic

Two people have independently suggested that the Babble and Prune model is similar to an approach in machine learning known as Generative Adversarial Networks, in which production of photorealistic images (say) is turned into a game between two neural nets, Generator, who learns to generate good counterfeits, and Discriminator, who works on finding the real stuff.

This is a manifestation of the eternal war between the Artist and the Critic, a war that is both exceedingly vicious and exceedingly productive. Artists of the ages have had some choice words for their critics. Beckett:

VLADIMIR
Moron!
ESTRAGON
That's the idea, let's abuse each other.
They turn, move apart, turn again and face each other.
VLADIMIR
Moron!
ESTRAGON
Vermin!
VLADIMIR
Abortion!
ESTRAGON
Morpion!
VLADIMIR
Sewer-rat!
ESTRAGON
Curate!
VLADIMIR
Cretin!
ESTRAGON
(with finality) Crritic!
VLADIMIR
Oh!
He wilts, vanquished, and turns away.

The opening lines of Hardy's A Mathematician's Apology:

It is a melancholy experience for a professional mathematician to
find himself writing about mathematics. The function of a
mathematician is to do something, to prove new theorems, to add
to mathematics, and not to talk about what he or other mathematicians
have done. Statesmen despise publicists, painters despise
art-critics, and physiologists, physicists, or mathematicians have
usually similar feelings: there is no scorn more profound, or on
the whole more justifiable, than that of the men who make for the
men who explain. Exposition, criticism, appreciation, is work for
second-rate minds.

I have a rule inspired by Solzhenitsyn, which is that every battle which occurs between human beings also plays out within each human heart. The proper locus of the fight between Artist and Critic is not cleanly between artists and critics, but between the Babble and Prune within each individual. After all, find me an artist who has never criticized, or a great critic who is never enjoyable to read for his own sake. Exercise: get some utility out of a bad book you've recently read by checking out the savage reviews online.

Like Generator and Discriminator, a good Artist and Critic pair can together ascend to heights that neither could reach alone, and having a filter is a healthy thing. However, I stand by my argument that the overdeveloped Critic is a manifestation of Satan:

Jordan Peterson says Satan is an intellectual figure, and this idea has fermented in my imagination. Satan is the cynical and nihilistic intellectual whose thesis is “things are so bad they do not deserve to exist.”
[...]
I would propose an embellishment of the figure of Satan as the nihilistic intellectual: Satan as the critic. One of the (many) disturbing things I have noticed about my high school curriculum is that English classes are factories for creating critics out of artists. At least in my experience, we wrote short stories, poems, and other free form essays in elementary and middle school, but turned exclusively to the analytical essay by the time high school rolled around.
How frightening is that? Take a generation of teenagers, present them with the greatest literature of our civilization. Then, instead of teaching them to do the obvious thing – imitate – we teach them to analyze – the derivative work of a critic.  The work of Satan: the intellectual whose ability to criticize far exceeds his ability to create. And so we find that the best students to come out of our high schools are created in the image of Satan. For every one budding novelist, we have a dozen teenage journalists, lawyers, and activists.
Satan is the voice in your ear who says, “You will never do this well enough for it to be worth doing.” This is the burrowing anxiety that puts me off writing for weeks at at time, the anxiety that anything I produce will not justify its own existence. The subroutine in your head constantly constructing impossibly high standards and handing them to you to use as excuses to do nothing. Satan is characterized by inaction, the inaction caused by paralyzing perfectionism.

Other Things that are Babble

The Bible is the best Babble ever produced. A common atheist refrain is that the Bible is so self-contradictory, so ambiguous, so open to interpretation as to be intrinsically meaningless. Any meaning you might extract from the Bible is just a reflection of your own beliefs.

I think this is a feature, not a bug.

Not only is the Bible open to interpretation, it invites interpretation. Its stories are so varied, fantastical and morally ambiguous that they demand interpretation. The Bible stood the test of time not because it is maximally packed with wisdom, but because it produced the most insightful and varied results when paired with outside sources of Prune. When the Christian is lost and desperate, he inputs 1 Corinthians to his Prune, and voilà! Faith is restored. Peterson's The Psychological Significance of the Bible series takes advantage of exactly this feature of the Bible: it is the fertile ground upon which each individual can tell their own story. Of course, perversions can result when broken Prune filters are applied, even to the best Babble.

Perhaps writers have been optimizing for the wrong thing. Instead of directly packing insight into an essay, we should try to design high-quality Babble, fertile input for the reader's Prune.

The Oulipo is Babble training on steroids - a group of writers and mathematicians who worked based on the apparent paradox that freedom is the enemy of creativity. Creativity, the state of having a better Babble generator, is designed to solve tough, heavily constrained problems, and the Oulipians produced creative writing by imposing stricter restraints. Most famously, this method produced Perec's novel La disparition, a 300-page novel written without the letter 'e,' about "a group of individuals looking for a missing companion, Anton Vowl."

By the way, did you notice the letter missing from this entire post?

All good conversations are therapeutic, and therapeutic conversations are about letting down your guard and allowing yourself to simply Babble. Babies have no Prune at all and babble all the phonemes their adorable little mouths can produce - that's how they learn the beginnings of language so quickly. Being in a safe space is reproducing this state of development, a place where Babble can be rapidly be optimized on its own terms. Healthy teamwork and collaboration shares this quality: bouncing half-formed, half-nonsensical ideas off others and Pruning them together. Double the Babble, double the fun.

Oh, and about that missing letter? Just kidding. Ain't nobody got time for that.

24 comments

Comments sorted by top scores.