Ulisse Mini's Shortform

Some realizations about memory and learning I've been thinking about recently EDIT: here are some great posts on memory which are a deconfused version of this shortform (and written by EY's wife!)

• Anki (and SRS in general) is a tool for efficiently writing directed graph edges to the brain. thinking about encoding knowledge as a directed graph can help with making good Anki cards.
• Memory techniques are somewhat-analogous to data structures as well, e.g. the link method corresponds to a doubly linked list
• "Memory techniques" should be called "Memory principles" (or even laws).
• The "Code is Data" concept makes me realize memorization is more widely applicable, you could e.g. memorize the algorithm for integration in calculus. Many "creative" processes like integration can be reduced to an algorithm.
• Truely part of you [LW · GW] is not orthogonal to memory techniques principles, it uses the fact that a densely connected graph is less likely to be disconnected from randomly deleting edges, similar to how the link and story methods. Just because you aren't making silly images doesn't mean you aren't using the principles.
• (untested idea for math): Journal about your thought processes after solving each problem, then generalize to form a problem solving algorithm / checklist and memorize the algorithm

In a recent post, [LW · GW] John mentioned how Corrigability being a subset of Human Values means we should consider using Corrigability as an alignment target. This is a useful perspective, but I want to register that $X\subseteq Y$ doesn't always imply that doing $X$ is "easier" than $Y$, this is similar to the problems with problem factorization for alignment [? · GW] but even stronger! Even if we only want to solve $X$ and not $Y$, $X$ can still be harder!

For a few examples of this:

• Acquiring half a Strawberry by itself is harder than a full strawberry (You have to get a full strawberry, then cut it in half) (This holds for X=MacBook, Person, Penny too)
• Let $L$ be a lemma used in a proof of  $T$ (meaning $L\subseteq T$ in some sense). It may be that $T$ can be immediately proved via a known more general theorem $T^{\prime }$. In this case $L$ is harder to directly prove then $T$.
• When writing an essay, writing section 3 alone can be harder than writing the whole essay, because it interacts with the other parts, you learn from writing the previous parts, etc. (Sidenote: There's a trivial sense^[1] in which writing section 3 can be no harder than writing the whole essay, but in practice we don't care as the whole point to considering a decomposition is to do better.)

In general, depending on how "natural" the subproblem in the factorization is, subproblems can be harder than solving the original problem. I believe this may (30%) be the case with corrigibility; mainly because (1) Corrigability is anti-natural in some ways, and (2) Humans are pretty good at human values while being not-that-corrigible.

1. ^{^}

   Just write the whole thing then throw everything else away!

There are a series of math books that give a wide overview of a lot of math. In the spirit of comprehensive information gathering [AF · GW], I'm going to try to spend my "fun math time" reading these.

I theorize this is a good way to build mathematical maturity, at least the "parse advanced math" part. I remember when I became mathematically mature enough to read Math Wikipedia, I want to go further in this direction till I can read math-y papers like Wikipedia.

↑ comment by Jack O'Brien (jack-o-brien) · 2022-12-01T09:02:42.551Z · LW(p) · GW(p)

which of these books are you most excited about and why? I also want to do more fun math reading

Replies from: ulisse-mini

↑ comment by Ulisse Mini (ulisse-mini) · 2022-12-01T18:14:43.614Z · LW(p) · GW(p)

(Note; I haven't finished any of them)

Quantum computing since Democritus is great, I understand Godel's results now! And a bunch of complexity stuff I'm still wrapping my head around.

The Road to Reality is great, I can pretend to know complex analysis after reading chapters 5,7,8 and most people can't tell the difference! Here's a solution to a problem in chapter 7 I wrote up.

I've only skimmed parts of the Princeton guides, and different articles are written by different authors—but Tao's explanation of compactness (also in the book) is fantastic, I don't remember specific other things I read.

Started reading "All the math you missed" but stopped before I got to the new parts, did review linear algebra usefully though. Will definitely read more at some point.

I read some of The Napkin's guide to Group Theory, but not much else. Got a great joke from it:

Here's a collection of links to recent "mind-reading" related results using deep learning. Comment ones I missed!

• Decoding images in the brain via a diffusion model
• Reconstructing internal language while watching videos using a language model
• Brain embeddings with shared geometry to artificial contextual embeddings, as a code for representing language in the human brain

PS: It seems like a good idea for alignment people (like Steve [AF · GW]) to have the capacity to run novel human-brain experiments like these. If we don't currently have this capacity, well... Free dignity points to be gained :)

I'm going to argue meditation/introspection skill is a key part of an alignment researcher's repituaire. I'll start with a somewhat fake Taxonomy of approaches to understanding intelligence/agency/value formation

• Study artificial intelligences
  • From outside (run experiments, predict results, theorize)
  • From inside (interpretability)
• Study biological intelligences
  • From outside (psychology experiments, theorizing about human value & intelligence formation, study hypnosis^[1])
  • From inside (neuroscience, meditation, introspection)

I believe introspection/meditation is a neglected way to study intelligence among alignment researchers

• You can run experiments on your own mind at any time. Lots of experimental bits free for the taking
• I expect interviewing high level meditators to miss most of the valueble illegible intuitions (both from lack of direct experience, and lacking the technical knowledge to integrate that experience with)
• It has known positive side effects like improved focus, reduced stress etc (yes it is possible to wirehead, I believe it to be worth the risk if you're careful though.)

What are you waiting for, get started!^[2]

Quick thoughts on creating a anti-human chess engine [LW(p) · GW(p)].

1. Use maiachess to get a probability distribution over opponent moves based on their ELO. for extra credit fine-tune on that specific player's past games.
2. Compute expectiminimax search over maia predictions. Bottom out with stockfish value when going deeper becomes impractical. (For MVP bottom out with stockfish after a couple ply, no need to be fancy.) Also note: We want to maximize (P(win)) not centipawn advantage.
3. For extra credit, tune hyperparameters via self-play against maia (simulated human). Use lichess players as a validation set.
4. ???
5. Profit.

People Power

To get a sense that more is possible [? · GW] consider

1. The AI box experiment, and its replication [LW · GW]
2. Mentalists like Derren Brown (which is related to 1 [LW(p) · GW(p)])
3. How the FBI gets hostages back with zero leverage (they aren't allowed to pay ransoms)

(This is an excerpt from a post I'm writing which I may or may not publish. the link aggregation here might be useful in of itself)

Petition to rename "noticing confusion" to "acting on confusion" or "acting to resolve confusion". I find myself quite good at the former but bad at the latter—and I expect other rationalists are the same.

For example: I remember having the insight thought leading to lsusr's post [LW · GW] on how self-reference breaks the orthogonality thesis, but never pursued the line of questioning since it would require sitting down and questioning my beliefs with paper for a few minutes, which is inconvenient and would interrupt my coding.

Exercise: What mistake is the following sentiment making?

If there's only a one in a million chance someone can save the world, then there'd better be well more than a million people trying.

Answer:

Are most uncertainties we care about logical rather than informational? All empirical ML experiments are pure computations a Bayesian superintelligence could do in its head. How much of our uncertainty comes from computational limits in practice, versus actual information bottlenecks?

KL-divergence and map territory distinction

Crosspost from my blog

The cross-entropy is defined as the expected surprise when drawing from $p\left(x\right)$, which we're modeling as $q\left(x\right)$. Our map is $q\left(x\right)$ while $p\left(x\right)$ is the territory.

$$H(p,q)=\sum _{x}p\left(x\right)\log \frac{1}{q\left(x\right)}$$

Now it should be intuitively clear that $H(p,q)\ge H(p,p)$ because an imperfect model $q\left(x\right)$ will (on average) surprise us more than the perfect model $p\left(x\right)$.

To measure unnecessary surprise from approximating $p\left(x\right)$ by $q\left(x\right)$ we define

$$D_{KL}(p\parallel q)=H(p,q)-H(p,p)$$

This is KL-divergence! The average additional surprise from our map approximating the territory.

Now it's time for an exercise, in the following figure $q^{\ast }\left(x\right)$ is the Gaussian that minimizes $D_{KL}(p\parallel q)$ or $D_{KL}(q\parallel p)$, can you tell which is which?