LessWrong 2.0 Reader

I'm going to burn some social capital on asking a stupid question, because it's something that's been bothering me for a long time. The question is, why do we think we know that it's necessary to understand a lot of mathematics to productively engage in FAI research?

My first line of skepticism can perhaps be communicated with a simplified analogy: It's 10,000 BC and two people are watching a handful of wild sheep grazing. The first person wonders out loud if it would be possible to somehow teach the sheep to be more docile.

The second person scoffs, and explains that they know everything there is to know about training animals, and it's not in the disposition of sheep to be docile. They go on to elaborate all the known strategies for training dogs, and how none of them can really change the underlying temperament of the animal.

The first person has observed that certain personality traits seem to pass on from parent to child and from dog to puppy. In a flash of insight they conceive of the idea of intentional breeding.

They cannot powerfully articulate this insight at the level of genetics or breeding rules. They don't even know for a fact that sheep can be bred to be more docile. But nonetheless, in a flash, in something like one second of cognitive experience they've gone from not-knowing to knowing this important secret.

End of analogy. The point being: it is obviously possible to have true insights without having the full descriptive apparatus needed to precisely articulate and/or prove the truth of the insight. In fact I have a suspicion that most true, important insight comes in the form of new understandings that are not well-expressed by existing paradigms, and eventually necessitate a new communication idiom to express the new insight. Einstein invented Einstein notation because not just because it's succinct, but because it visually rearranges the information to emphasize what's actually important in the new concept he was communicating and working with.

So maybe my steelman of "why learn all this math" is something like "because it gives you the language that will help you construct/adapt the new language which will be required to express the breakthrough insight." But that doesn't actually seem like it would be important in being able to come up with that insight in the first place.

I will admit I feel a note of anxiety at the thought that people are looking at this list of "prerequisites" and thinking, wow, I'm never going to be useful in thinking about FAI. Thinking that because they don't know what Cantor's Diagonalization is and don't have the resources in terms of time to learn, their brainpower can't be productively applied to the problem. Whereas, in contrast, I will be shocked if the key, breakthrough insight that makes FAI possible is something that requires understanding Cantor's Diagonalization to grasp. In fact, I will be shocked if the key, breakthrough insight can't be expressed almost completely in 2-5 sentences of jargon-free natural language.

I have spent a lot of words here trying to point at the reason for my uncertainty that "learn all of mathematics" is a prerequisite for FAI research, and my concerns with what I perceive to be the unproven assumption that the pathway to the solution necessarily lies in mastering all these existing techniques. It seems likely that there is an answer here that will make me feel dumb, but if there is, it's not one that I've seen articulated clearly despite being around for a while.

Before I read (2), I want to note that a universal idea that one is responsible for all the consequences of one's accurate speech - in an inevitably Asymmetric Justice / CIE fashion - seems like it is effectively a way to ban truth-seeking entirely, and perhaps all speech of any kind. And the fact that there might be other consequences to true speech that one may not like and might want to avoid, does not mean it is unreasonable to point out that the subclass of such consequences that seems to be in play in these examples, seems like a subclass that seems much less worth worrying about avoiding. But yes, Kant saying you should tell the truth to an Axe murderer seems highly questionable, and all that.

And echo Jessica that it's not reasonable to say that all of this is voluntary within the frame you're offering, if the response to not doing it is to not be welcome, or to be socially punished. Regardless of what standards one chooses.

I think that is a far from complete description of my decision theory and selection of virtues here. Those are two important considerations, and this points in the right direction for the rest, but there are lots of others too. Margin too small to contain full description.

At some point I hope to write a virtue ethics sequence, but it's super hard to describe it in written form, and every time I think about it I assume that even if I do get it across, people who speak better philosopher will technically pick anything I say to pieces and all that and I get an ugg field around the whole operation, and assume it won't really work at getting people to reconsider. Alas.

In my own model, people have an emotional need for a certain amount of attention. Once that need however is satisfied, they usually don't have a strong need for more and thus don't engage in actions that are just done to get attention.

Interacting with people who do what they are doing just to get attention because they are starved of it instead of having other goals feels shallow.

Another interesting aspect is that in my experience they need for the attention of other people goes down with the ability to give attention to yourself and feel clearly that one exists.

Just to share my two cents on the matter, the distinction between abstract vectors and maps on the one hand, and columns with numbers in them (confusingly also called vectors) and matrices on the other hand, is a central headache for Linear Algebra students across the globe (and by extension also for the lecturers). If the approach this book takes works for you then that's great to hear, but I'm wary of `hacks' like this that only supply a partial view of the distinction. In particular matrix-vector mulitplication is something that's used almost everywhere, if you need several translation steps to make use of this that could be a serious obstacle. Also the base map ${\Phi }_B:F^n\to V$ that limerott mentions is of central importance from a category-theoretic point of view and is essential in certain more advanced fields, for example in differential geometry. I'm therefore not too keen on leaving it out of a Linear Algebra introduction.

Unfortunately I don't really know what to do about this, like I said this topic has always caused major confusion and the trade-off between completeness and conciseness is extremely complicated. But do beware that, based on only my understanding of your post, you might still be missing important insights about the distinction between numerical linear algebra and abstract linear algebra.

To list a few communities I would consider to be truth-seeking and with whom I have interacted: Chaos Computer Club, General Semantics, Quantified Self/Biohacking, Radical Honesty, Perceptive Pedagogy, Debating (BPS-based), Wikidata and Skeptics.SE

Agree strongly with this decomposition of integrity. They're definitely different (although correlated) things.

My biggest disagreement with this model is that the first form (structurally integrated models) seems to me to be something broader? Something like, you have structurally integrated models of how things work and what matters to you, and take the actions suggested by the models to achieve what matters to you based on how things work?

Need to think through this in more detail. One can have what one might call integrity of thought without what one might call integrity of action based on that thought - you have the models, but others/you can't count on you to act on them. And you can have integrity of action without integrity of thought, in the sense that you can be counted on to perform certain actions in certain circumstances, without integrity of thought, in which case you'll do them whether or not it makes any sense, but you can at least be counted on. Or you can have both.

And I agree you have to split integrity of action into keeping promises when you make them slash following one's own code, and keeping to the rules of the system slash following others' codes, especially codes that determine what is blameworthy. To me, that third special case isn't integrity. It's often a good thing, but it's a different thing - it counts as integrity if and only if one is following those rules because of one's own code saying one should follow the outside code. We can debate under what circumstances that is or isn't the right code, and should.

So I think for now I have it as Integrity-1 (Integrity of Thought) and Integrity-2 (Integrity of Action), and a kind of False-Integrity-3 (Integrity of Blamelessness) that is worth having a name for, and tracking who has and doesn't have it in what circumstances to what extent, like the other two, but isn't obviously something it's better to increase than decrease by default. Whereas Integrity-1 is by default to be increased, as is Integrity-2, and if you disagree with that, this implies to me there's a conflict causing you to want others to be less effective, or you're otherwise trying to do extraction or be zero sum.

If I think about preserving system integrity over time, or an agent preserving goal integrity over time, I think of the ability for the system/agent to move through a wide variety of environments without being broken / fundamentally changed in ways it doesn't want by outside forces. This conceptualisation of integrity - being able to preserve the core parts of you and your goals over time - seems good to me. (Reminds me of Ray / Critch talking about being a robust agent.) Someone with integrity is wholly them and will stay whole over the long run, even if crazy things are thrown at them. It's not a claim about their competences/goals/beliefs now, it's a claim about the long-term integrity of their competences/goals/beliefs.

Most of these are symmetric weapons too which don’t rely on truth.

The whole point of pro-truth norms is that only statements that are likely to be true get intersubjectively accepted, though...

This makes me think that you're not actually tracking the symmetry/asymmetry properties of different actions under different norm-sets.

Here's .tmux.conf, however it mostly covers the in-tmux things like split/tab management (e.g. I open & switch to new tabs with alt-1/2/... instead of default C-b 1/2/... This mirrors the browser behavior and is 1 less keypress):

https://github.com/mwgkgk/dotfiles/blob/master/tmux/.tmux.conf

Tmux allows neat tricks like sending a window between sessions or sending keypresses to a session. E.g. I have a script called "portal" that opens a new window in a target tmux session (that we're opening a "portal" to) with the current directory, and brings that window to the foreground.

Another benefit of tmux is that all of my editor sessions are independent of Xorg and so can survive a restart of X or be reused from a different X session (e.g. when testing a WM).

Here's sort of a teaser of which tmux / urxvt sessions I have bound in sxhkd (some are still bound from dwm config):

https://github.com/mwgkgk/dotfiles/blob/master/sxhkd/sxhkdrc

The launchers themselves (e.g. "ship", "tower", "girl") are unfortunately not online at this point. What these files do is open (with few exceptions) a floating window with the named tmux session and bring it to front, or run the args in a new tmux window of the target session. These are the different-purpose knowledge-management sessions I was referring to.

Between those are 2 firefox sessions, which is another thing perhaps worth mentioning. I run 8 thunderbird sessions with RSS feeds and and ~20 firefox sessions. Two of those found in sxhkd config are floating, for quick anonymous / non-anonymous-programming-related lookups. I find separating the browser sessions very valuable for honing the suggestion streams that google / youtube / ... throw at us.