Comment by nym-1 on [deleted post] 2023-08-15T06:58:06.197Z

total token activation values

Is this a straight sum, where negative actvals cancel with positive ones? If so, if you instead summed the absolute values of activations be more indicative of "distributed effort"? Or if only positive actvals have an effect on downstream activations, maybe a better metric for "effort" would be to sum only the positive ones?

I'm not sure whether total actvals for a token is a good measure of the "effort" it takes to process it. Maybe. In brains, the salience (somewhat analogous to actvals) of some input is definitely related to how much effort it takes to process it (as measured by the number of downstream neurons affected by it^[1]), but I don't know enuf about transformers yet to judge if and how it analogises.

1. ^{^}

   For sufficiently salient input, there's a threshold at which it enters "consciousness", where it's processed in a loop for a while affecting a much larger portion of the network compared to inputs that don't reach the threshold.

   Another way transformers are different: every tensor operation involves the same number of cells & bits, so computational resources spent per token processed is constant; unless I'm mistaken?

Comment by wyrd (nym-1) on If I Was An Eccentric Trillionaire · 2023-08-10T01:14:05.292Z · LW · GW

ohmygodthatlojbanbabyissocute! —but anyway I don't think you need to be raised speaking a new language for a good one to have large effect on your ability to think.

I find it weird that people call it the "Sapir-Whorf hypothesis" as if there's an alternative way people can robustly learn to think better. Engineering a language isn't really about the language, it's about trying to rewrite the way we think. LessWrong and other academic disciplines have had decent success with this on the margin, I'd say—and the phrase "on the margin" is a good example of a recent innovation that's marginally helped us think better.

There seems to be a trend that breakthrough innovations often arise from somebody trying to deeply understand and reframe the simplest & most general constituents of whatever field they're working in. At least it fits with my own experience and the experience of others I've read. I think it's fairly common advice in math research especially.

The reason I'm enthusiastic about the idea of creating a conlang is that all natural languages have built up a large amount of dependency debt that makes it very difficult to adapt them to fit well with whatever specialised purposes we try to use it for. Just like with large code projects, it gets increasingly expensive to refactor the base if it needs to be adapted to e.g. serve novel purposes.^[1]

For language, you also face the problem that even if you've correctly identified a pareto-improvement in theory, you can't just tell people and expect them to switch to your system. Unless they do it at the same time (atomic commit), there's always going to be a cost (confusion, misunderstanding, embarrassment, etc) associated with trying to push for the change. And people won't be willing to try unless they expect that other people expect it to work.

Those are some of the reasons I expect natural languages to be very suboptimal relative to what's possible, and just from this I would expect it to be easy to improve upon it for people who've studied cognition to the extent that LessWrongers have—iff those changes could be coordinated on. For that, we first need a proof of concept. It's not that it's futile or pointless—just nobody's tried. Lojban doesn't count, and while Ithkuil is probably the closest, it doesn't have the right aims. You'd really be willing to spend only ~40M on it?

1. ^{^}

   Let's say you're trying to rewrite a very basic function that was there from the beginning, but you notice that 40 other functions depend on it. The worst-case complexity of trying to refactor it isn't limited to those 40 functions: even if you only have to adapt 10 of them to fit your new ontology, those might have further dependencies you have to control for. When the dependencies are obscure, it can get whac-a-mole-y: for each change you consider, you have to search a branching graph of dependencies to check for new problems.
   
   Language is just worse because A) you have to coordinate a change with many more people, and B) very few words have "internal definitions" that make it easy to predict the consequences of intervening on them. Words usually have magnetic semantics/semiotics, where if you try to shift the meaning of one word, the meaning of other words will often 1) move in to fill the gap, 2) be dragged along by association, 3) be displaced, or 4) be pushed outward by negative association.