LessWrong 2.0 Reader

That sounds about right. And "people sometimes feel that way" is a good explanation for the downvote in my opinion. I was arguing the object-level premises of the post because the "disagree" downvote was factually wrong, and this factual wrongness, I argue, is caused by a faulty understanding of how truth works, and this faulty understanding is most common in the western world and in educated people, and in the ideologies which correlate with western thought and academia.

If you disagree with something which is true, I think the only likely explanations are "Does not understand" and "Has a dislike of", and the bias I pointed out covers both of these possibilities (the former is a "map vs territory" issue and the latter is a "morality vs reality" issue).

I think you figured out what went wrong nicely, but in the end the disagreement remains. I still consider my point likely. If somebody comes along and tells me that they disagreed with it for other reasons, I might even argue that they're lying to themselves, as I'm way to disillusioned to think that a "will to truth" exists. I think social status, moral values and other such things are stronger motivators than people will admit even to themselves.

One reason is just that eating food is enjoyable.  I limit the amount of food I eat to stay within a healthy range, but if I could increase that amount while staying healthy, I could enjoy that excess.

I think there are two aspects to the enjoyment of food.  One is related to satiety.  I enjoy the feeling of sating my appetite, and failing to sate it leaves me with te negative experience of craving food (negative if I don't satisfy those cravings.

But the other aspect is just the enjoyment of eating each individual bite of food.  Not the separate enjoyment of sating my appetite, but just the experience of eating.*

When I was younger and much more physically active I ate very large amounts of food.  I miss being able to do that.  I'm just as sated now with the much smaller portions I eat, but eating a small breakfast instead of a large one is a different experience. 

This probably doesn't justify some sort of risky intervention in increasing liver size.  Food is enjoyable, but so are a lot of other things in life.  But shifting to a higher protien diet seems like the kind of safe intervention, potentially even also healthier in other respects, that, if it has the side effect of being able to eat a little more food, could improve quality of life with minimal other costs.  Potential costs I see are related to the price of protein relative to other sources of nutrition, the cost of additional food (if the point is being able to eat more, you've got spend money for that excess), and, depending on one's moral views, something related to the source of the protien being added.

*I think Kahneman's remembering vs. expereincing selves adds some confusion here as well. When we remember a meal we don't necessarily remember the enjoyment we got from every bite, but probably put more weight on the feeling of satiety and the peak experience (how good did it taste at its best?).  But the experiencing self experiences every bite.  How much you want to weight the remembering vs. experiencing self is a philosophical issue, but I just want to note that it comes up here.

I wonder if anybody has tried to quantify how much it's worth to be a swing voter. I imagine if you are the government contractor up for renewal then it's worth quite a lot, but I wonder how much of the money/benefits the average Joe sees.

I don't know much about swing state benefits except that Milwaukee, Wisconsin got their lead pipes replaced by the fed and the workers were required to be local and they say they were paid quite well https://youtube.com/watch?v=4VpwgG0P8VU

Aw man we used the same word for different things again

Your examples fit the definition quite well. Apparently this is in the dictionary now. https://www.merriam-webster.com/dictionary/gaslighting

Why would we stretch the definition of lawyer in such a way? That's not what the word "lawyer" means, either in the dictionary sense or in the sense of how people use the word. And even if you can come up with a reason to stretch it to include all those professions, what makes you think that's what Eliezer was doing?

I'm missing something here. Why would I want a bigger liver? I mean, from this account, liver size is obviously something that the body is controlling. You list various interventions to make it bigger, which predictably have bad effects. But why would I want to change something that my body is already managing perfectly well?

The only reason I could find was this:

Athletes have higher resting metabolic rates than non-athletes; their bodies use more energy, even when they’re not exercising. That means they can eat more without getting fat.

Is that it? Why not just^[1]...not eat more? These are athletes. They eat to sustain themselves in the pursuit of athletic excellence. They can already "just" not eat more. If they couldn't, they would not be athletes.

I agree there are people, notably Eliezer, who can't "just" not eat more without being as unable to function as if they were starving. I can't see a larger liver burning up more energy helping with that.

The "sequential calculation steps" I'm referring to are the ones that CoT adds above and beyond what can be done in a single forward pass.  It's the extra sequential computation added by CoT, specifically, that is bottlenecked on the CoT tokens.

There is of course another notion of "sequential calculation steps" involved: the sequential layers of the model.  However, I don't think the bolded part of this is true:

replacing the token by a dot reduces the number of serial steps the model can perform (from mn to m+n, if there are m forward passes and n layers)

If a model with N layers has been trained to always produce exactly M "dot" tokens before answering, then the number of serial steps is just N, not M+N.

One way to see this is to note that we don't actually need to run M separate forward passes.  We can just pre-fill a context window containing the prompt tokens followed by M dot tokens, and run 1 forward pass on the whole thing.

Having the dots does add computation, but it's only extra parallel computation – there's still only one forward pass, just a "wider" one, with more computation happening in parallel inside each of the individually parallelizable steps (tensor multiplications, activation functions).

(If we relax the constraint that the number of dots is fixed, and allow the model to choose it based on the input, that still doesn't add much: note that we could do 1 forward pass on the prompt tokens followed by a very large number of dots, then find the first position where we would have sampled a non-dot from from output distribution, truncate the KV cache to end at that point and sample normally from there.)

If you haven't read the paper I linked in OP, I recommend it – it's pretty illuminating about these distinctions.  See e.g. the stuff about CoT making LMs more powerful than $T{C}^0$ versus dots adding more power withing $T{C}^0$.

It's possible no one tried literally "recreate OkC", but I think dating startups are very oversubscribed by founders, relative to interest from VCs

If this is true, it's somewhat cruxy for me.

I'm still disappointed that no one cared enough to solve this problem without VC funding.

I only skimmed the retrospective now, but it seems mostly to be detailing problems that stymied their ability to find traction.

Right. But they were not relentlessly focused on solving this problem. 

I straight up don't believe that that the problems outlined can't be surmounted, especially if you're going for a cashflow business instead of an exit.