LessWrong 2.0 Reader

After a bit of prompting Claude 3.5 Sonnet (new) still knows the canary string 

Also, in this case you want to actually spend the money before the world ends. So actually losing money on interests payments isn't the real problem, the real problem is that if you actually enjoy the money you risk losing everything and being bankrupt/in debtors prison for the last two years before the world ends. There's almost no situation in which you can be so sure of not needing to pay the money back that you can actually spend it risk-free. I think the riskiest short-ish thing that is even remotely reasonable is taking out a 30-year mortgage and paying just the minimum amount each year, such that the balance never decreases. Worst case you just end up with no house after 30 years, but not in crippling debt, and move back into the nearest rat group house.

Gotcha, I interpreted your comment as implying you were interested in trying to improve your views on the topic in collaboration with someone else (who is also interested in improving their views on the topic).

So I thought it was relevant to point out that people should probably mostly care about a different question.

the magnitude of effects of such work is large enough that "positive sign" often is enough information to decide that it dominates many alternatives, though certainly not all of them

FWIW, my guess is that this is technically true if you mean something broad by "many alternatives", but if you mean something like "the best several alternatives that you would think of if you spent a few days thinking about it and talking to people" then I would disagree.

Amazingly detailed article covering malevolence, interaction with power, and the other nuances! Have been thinking of exploring similar topics, and found this very helpful. Besides the identified research questions, some of which I highly agree with, one additional question I was wondering is: do self-awareness of one's own malevolence factors help one to limit the malevolence factors? if so how effective would that be? how would this change when they have power? 

Interesting. Looks like they are starting with a deep tunnel (530 m) and may eventually move to the deepest tunnel in Europe (1444 m). I wish I could find numbers on how much weight will be moved or the total energy storage of the system. (They say quote 2 MW, but that's power, not energy—how many MWh?)

According to this article, a Swiss company is building giant gravity storage buildings in China and out of 9 total buildings, there should be a total storage of 3700 MWh, which seems quite good! Would love to know more about the technology.

I’m not “trying to figure out” whether to work on capabilities, having already decided I’ve figured it out and given up such work.  Are you interested in talking about this to someone like me?  I can’t tell whether you want to restrict discussion to people who are still in the figuring out stage.  Not that there’s anything wrong with that, mind you.

Sure. But what’s interesting to me here is the implication that, if you restrict yourself to programs below some maximum length, weighing them uniformly apparently works perfectly fine and barely differs from Solomonoff induction at all.

This resolves a remaining confusion I had about the connection between old school information theory and SLT. It apparently shows that a uniform prior over parameters (programs) of some fixed size parameter space is basically fine, actually, in that it fits together with what algorithmic information theory says about inductive inference.

I'm less interested in what existing groups of things we call "species" and more interested in what the platonic ideal of a species is & how we can use it as an intuition pump. This is also why I restrict "species" in the blogpost to "macrofauna species", which have less horizontal gene transfer & asexual reproduction.

I haven't looked much at the extended phenotype literature, although that is changing as we speak. Thanks for pointing me in that direction!

The thing I wanted to communicate was less "existing groups of things we call species are perfect examples of how super-organisms should work" and more "the definition of an ideal species captures something quite salient about what it means for a super-organism to be distinct from other super-organisms and its environment." In practice, yes, looking at structure does seem to be better.