LessWrong 2.0 Reader

Really interesting work, especially the three stages you find. Regarding your question whether your classifier will generalize to negated statements: This has been explored in our Neurips 2024 paper "Truth is Universal: Robust Detection of Lies in LLMs" (https://arxiv.org/abs/2407.12831). In fact, true and false negated statements separate along a different direction than statements without negation, so a classifier would not generalize. The truthfulness representation found in this paper is also universal and can be found in multiple LLMs which is consistent with your findings :) 

So why don't the four states sign a compact to assign all their electoral votes in 2028 and future presidential elections to the winner of the aggregate popular vote in those four states? Would this even be legal?

It would be legal to make an agreement like this (states are authorized to appoint electors and direct their votes however they like; see Chiafalo v. Washington) but it's not enforceable in the sense that if one of the states reneges, the outcome of the presidential election won't be reversed.

The following is probably just an ELI5 version of Dacyn's answer [LW(p) · GW(p)]:

Just because you use a word (such as "set"), it doesn't mean that it has an unambiguous meaning.

Imagine the same discussion about "numbers". Can you subtract 5 from 3? In the universe of natural numbers, the answer is no. In the universe of all integers, the answer is yes. Is there a number such that if you multiply it by itself, the result is 2? In the universe of rational numbers, the answer is no; in the universe of real numbers, the answer is yes.

Here you probably don't see any problem. Some statements can be true about real numbers and false about rational numbers, because those are two different things. A person who talks about "numbers" in general, needs to be more specific. As we see here, defining addition, subtraction, multiplication, and division is still not enough to allow us to figure out the answer to "∃a: a × a = 2".

It's similar with the sets. The ZF(C) axioms are simply not enough to pinpoint what you actually mean by a "set". They reduce the space of possible meanings, sufficiently to let us prove many interesting things, but there are still (infinitely) many possible meanings compatible with all of the axioms. For some of those meanings, CH is true, for other meanings, CH is false.

Is there a "set" greater than ℵ₀ but smaller than 2^ℵ₀? It depends.

Is there a "number" that is greater than 2 but smaller than 3? It depends.

What makes certain axioms "true" beyond mere consistency?

Nothing. What do you mean by "true" here? Matching our physical universe? That in general is not what math does. The natural numbers may already include some that exceed the number of particles in our universe. The real numbers are inspired by measuring actual things, but do we really need an infinite number of decimal places? On top of all that, sets are merely mental constructs. A set of {2, 8, 33897798} does not imply anything about our world.

Is there a meaningful distinction between mathematical existence and consistency?

No.

Can we maintain mathematical realism while acknowledging the practical utility of the multiverse approach?

Maybe I am missing some important aspect, but the "multiverse" seems to me just like a intuitively helpful metaphor, but the actual problem is more like this: is the natural number "2" the same object as the integer "2", the real number "2.0", the Gaussian integer "2+0i", the complex number "2.0+0.0i", etc.?

One possible approach is to say: those are different domains of discourse... uhm, let's call them parallel universes to make it intuitive for the sci-fi fans. The object in a parallel universe is a different object, but also in some sense the captain Picard from the parallel universe is a natural counterpart to our captain Picard. They are generally the same unless specified otherwise for the plot relevant reasons, just like "2.0" from the real number universe is the natural counterpart to "2" from the integer universe, except that the former can be divided by three and the latter cannot. (Some things do not have a counterpart in the other universe, etc.) This feels like a natural approach for real vs complex numbers, and probably like an overkill for natural numbers vs integers.

The assumption of different universes kinda goes against the Occam's razor; we could simply move all these objects into the same universe (different planets perhaps) and make a story about a spaceship captain from Earth and a spaceship captain from Mars. Now we don't have the concept of a natural counterpart, and the analogies need to make explicitly: the horses on Earth correspond to the giant six-legged lizards on Mars. There is a set of natural numbers, the set of real numbers, and a function N -> R which maps the object "2" to the object "2.0". More importantly, there is no such thing as "addition"; there are actually two different things, "natural number addition" and "real number addition", and we call the latter the extension of the former, if for each pair of natural numbers, the counterpart of their sum is the same as the sum of their counterparts. The question whether "2" and "2.0" are intrinsically the same object can become kinda meaningless, if we always talk about numbers qua members of one or the other set. They could be the same object, or they could be different objects; the important thing is what they do, i.e. how they participate in various functions and relations.

(This kinda reminds me of Korzybski's "Aristotelian" vs "Non-Aristotelian" thinking, where the former is about what things are, while the latter is about how things are related to each other. Is "2" the same as "2.0"? A meaningless question, from the non-A perspective. The important thing is what they do; how are they related to other numbers. The important facts about "2" are that "1+1=2" and "2+2=4" etc. We can show that we can map N to R in a way that preserves all existing addition and multiplication, and whenever we do so, "2.0" is the image of "2". And that's all there is.)

With sets, I guess it is similar. If we have different definitions of what a "set" means, is the empty set according to definition X the same mathematical object as the empty set according to definition Y? The question is meaningless, from the non-A perspective; but to avoid all the complicated philosophy, it is easier to say that one lives in the universe X, and the other lives in the universe Y, so they are "kinda the same, but not the same". But to be precise, there is no such thing as an "empty set", only something that plays the role of an empty set in a certain system. Some systems could not even have such role, or they could have multiple distinct empty sets -- for example, we could imagine a system where each set has a type, and the "empty set of integers" is different from the "empty set of reals", because it has a different content type.

(Now I suspect I have opened a new can of worms, like how to reconcile Platonism with Korzybski's non-A thinking, and... that would be a long debate that I would prefer to avoid. My quick opinion is that perhaps we should aim for some kind of "Platonism of function" rather than "Platonism of essence", i.e. what the abstract objects do rather than what they are. The question is whether we should still call this approach "Platonism", perhaps some other name would be better.)

Playing ultimatum game against an agent that gives in to $9 from rocks but not from us is not in the fair problem class, as the payoffs depend directly on our algorithm and not just on our choices and policies.

https://arbital.com/p/fair_problem_class/

A simpler game is “if you implement or have ever implemented LDT, you get $0; otherwise, you get $100”.

LDT decision theories are probably the best decision theories for problems in the fair problem class.

(Very cool that you’ve arrived at the idea of this post independently!)

I don't think I'm the right person to look into this.

I just updated quickly via conservation of expected probability. (I agree though that I'd be a bit concerned about most people updating that quickly. If you think I've gone slightly psychotic please bet with me so I update harder if I notice you're right.)

(EDIT: actually it's sorta shitty because we might not get more evidence because I have even more important things to do and probably don't have time to look into it myself, but i'm happy to bet, though I'd probably want to revise my betting probability.)

I'm happy to bet on "By the end of 2034, does Tsvi think that it's >60% likely that orcas could do superhuman science if they had similar quality and quantity of science education as scientists and were motivated for this, conditional on Tsvi having talked to me for at least 2 hours about this {sometime in 2030-2034}/{when I might have more evidence}?"

I'd currently be at like 30%26% on this, though if you take more time to think about it I might adjust this estimate I am willing to bet on.

I'm happy to bet up to 200$ per bit (or maybe more but would have to think about it). Aka if it's resolved "Yes", money flowing from you to me would be $200\$\cdot \left(log2\right(p_{me})-log2(p_{Tsvi}\left)\right)$ (and if it's resolved "No" it would be $200\$\cdot \left(log2\right(1-p_{me})-log2(1-p_{Tsvi}\left)\right)$ ). (Where negative money flow indicates flow into the other direction.)

(Also obviously you'd need to commit to talking to me for 2h sometime when i have more evidence, and not just avoid resolution by not talking to me.)

I don't know what you mean by this:

The thing is, there's probably gonna be like ten other posts in the reference class of this post, and they just... don't leave much of a dent in things?

• I don't think it makes that much sense to just look at cortical neuron counts. Big bodies ask for many neurons, including cortical motor neurons. Do cetaceans have really big motor cortices? Visual cortices? Olfactory bulbs? Keyword "allometry". Yes, brains are plastic, but that doesn't mean orcas are actually ever doing higher mathematics with their brains.

See this comment.

• Scale matters, but I doubt it's very close to being the only thing! Humans likely had genetic adaptations for neuroanatomical phenotypes selected-for by some of: language; tool-making; persisting transient mental content; intent-inference; intent-sharing; mental simulation; prey prediction; deception; social learning; teaching; niche construction/expansion/migration. Orcas have a few of these. But how many, how much, for how long, in what range of situations and manifestations?

I already considered this. (I just posted a question about this [LW · GW].) I don't have good information on to what extent orcas have those, but my guesses are already reflected in my overall guess in the post. 
Why do you think orcas have few of those? For me it seems plausibe that orcas have everything except tool use and niche construction.

I do think there was some significant selection for some kind of intelligence in dolphins and orcas - the main question here is whether being optimized on tool use (IF that was a significant driver in what selected humans for intelligence) would be significantly more useful for having the brain potential generalize to doing science than if the brains were optimized because of social dynamics or hunting strategies.

But of course there are other considerations like "maybe you need fully recursive language to be able to have the abstract reasoning take off, and this might very well come from some adaptations that are not just about neoron counts, and maybe orcas don't have that".

I already took all my current uncertain consideration on this into account when I said "50% that they would be superhuman at science if they had similar quality and quantity of science education as scientists and were motivated for this".

Or do you think a cow brain scaled to 40 billion neurons would be superhuman?

I don't know what you're asking for here. I don't think organisms end up with 40 billion cortical neurons without either some strong selection for at least some sub-dimensions of intelligence, or being as big as Godzilla.

I'm not really excited about just smushing together more brain tissue without it having been optimized to work well together, but orca brains were optimized.

• Culture matters. The Greeks could be great philosophers... But could a kid living in 8000 BCE, who gets to text message with an advanced alien civilization of kinda dumb people, become a cutting edge philosopher in the alien culture? Even though almost everyone ze interacts with is preagricultural, preliterate? I dunno, maybe? Still seems kinda hard actually?

Yep that's why I'm only at like 15% that we get very significant results out of it in the next 30 years even if we tried hard. (aka 30% conditional on orcas being smart enough.)

• Superbabies is good. It would actually work. It's not actually that hard. There's lots of investment already in component science/tech. Orcas doesn't scale. No one cares about orcas. There's not hundreds of scientists and hundreds of millions in orca communications research. Etc. The sense of this plan being weird is a good sense to investigate further. It's possible for superficial weirdness to be wrong, but don't dismiss the weirdness out of hand.

I mean if Orcas are smarter they might be super vastly smarter so you wouldn't need that many.

Superbabies would work well given multiple generations but also only like 30% that we'd get +7std humans born within 10 years even if we tried similarly hard^[1], and I think it's pretty unlikely we have more than 40 years left without strong governance success. (E.g. afaik we still have problems cloning primates well (even though it's been a thing for long) and those are just sub-difficulties^[2] of e.g. creating superbabies through repeated embryo selection.)

1. ^{^}

   (e.g. 1 billon dollars and a few very smart geniuses going into trying to make communication with orcas work well)

2. ^{^}

   Though I don't know what much and wouldn't shock me if we somehow find a way around this.

I found your example problem very interesting, and started thinking about social dynamics that match "tell me if you do it, but don't do it".

The closest cultural anchor I could find is that of sin, confession, priest. Modelling the situation as a confession might be an apt anchor

When it comes to being up- or down-voted its a real gamble. There are of course the same mechanism at play here just as on other social (media) platforms, i.e., there are certain individuals, trends and thought patterns that are hailed and praised and vice versa without any real justification. But hey, that is what makes us human, these unexplainable things called feelings.

PS perhaps a new hashtag on X would be appropriate #stopthedownvoting

Perhaps if you did have the full solution, but it feels like that there are some things of a solution that you could figure out, such that that part of the solution doesn't tell you as much about the other parts of the solution.

I agree with that.

I don't follow this model yet.  I see why, under this model, a party would want the opponent's candidate to enrage people / have a big blind spot (and how this would keep the extremes on their side engaged), but I don't see why this model would predict that they would want their own candidate to enrage people / have a big blind spot.