I'm not familiar with the details of Robin's beliefs in the past, but it sure seems lately he is entertaining the opposite idea. He's spending a lot of words on cultural drift recently, mostly characterizing it negatively. His most recent on the subject is Betrayed By Culture.

In the past couple of weeks lots of people have been saying the scaling labs have hit the data wall, because of rumors of slowdowns in capabilities improvements. But before that, I was hearing at least some people in those labs saying that they expected to wring another 0.5 - 1 order of magnitude of human-generated training data out of what they had access to, and that still seems very plausible to me

Epoch's analysis from June supports this view, and suggests it may even be a bit too conservative:

(and that's just for text -- there are also other significant sources of data for multimodal models, eg video)

350F

35°F, surely?

I am not following the context of the comment above. Help me understand the connection? The main purpose of my comment above was to disagree with this sentence two levels up:

The frenzy to couple everything into a single tangle of complexity is driven by the misunderstanding that complacency is the only reason why your ideology is not the winning one

… in particular, I don’t think it captures the dominant driver of “coupling” or “bundling”.

Does the comment one level up above disagree with my claims? I’m not following the connection.

According the many-worlds interpretation (MWI) of quantum mechanics, the universe is constantly splitting into a staggeringly large number of decoherent branches containing galaxies, civilizations, and people exactly like you and me

MWI is more than one theory. The version stated above is known to be false.

There is an approach to MWI based on coherent superpositions, and a version based on decoherence. These are (for all practical purposes) incompatible. Coherent splitting gives you the very large numbers of "worlds"..except that they are not worlds, conceptually.

Many worlders are pointing at something in the physics and saying "that's a world"....but whether it qualifies as a world is a separate question , and a separate kind of question, from whether it is really there in the physics. One would expect a world, or universe, to be large, stable, non-interacting, objective and so on . A successful MWI needs to jump three hurdles: mathematical correctness, conceptual correctness, and empirical correctness.

Decoherent branches are expected to be large, stable, non interacting, objective and irreversible...everything that would be intuitively expected of a "world". But there is no empirical evidence for them , nor are they obviously supported by the core mathematics of quantum mechanics, the Schrödinger equation. Coherent superpositions are small scale , down to single particles, observer dependent, reversible, and continue to interact (strictly speaking , interfere) after "splitting".

(Note that Wallace has given up on the objectivity of decoherent branches. That's another indication that MWI is not a single theory).

There isn’t the slightest evidence that irrevocable splitting, splitting into decoherent branches occurs at every microscopic event -- that would be combining the frequency of coherentism style splitting with the finality of decoherent splitting. As well as the conceptual incoherence, there is In fact plenty of evidence—eg. the existence of quantum computing—that it doesnt work that way

“David Deutsch, one of the founders of quantum computing in the 1980s, certainly thinks that it would. Though to be fair, Deutsch thinks the impact would “merely” be psychological – since for him, quantum mechanics has already proved the existence of parallel uni- verses! Deutsch is fond of asking questions like the following: if Shor’s algorithm succeeds in factoring a 3000-digit integer, then where was the number factored? Where did the computational resources needed to factor the number come from, if not from some sort of “multiverse” exponentially bigger than the universe we see? To my mind, Deutsch seems to be tacitly assuming here that factoring is not in BPP – but no matter; for purposes of argument, we can certainly grant him that assumption. It should surprise no one that Deutsch’s views about this are far from universally accepted. Many who agree about the possibil- ity of building quantum computers, and the formalism needed to describe them, nevertheless disagree that the formalism is best inter- preted in terms of “parallel universes.” To Deutsch, these people are simply intellectual wusses – like the churchmen who agreed that the Copernican system was practically useful, so long as one remembers that obviously the Earth doesn’t really go around the sun. So, how do the intellectual wusses respond to the charges? For one thing, they point out that viewing a quantum computer in terms of “parallel universes” raises serious difficulties of its own. In particular, there’s what those condemned to worry about such things call the “preferred basis problem.” The problem is basically this: how do we define a “split” between one parallel universe and another? There are infinitely many ways you could imagine slic- ing up a quantum state, and it’s not clear why one is better than another! One can push the argument further. The key thing that quan- tum computers rely on for speedups – indeed, the thing that makes quantum mechanics different from classical probability theory in the first place – is interference between positive and negative amplitudes. But to whatever extent different “branches” of the multiverse can usefully interfere for quantum computing, to that extent they don’t seem like separate branches at all! I mean, the whole point of inter- ference is to mix branches together so that they lose their individual identities. If they retain their identities, then for exactly that reason we don’t see interference. Of course, a many-worlder could respond that, in order to lose their separate identities by interfering with each other, the branches had to be there in the first place! And the argument could go on (indeed, has gone on) for quite a while. Rather than take sides in this fraught, fascinating, but perhaps ultimately meaningless debate...”..Scott Aaronson , QCSD, p148

Also see

https://www.lesswrong.com/posts/wvGqjZEZoYnsS5xfn/any-evidence-or-reason-to-expect-a-multiverse-everett?commentId=o6RzrFRCiE5kr3xD4 [LW(p) · GW(p)]

I’m not going to argue for this view as that was done very well by Eliezer in his Quantum Physics sequence. An

It's riddled.with errors which have been pointed out many times.

[@dr_s](/users/dr_s?mention=user)[@dr_s](/users/dr_s?mention=user)

I feel like branches being in fact an uncountable continuum is essentially a given

Decohereht branches being being countable, uncountable, or anything else is not given, since there is no established theory of of decoherence.

Enough of the physics confusions -- onto the ethics confusions.

There's broadly three areas where MWI has ethical implications. One is concerned with determinism, freedom of choice, and moral responsibility. One is over the fact that MW means low probability events have to happen every time -- as opposed to single universe physics, where they usually don't. The other is over whether they are discounted in moral significance for being low in quantum mechanical measure or probability

MWI and Free Will

MWI allows probabilities of world states to change over time, but doesn't allow them to be changed, in a sense amounting to libertarian free will. Agents are just part of the universal wave function, not anything outside the system, or operating by different rules.MWI is, as it's proponents claim, a deterministic theory, and it only differs from single world determinism in that possible actions can't be refrained from, and possible futures can't be avoided. Alternative possibilities are realities, in other words.

MWI, Moral Responsibility, and Refraining.

A standard argument holds that causal determinism excludes libertarian free will by removing alternative possibilities. Without alternative possibilities, you could but have done other than you did, and , the argument goes, you cannot be held responsible for what you had no choice but to do.

Many world strongly implies that you make all possible decisions: according to David Deutsch's argument that means it allows alternative possibilities, and so removes the objection from moral responsibility despite being a basically deterministic theory.

However, deontology assumed that performing a required act involves restraining from alternatives.. and that it is possible to retain from forbidden acts. Neither is possible under many worlds. Many worlds creates the possibility, indeed the necessity, of doing otherwise, but removes the possibility of refraining from an act. Even though many worlds allows Alternative Possibilities, unfortunately for Deutschs argument, that other aspects create a similar objection on the basis of moral responsibility: why would you hold someone morally responsible for an act if they could not refrain from it?

MWI, Probability, and Utilitarian Ethics.

The point of the Born rule is to get a set of ordinary probabilities, which you can then test frequentistically, over a run of experiments. Quantum mechanical measure—amplitude—isn’t ordinary probability, but that’s the thing you put into the Born rule, not the thing you get out of it. And it has it’s own role, which explaining how much contribution to a coherent superposition each component state makes.

There is a further problem interpreting the probabilities of fully decohered branches. (Calling then Everett branches is very misleading—a clear theory of decoherence is precisely what’s lacking in Everett’s work) Whether you are supposed to care about them ethically is very unclear, since it is not clear how utilitarian style ethics would apply, even if you could make sense of the probabilities. But you are not supposed to care about them for the purposes of doing science, since they can no longer make any difference to your branch. MWI works like a collapse theory in practice. MWI creates the puzzle that low probability outcomes still happen, and have to be take. Into account ethically. Many rationalists assume that they simply matter less, because that is the only way to e store anything g like a normal view of ethical action -- but one should.not assume something merely because it is convenient.

It can be argued that probability calculations come out the same under different interpretations of QM...but ethics is different. The difference stems from the fact that what other people experience is relevant to them, wheareas for a probability calculation, I only need to be able to statistically predict my own observations. Using QM to predict my own observations, I can ignore the question of whether something has a ten percent chance of happening in the one and only world, or a certainty of happening in one tenth of possible worlds. However, these are not necessarily.equivalent ethically.

It is not clear how low measure worlds should be considered in utilitarian ethics. It's tempting to ethically discount low measure worlds in some way, because that most closely approximates conventional single world utilitarianism. The alternative might force one to the conclusion that overall good outcomes are imposed to attain , so long as one cannot reduced the measures of worlds full of suffering zero. However, one should not jump to the conclusion that something is true just because it is convenient. And of course, MWI is a scientific theory so it doesn't comes with built in ethics.

The alternative view starts with the question of whether a person low measure world still.count as a full.person?.If they should not, is that because they are a. near-zombie, with a faint consciousness that weighs little in a hedonic utilitarian calculus? If they are not such zombies, why would they not count as a full person -- the standard utilitarian argument that people in far-off lands are still moral patients seems to apply. Of course, MWI doesn't directly answer the question about consciousness.

These are not questions that can be answered scientifically.

You can have objective information about observations, and if your probability calculus is wrong , you will get wrong results and know that you are getting wrong results. That is the negative feedback that allows physics to be less wrong.

You can have subjective information about your own mental states, and if your personal calculus is wrong , you will get wrong results and know that you are getting wrong results. That is the negative feedback that allows personal decision theory to be less wrong.

Altruistic ethics is different. You don't have either kind of direct evidence, because you are concerned with other people's subjective sensations , not objective evidence , or your own subjectivity. Questions about ethics are downstream of questions about qualia, and qualia are subjective, and because they are subjective, there is no reason to expect them to behave like third person observations.

We have to infer that someone else is suffering , and how much, using background assumptions. For instance, I assume that if you hit your thumb with a hammer , it hurts you like it hurts me when I hit my thumb with a hammer.

MWI, Measure and Morality.

One can have a set of ethical axioms saying that I should avoid causing death and suffering to others, but to apply them under many worlds assumptions, I need to be able to calculate how much death and suffering my choices cause in relation to the measure. Which means I need to know whether the measure or probability of a world makes a difference to the intensity of subjective experience.. including the option of "not at all", and I need to know whether the deaths of then people in a one tenth measure world count as ten deaths or one death.

Suppose they are discounted.

If people in low measure worlds experience their suffering fully, then a 1%, of creating a hell-world would be equivalent in suffering to a 100% chance. But if people in low measure worlds are like philosophical zombies, with little or no phenomenal consciousness, so that their sensations are faint or nonexistent, the moral hazard is much lower.

A similar, but slightly less obvious argument applies to causing death. Causing the "death" of a complete zombie is presumably as morally culpable as causing the death of a character in a video game...which, by common consent, is not problem at all. So... causing the death of a 50% zombie would be only half as bad as killing a real person...maybe.

There is an alternative way of cashing out quantum mechanical measure due to David Deutsch. He supposes that you can have exact duplicates of worlds , which form sets of identical worlds, and which have no measure of their own. Each set contains different worlds, making the sets unique, each world within a set is identical to the others. Counting the worlds in an equivalence set gives you the equivalent of measure.

Under this interpretation, you should ethically discount low-measure worlds (world sets) because there are fewer people in them.

The approach where suffering and moral worth are discounted according to measure has the convenient property that it works like probability theory, so that you don't have to take an interpretation of QM into account. But , if you are arguing to the conclusion that the interpretation of QM is ethically neutral, you can't inject the same claim as an assumption, without incurring circularity.

What's needed is a way of deciding the issue of measure versus reality versus consciousness versus ethical value that is not question begging. We are not going to get a rigourous theory based on physics alone, because physics does not explicitly deal with reality, consciousness or ethical value. But there are some hints. One important consideration is that under many world's theory, all worlds must have measures summing to 1, so any individual world,including ours, has measure less than 1. But our world and our consciousness seem fully real to us; if there is nothing special about our world, then the inhabitants of other worlds presumably feel the same. And that is just the premise that leads to the conclusion that many world's theory does impact ethics.

Test

I don't think agents that avoid the money pump for cyclicity are representable as satisfying VNM, at leasting holding fixed the objects of preference (as we should). Resolute choosers with cyclic preferences will reliably choose B over A- at node 3, but they'll reliably choose A- over B if choosing between these options ex nihilo. That's not VNM representable, because it requires that the utility of A- be greater than the utility of B and. that the utility of B be greater than the utility of A-

Personally, I don’t care about the shrimp. At all. The anthropomorphising is absurd, and saying “but even if the suffering were much less it would still be huge” is either a basic error of thinking or dark arts. Anchor the reader by picking a huge number, then back off and point out that it’s still huge. How about epsilon? Is that still huge? How about zero? Anyone can pluck figures out of the air, dictated by whatever will support their bottom line.

I see that already one person has let this article mug his brain for $1000. His loss, though he think it gain.

It also makes it behaviorally indistinguishable from an agent with complete preferences, as far as I can tell.

That's not right. As I say in another comment:

And an agent abiding by the Caprice Rule can’t be represented as maximising utility, because its preferences are incomplete. In cases where the available trades aren’t arranged in some way that constitutes a money-pump, the agent can prefer (/reliably choose) A+ over A, and yet lack any preference between (/stochastically choose between) A+ and B, and lack any preference between (/stochastically choose between) A and B. Those patterns of preference/behaviour are allowed by the Caprice Rule.

Or consider another example. The agent trades A for B, then B for A, then declines to trade A for B+. That's compatible with the Caprice rule, but not with complete preferences.

Or consider the pattern of behaviour that (I elsewhere argue [LW · GW]) can make agents with incomplete preferences shutdownable. Agents abiding by the Caprice rule can refuse to pay costs to shift probability mass between A and B, and refuse to pay costs to shift probability mass between A and B+. Agents with complete preferences can't do that.

The same updatelessness trick seems to apply to all money pump arguments.

[I'm going to use the phrase 'resolute choice' rather than 'updatelessness.' That seems like a more informative and less misleading description of the relevant phenomenon: making a plan and sticking to it. You can stick to a plan even if you update your beliefs. Also, in the posts on UDT, 'updatelessness' seems to refer to something importantly distinct from just making a plan and sticking to it.]

That's right, but the drawbacks of resolute choice depend on the money pump to which you apply it. As Gustafsson notes, if an agent uses resolute choice to avoid the money pump for cyclic preferences, that agent has to choose against their strict preferences at some point. For example, they have to choose B at node 3 in the money pump below, even though - were they facing that choice ex nihilo - they'd prefer to choose A-.

There's no such drawback for agents with incomplete preferences using resolute choice. As I note in this post [LW · GW], agents with incomplete preferences using resolute choice need never choose against their strict preferences. The agent's past plan only has to serve as a tiebreaker: forcing a particular choice between options between which they'd otherwise lack a preference. For example, they have to choose B at node 2 in the money pump below. Were they facing that choice ex nihilo, they'd lack a preference between B and A-.

personality( ground_truth ) --> stated_truth

Or, what someone says is a subset of what they're really thinking, transformed by their personality.
(personality plus intentions, context, emotions, etc.)

(You can tell I'm not a mathematician because I didn't express this in latex xD  but I feel like there's a very elegant Linear Algebra description where, ground truth is a high dimensional object, personality transforms to make it low-dimensional enough to communicate, and changes its "angle" (?) / vibe so it fits their goals better)

So, if you know someone's personality, meaning, what kind of way they reshape a given ground truth, you can infer things about the ground truth.

Anecdote

I feel like everyone does this implicitly, it's one of those social skills I struggle with. I always thought it was a flaw of other people, to not simply take things at face value, but to think at the level of "why do they say this", "this is just because they want <x>", while I'm always thinking about "no but what did they literally say? is it true or not?".

It's frustrating because usually, the underlying social/personality context completely dominates what someone says, like, 95% of the time it literally doesn't fucking matter one bit what someone is saying at the object level. And maybe, the fact that humans have evolved to think in this way, is evidence that in fact, 95% of what people say doesn't matter and shouldn't matter object-level, that the only thing that matters is implications about underlying personality and intentions.

My recollection of sitting through "adult conversations" of family and relative visits for probably many hundreds of hours is, "I remember almost zero things that anybody has ever talked about". I guess the point is that they're talking, not what it's about.

Practical

This reframing makes it easier to handle social situations. I can ask myself questions like, "what is the point of this social interaction?", "why is this person saying this to me?"

Often the answer is, "they just wanna vibe, and this is an excuse to do so".

the untrusted model is stateless between queries and only sees the command history and system state.

What justifies the assumption that untrusted models will be stateless in deployment? There are clear performance advantages in many cases to giving models memory, and in practice they already typically receive context containing the history of their actions or interactions.

If you're assuming that commercial labs will stop doing that for the sake of safety despite its advantages, it seems worth making that assumption explicit.