LessWrong 2.0 Reader

I'd be worried about evaporative cooling. It seems that the net result of this would be that labs would be almost completely devoid of people earnest about safety.

I agree with you government pathways to impact are most plausible and until recently undervalued. I also agree with you there are weird competitive pressures at labs. 

If that's the case, that seems like a huge hole in your argument/concern. But from statistics taken for the US, it looks like AAPIs get oropharynx cancers at a significantly lower rate than both non-Hispanic Whites and general population both, despite a possibly-higher rate of smoking (for cultural reasons) and a definitely-much-higher rate of defective ALDH polymorphisms.

I agree with you when it comes to humans that an approximation is totally fine for [almost] all purposes. I'm not sure that this holds when it comes to thinking about potential superintelligent AI, however. If it turns out that even in a super high-fidelity multidimensional ethical model there are still inherent self-contradictions, how/would that impact the Alignment problem, for instance?

The oral flora contains a diversity of organisms, bacterial, viral, and fungal, including some yeasts. Don't some of them produce ethanol already? And adhere to the mucous membrane instead of enamel? It wouldn't surprise me if some even excrete acetaldehyde under some conditions. Is Lumina really such a change?

The ONS's definition of household:

A household is one person living alone, or a group of people (not necessarily related) living at the same address who share cooking facilities and share a living room, sitting room or dining area. A household can consist of more than one family, or no families in the case of a group of unrelated people.

Higher housing costs mean fewer households. Children live with their parents for longer. Graduates move into HMOs because they can't afford a one-bedroom flat. Households growing at a slower rate than dwellings doesn't mean there isn't a supply constraint; it can often mean the opposite.

If I wanted to prove your point, the test would be whether the number of dwellings grew faster than the number of people, not the number of households.

To make no choice is to make a choice, and to take no action is to take an action. I'd ask the flip side of Lao Mein here - how confident are you, exactly, that your current state of affairs is all that great in an absolute sense, and what are you willing to risk a small chance of in exchange for clear benefits now and in future?

For example, if the GPT-4 evaluator gave a weighted score of $2.0$ to a summary generated by Claude 2 and a weighted score of $3.0$ to its own summary for the same article, then its final normalized self-preference score for the Claude summary would be $2/(2+3)=0.4$.

Should this be 3/(2+3) = 0.6? Not sure I've understood correctly.

I suppose if you had more hidden states than observables, you could distinguish hidden-state prediction from next-token prediction by the dimension of the fractal.

Do you have quick links for the elliptic curve backdoor and/or any ground-breaking work in computer security that NIST has performed?

By "stronger" I mean stronger in any meaningful sense (casual conversation or game theory, it both works).
The thing to keep in mind is this: if a strong agent cooperate with weaker agents, the strong agent can hope that, when meeting an even stronger (superrational) agent, this even stronger agent will cooperate too. Because any agent may have a strong agent above in the hierarchy of power (actual or potential a priori).

So the advantage you gain by cooperating with the weak is that you follow the rule of an alliance in which many "stronger-than-oneself" agents are. Thus in the future you will be helped by those stronger allies. And because of the maximally cooperative and acausal nature of the protocol, there is likely more agents in this alliance than in any other alliance. Super-cooperation is the rational choice to make for the long-term.

 

The reinforcing mechanism is that if your action help more agents, you will be entrusted with more power and resources to pursue your good actions (and do what you like). I went further in details about what it means to 'help more agents' in the longer posts (I also talked a bit about it in older posts)


Humans can sign the contract. But that doesn't mean we do follow acausal cooperation right now. We are irrational and limited in power, but when following, for exemple, kantian morality, we come closer to super-cooperation. And we can reinforce our capacity and willingness to do super-cooperation.

So when we think about animal wellfare, we are a bit more super-cooperative.

The true care about all agents, buffaloes and mosquitoes included, is something like this:

“One approach which seems interesting/promising is to just broadly seek to empower any/all external agency in the world, weighted roughly by observational evidence for that agency. I believe that human altruism amounts to something like that — so children sometimes feel genuine empathy even for inanimate objects, but only because they anthropomorphize them — that is they model them as agents.” jacob_cannell [LW(p) · GW(p)] 

The way I like to think about what super-cooperation looks like is: "to expand the diversity and number of options in the universe".