LessWrong 2.0 Reader

I expect (1) and (2) to go mostly fine (though see footnote^[6]), and for (3) to completely fail. For example, suppose $C_b$ has a bunch of features for things like “true according to smart humans.” How will we distinguish those from features for “true”? I think there’s a good chance that this approach will reduce the problem of discriminating $C_g$ vs. $C_b$ to an equally-difficult problem of discriminating desired vs. undesired features.

However, if we found that the classifier was using a feature for "How smart is the human asking the question?" to decide what answer to give (as opposed to how to then phrase it), that would be a red flag.

Why there isn't more ground-up interest: it's expensive and people can't easily tell if it's worth the cost. Also anything where UV is touching you has to overcome people's safety concerns.

Good question on the large effects are easily measured thing--has to do with the distinction between: 1) in what environments you are cleaning the air, 2) how much you are cleaning the air there, 3) how much pathogen people inhale, and 4) how much pathogen is required to actually make people sick. It's not just a far-UV problem, it'd be a problem for any air cleaner, it's just that far-UV is especially expensive to install and especially faces negative "UV" associations.

Far-UV has a large effect on airborne pathogen concentration, and that large effect is in fact easy to measure in a chamber! But once you add it to a room where people are moving around and talking to each other, how much pathogen are they actually inhaling? Is the air in the room well-mixed? Is the far-UV reaching the infectious air before people inhale it? Even if they inhale air that has living pathogens, are they getting sick? If they get sick, did they get sick from that room or from a different environment that they were in previously? Study endpoints matter a lot. 

Being able to understand intervention efficacy especially becomes a problem if a disease is largely spread via superspreaders/had high variance in infection sources. COVID, at least early in the pandemic, had very high variance, whereas eg seasonal flu doesn't usually. Therefore, if your study intervention is installed in public spaces, it's possible for it not to show much effect on seasonal flu but a large effect on a disease with early-COVID-like dynamics, which means you have to wait for that COVID-like disease to come along to see the effect--but that would be worth it; high-variance diseases are very concerning! 

Another way of saying all this is that it's not the case that the effect will be super hard to measure given enough people and time, it's that the effect is hard to measure given that you need a lot of people in your study to account for the distinctions listed above, and/or you need a highly controlled environment, and that's just expensive.

We know well enough what people mean by "world" - the stuff they care about. The fact that physics keeps on happening if humanity is snuffed out is no comfort at all to me or to most humans.

Arguing epistemology is not going to prevent a nuclear apocalypse or us being wiped out by the new intelligent species we are inventing. The fact that you don't know what's happening on the other side of the world has no bearing on existential dangers facing those people. That's what I mean by saving the world, and I expect what the author meant. This is a different thing than just helping people by your own values and estimates.

I very much agree that pithy mysterious statements for others to argue over is not a good use of the quick takes here.

Is the complaint that you can't do predicate calculus on the probabilities? Because I can certainly use predicate calculus all I want on the expressions within the probabilities.

And if that is the complaint, then my question is: why do we want to do predicate calculus on the probabilities? Like, what would be one concrete application in which we'd want to do that? (Self-reference and things in that cluster would be the obvious use-case, I'm mostly curious if there's any other use-case.)

I have no interest in honor if it's celebrated on a field of the dead. Virtue ethics is fine, as long as it's not an excuse to not figure out what needs doing and how it's going to get done.

Doing ones own part and trusting that the other parts are done by anonymous unknown others is a very silly coordination strategy. We need plans that amount to success, not just everyone doing whatever sounds nice to them.

Edit: I very much agree that saving the world is a team sport. Perhaps it's relevant that successful teams always do some planning and coordinating.

the thing a logician would call a "logic" or possibly a "logic augmented with some probabilities"

The main point of the article is that once you add probabilities you can't do predicate calculus anymore. It's a mathematical operation that's not defined for the entities that you get when you do your augmentation.

Interesting. There's certainly a lot going on in there, and some of it very likely is at least vague models of future word occurrences (and corresponding events). The definition of model-based gets pretty murky outside of classic RL, so it's probably best to just directly discuss what model properties give rise to what behavior, e.g. optimizing for reward.

Model-free systems can produce goal-directed behavior. The do this if they have seen some relevant behavior that achieves a given goal, and their input or some internal representation includes the current goal, and they can generalize well enough to apply what they've experienced to the current context. (This is by the neuroscience definition of habitual vs goal-directed: behavior changes to follow the current goal, usually hungry, thirsty or not).

So if they're strong enough generalizers, I think even a model-free system actually optimizes for reward.

I think the claim should be stronger: for a smart enough RL system, reward is the optimization target.

So you think that looking up "random idiots" helps me find "arguments related to or a more detailed discussion about this disagreement"?

You want to help? Figure out what kind of incremental changes you can begin to introduce in any of them, in order to begin extinguishing the sort of problems you've now elevated to the rank of "saving-worthy" in your own head. Note that, in all likelihood, by extinguishing one you will merrily introduce a whole bunch of others - something you won't get to discover until much later one. Yet that is, realistically, what you can actually go on to accomplish.

I read this paragraph as saying ~the same thing as the original post in a different tone

It is true that there are some favorable properties that many systems other than the best system has compared to FPTP.

I like methods that are cloneproof and which can't be spoofed by irrelevant alternatives, and if there is ONLY a choice between "something mediocre" and "something mediocre with one less negative feature" then I guess I'll be in favor of hill climbing since "some mysterious force" somehow prevents "us" from doing the best thing.

However, I think cloning and independence are "nice to haves" whereas the condorcet criterion is probably a "need to have"

((The biggest design fear I have is actually the "participation criterion". One of the very very few virtues of FPTP is that it at least satisfies the criterion where someone showing up and "wasting their vote on a third party" doesn't cause their least preferred candidate to jump ahead of a more preferred candidate. But something similar can happen in every method I know of that reliably selects the Condorcet Winner when one exists :-(

Mathematically, I've begun to worry that maybe I should try to prove that Condorcet and Participation simply cannot both be satisfied at the same time?

Pragmatically, I'm not sure what it looks like to "attack people's will to vote" (or troll sad people into voting in ways that harm their interests and have the sad people fight back righteously by insisting that they shouldn't vote, because voting really will net harm their interests).

One can hope that people will simply "want to vote" because it make civic sense, but it actually looks like a huge number of humans are biased to feel like a peasant, and to have a desire to be ruled? Or something? And maybe you can just make it "against the law to not vote" (like in Australia) but maybe that won't solve the problems that could hypothetically "sociologically arise" from losing the participation criterion in ways that might be hard to foresee.))

In general, I think people should advocate for the BEST thing. The BEST thing I currently know of for picking an elected civilian commander in chief is "Ranked Pairs tabulation over Preference Ballots (with a law that requires everyone to vote during the two day Voting Holiday)".