LessWrong 2.0 Reader

In Newcombe's scenario, an agent that believes they have a probability of 99.9% of being able to fool Omega should two-box. They're wrong and will only get $1000 instead of $1000000, but that's a cost of having wildly inaccurate beliefs about the world they're in, not a criticism of any particular decision theory.

Setting up a scenario in which the agent has true beliefs about the world isolates the effect of the decision theory for analysis, without mixing in a bunch of extraneous factors. Likewise for the fairness assumption that says that the payoff distribution is correlated only with the agents' strategies and not the process by which they arrive at those strategies.

Violating those assumptions does allow a broader range of scenarios, but doesn't appear to help in the evaluation of decision theories. It's already a difficult enough field of study without throwing in stuff like that.

I created a simple Google Doc for anyone interested in joining/creating a new org to put down their names, contact, what research they're interested in pursuing, and what skills they currently have. Overtime, I think a network can be fostered, where relevant people start forming their own research, and then begin building their own orgs/get funding. https://docs.google.com/document/d/1MdECuhLLq5_lffC45uO17bhI3gqe3OzCqO_59BMMbKE/edit?usp=sharing 

I think antivaxxers could plausibly pose a higher infection risk because they're unusually likely to hang out with other unvaccinated people, or to do other bad decisionmaking. Someone who's unvaccinated because they're scared of needles might still make good decisions otherwise - like they might stay home if they're feeling a sniffle, or test themselves for COVID if their housemate is sick.

Also, you want to exclude unvaccinated people because they pose an infection risk, so you already wanted to exclude anyone who posts "I hate vaccines" on Facebook. You're just worried about the incentives or selection effects if you use an honour system, because some people will lie and say they're vaccinated when they aren't. I'm suggesting that the incentives or selection effects aren't as negative if you only require silence, so nobody has to actually lie. 

I don't understand the focus of this experiment—what is the underlying motivation to understand the reversal curse - in like what alignment concept are you trying to prove or disprove? is this a capabilities check only?  

Additionally, the supervised, labeled approach used for injecting false information doesn't seem to replicate how these AI systems learn data during training. I see this as a flaw in this experiment. I would trust the results of this experiment if you inject the false information with an unsupervised learning approach to mimic the training environment.

Two updates:

• I went and coded a verbosifier, so it's possible to write <msup>a 2</msup> + <msup>b 2</msup> = <msup>c 2</msup> and not <msup><mi>a</mi><mn>2</mn></msup><mo>+</mo>...: https://github.com/jeffkaufman/mathml_verbosifier

• It turns out Feedly (and maybe other RSS readers?) strip out MathML, so even if they're using a rendering engine that supports it you see nothing. So I'll stick to ascii math for a while longer.

After shooting my mouth off I went and tried it, and even programs that I would expect to handle it well (ex: Google Docs) didn't. Sorry!

(I think my statement is still literally true, except for the problem that ~no destination program currently supports it)

i.e., it looks bad.

For what it's worth I think the Linux screenshot is fine -- that's the default font on that system.

I feel like the crux of this discussion is how much we should adjust our behavior to be "less utilitarian", to preserve our utilitarian values.

The expected utility that a person created could be measured by (utility created by behavior) x (odds that they will actually follow through on their behavior), where the odds of follow-up decrease as the behavior modifications become more drastic, but the utility created if followed through increases. 

People are already implicitly taking this account when evaluating what the optimal amount of radicality in activism is. If PETA advocates for everyone to completely renounce animal consumption, conduct violent attacks on factory farms, and aggressively confront non-vegans, that (theoretically) would reduce animal suffering by an extremely large amount. But in practice almost nobody would follow through. On the other hand, if PETA mistakenly centers their activism on calling for people to skip a single chicken dinner, a completely realistic goal that many millions of people would presumably execute, they would also be missing on a lot of expected utility.

Alice is arguing that Bob could maximize expected utility by shifting his behavior to a part of the curve that involves more behavior change, and therefore utility, and less probability of follow-through. Bob is arguing that he's already at the optimal point of the curve. 

It's a priori very unlikely that any post that's clearly made up of English sentences actually does not even try to communicate anything.

My point is that basically, you could have posted this as a comment on the post instead of it being rejected.

Whenever there is room to disagree about what mistakes have been made and how bad those mistakes are, it becomes more of a problem to apply an exclusion rule like this.

There's a lot of questions here: how far along the axis to apply the rule, which axis or axes are being considered, and how harsh the application of the rule actually is.

It should always be smooth gradients, never sudden discontinuities. Smooth gradients allow the person you're applying them to to update. Sudden discontinuities hurt, which they will remember, and if they come back at all they will still remember it.

I think most people actually knows this. When you try to cancel an upvote on mobile, you have to cycle from upvote -> strong upvote -> no vote