Posts

(answered: yes) Has anyone written up a consideration of Downs's "Paradox of Voting" from the perspective of MIRI-ish decision theories (UDT, FDT, or even just EDT)? 2020-07-06T18:26:01.933Z · score: 10 (5 votes)
[ongoing] Thoughts on Proportional voting methods 2020-07-03T06:05:31.055Z · score: 30 (7 votes)
Is voting theory important? An attempt to check my bias. 2019-02-17T23:45:57.960Z · score: 42 (14 votes)
EA grants available (to individuals) 2019-02-07T15:17:38.921Z · score: 35 (12 votes)
Does the EA community do "basic science" grants? How do I get one? 2019-02-06T18:10:00.827Z · score: 8 (3 votes)
A Rationalist Argument for Voting 2018-06-07T17:05:42.668Z · score: 13 (8 votes)
The Devil's Advocate: when is it right to be wrong? 2018-05-15T17:12:16.681Z · score: 16 (3 votes)
Upvotes, knowledge, stocks, and flows 2018-05-10T14:18:15.087Z · score: 44 (10 votes)
Multi-winner Voting: a question of Alignment 2018-04-17T18:51:09.062Z · score: 65 (17 votes)
5 general voting pathologies: lesser names of Moloch 2018-04-13T18:38:41.279Z · score: 107 (35 votes)
A voting theory primer for rationalists 2018-04-12T15:15:23.823Z · score: 243 (80 votes)

Comments

Comment by jameson-quinn on [ongoing] Thoughts on Proportional voting methods · 2020-07-15T03:27:10.764Z · score: 4 (2 votes) · LW · GW

Beginning "a proposed solution" but it's slow going — haven't actually gotten to the solution part of the solution yet. Also wrote a bit more above on what "MSE" means in this context — it's less intiutive than you might think when you first read the acronym.

Comment by jameson-quinn on (answered: yes) Has anyone written up a consideration of Downs's "Paradox of Voting" from the perspective of MIRI-ish decision theories (UDT, FDT, or even just EDT)? · 2020-07-14T11:55:19.735Z · score: 4 (2 votes) · LW · GW

Consider a 2-option election, with 2N voters, each of whom has probability p of choosing the first option. If p is a fixed number, then as N goes to infinity, (chances of an exact tie times N) go to 0 if N isn't exactly .5, and to infinity if it is. Since the event of p is exactly .5 has measure 0, this model supports the paradox of voting (PoV).

But! If p itself is drawn from an ordinary continuous distribution with nonzero probability density d around .5, then (chances of an exact tie times N) go to ... I think it's just d/2. Maybe there's some correction factor that comes into play for bizarre distributions of p, but if we make the conventional assumption that it's beta-distributed, then d/2 is the answer.

I think that the PoV literature is relying on the "fixed p" model. I think the "uncertain p" model is more realistic, but it's still worth engaging with "fixed p" and seeing the implications of those assumptions.

Comment by jameson-quinn on (answered: yes) Has anyone written up a consideration of Downs's "Paradox of Voting" from the perspective of MIRI-ish decision theories (UDT, FDT, or even just EDT)? · 2020-07-07T14:07:17.729Z · score: 2 (1 votes) · LW · GW

That book is from 2006. I understand that it deals with the Paradox of Voting, but does it have anything that would be directly relevant to considering it in light of "acausal decision theories"? As far as I know, such theories pretty much didn't exist back then.

Comment by jameson-quinn on (answered: yes) Has anyone written up a consideration of Downs's "Paradox of Voting" from the perspective of MIRI-ish decision theories (UDT, FDT, or even just EDT)? · 2020-07-07T13:46:15.009Z · score: 6 (3 votes) · LW · GW

Wait, what?

"You play the game with many other CDT agents" — this seems demonstrably false, at least, if we accept the Paradox of Voting as being a thing, in which case, CDT agents have by assumption removed themselves from the game. (I understand your response that voting may be altruistically-CDT-rational; as you know, it's been discussed before, and very rightly so. But I also think it's still worth considering the boundedly-altruistic/diagonally-dominant case.)

It seems to me that the only way you can claim there's "many other CDT agents" is if "CDT" is being used as a catch-all for "not explicitly FDT/UDT", and I'd strongly dispute that usage. I think that memetically/genetically evolved heuristics are likely to differ systematically from CDT. It may be best to create an entirely separate model for people operating under such heuristics, but if you want to force them into a pure CDT-vs-UDT-vs-random-noise (ie, mixture distribution) paradigm, I'd say they would be substantially more than 0% UDT.

ETA: I guess I can parse "other voters are CDT" as a sensible assumption if you're explicitly doing repeated-game analysis, but such an analysis would pretty much dissolve both the Paradox of Voting and the CDT vs. acausal-DTs distinction.

Comment by jameson-quinn on (answered: yes) Has anyone written up a consideration of Downs's "Paradox of Voting" from the perspective of MIRI-ish decision theories (UDT, FDT, or even just EDT)? · 2020-07-06T23:12:22.648Z · score: 2 (1 votes) · LW · GW

I agree that this definition of "matters" is odd; not the one most people use in everyday speech. I think that there are ways to make other definitions rigorous (in ways that aren't addressed in the wikipedia article I linked). But this is the narrowly consequentialist definition, so it does deserve analysis.

Comment by jameson-quinn on Classifying games like the Prisoner's Dilemma · 2020-07-06T21:27:56.249Z · score: 4 (2 votes) · LW · GW

Nicely done.

In my experience, "Farmer's Dilemma" (aka Chicken/Snowdrift) is both more common than PD, and harder for human players to coordinate away from pathological outcomes. I think it should be the prototypical "nasty" game instead of PD. We've all been assigned a group project at school; we have not all been interrogated by the police in a situation where the payoffs are truly PD.

Comment by jameson-quinn on (answered: yes) Has anyone written up a consideration of Downs's "Paradox of Voting" from the perspective of MIRI-ish decision theories (UDT, FDT, or even just EDT)? · 2020-07-06T21:03:16.371Z · score: 3 (2 votes) · LW · GW

I very much do not include superrationality in my assumptions. I'm not assuming that all other voters, or even any specific individual other voter, is explicitly using a meta-rational decision theory; I'm simply allowing the possibility that the "expected acausal impact" of my decision is greater than 0 other voters. There are, I believe, a number of ways this could be "true".

In simpler terms: I think that my beliefs (and definitions) about whether (how many) other voters are "like" me are orders of magnitude different from yours, in a way that is probably not empirically resolvable. I understand that taking your point of view as a given would make my original question relatively trivial, but I hope you understand that it is still an interesting question from my point of view, and that exploring it in that sense might even lead to productive insights that generalize over to your point of view (even though we'd probably still disagree about voting).

If you like, I guess, we could discuss this in a hypothetical world with a substantial number of superrational voters. For you this would merely be a hypothetical, which I think would be interesting for its own sake. For me, this would be a hypothetical special case of acausal links between voters, links which I believe do exist though not in that specific form.

Comment by jameson-quinn on What are your thoughts on rational wiki · 2020-07-06T20:53:01.693Z · score: 2 (1 votes) · LW · GW

I think that the ideal community size (the regime at which positive network effects clearly dominate negative effects) is much larger for a wiki than for a forum like here. Thus, though I don't have experience with RationalWiki, my prior would be to be skeptical of its value.

Comment by jameson-quinn on (answered: yes) Has anyone written up a consideration of Downs's "Paradox of Voting" from the perspective of MIRI-ish decision theories (UDT, FDT, or even just EDT)? · 2020-07-06T20:44:16.990Z · score: 2 (1 votes) · LW · GW

But there is an obvious acausal path in this case. If other voters are using the same algorithm you are to decide whether or not to vote, or a "sufficiently similar" one (in some sense that would have to be fleshed out), then that inflates the probability that "your" decision of whether or not to vote is pivotal, because "you" are effectively multiple voters.

Is that sufficient, or do you need actual numbers? (I'd guess it is and you don't.)

Comment by jameson-quinn on [ongoing] Thoughts on Proportional voting methods · 2020-07-06T17:55:53.041Z · score: 2 (1 votes) · LW · GW

Thanks, fixed. (A numerator of 1 gives an answer in dimensionless units of "fraction of voters"; using V gives units of voters. I tend to prefer the former but I agree that at first read the latter is more intuitive.)

Comment by jameson-quinn on [ongoing] Thoughts on Proportional voting methods · 2020-07-06T17:54:31.300Z · score: 2 (1 votes) · LW · GW

Fleshing out "allocating responsibility" and beginning "Pascal's other wager".

Comment by jameson-quinn on The silence is deafening – Devon Zuegel · 2020-07-04T02:54:49.287Z · score: 11 (6 votes) · LW · GW

What about trolls? What about pile-ons?

Trolls: some people are not upset by negative feedback or even actively seek it. I think this could be structured such that this negative feedback would not be rewarding to such people, but it merits consideration, because backfire is at least in principle possible.

Pile-ons: There are documented cases of organized downvote brigades on various platforms, who effectively suppress speech simply because they disagree with it. Now, I wouldn't object to a brigade of mathematicians on a proof wiki downvoting and pages they disagreed with and thereby censoring the pages or driving away their authors; but in most other cases, I think such brigades would be a problem. Again, you might be able to design a version that successfully discouraged such brigades (for instance: have "number of downvotes", "correlation with average downvoter", and "correlation with most-similar downvoter" all visible in someone's profile?), but it merits thought.

Comment by jameson-quinn on [ongoing] Thoughts on Proportional voting methods · 2020-07-04T02:43:16.632Z · score: 3 (2 votes) · LW · GW

Began the "allocating responsibility" section.

Comment by jameson-quinn on [ongoing] Thoughts on Proportional voting methods · 2020-07-04T01:31:34.605Z · score: 2 (1 votes) · LW · GW

I've switched over to responding to your comments/suggestions one-by-one.

On your point 1, regarding loss functions: I agree that a strictly utilitarian/consequentialist PoV would care about which side won, not about vote totals. I think there are three reasons to nevertheless build a loss function around vote totals.

1. Mathematically more well-behaved. For instance, the whole "MSE decomposition" idea I bring in later would be much much messier with a binary-outcome-based loss function.

2. I believe that in practice, if there's a question where the target support would be, say, 70%, but the legislature supports it at 90%, you can probably use it to construct some other at-least-somewhat-reasonable question where the target support would be 40% but the legislature supports it at 60%. That is, in practice, errors in vote totals go hand-in-hand with errors in outcomes, even if this is not a logical necessity (at least, not without additional assumptions about convex lotteries and stuff).

3. Some of the votes/decisions of the legislature may be made in a non-majority-rules fashion. For instance, you could have some situations where each legislator gets to allocate a share of some resource. In such cases, the vote-total-based loss function is clearly correct even from a consequentialist standpoint. (This might be seen as a special case of 2, but it's different enough to list separately.)

I think that saying 1 and 2 in the main article would be too much of a digression, but I will think further about whether there's a way to include point 3.

Comment by jameson-quinn on Second Wave Covid Deaths? · 2020-07-04T01:16:55.135Z · score: 2 (1 votes) · LW · GW

When you train on old data, you get a lag of about 10-12 days between changes in cases and corresponding changes in deaths. There are several reasons that could be not true on new data:

1. Cases are getting caught earlier by more/faster testing.

2. Cases are leading to fewer or slower deaths (due to either treatment or population effects)

3. The lag on old data is using the reported date of death, but that's not the same as the date of the reporting of the death, which has an additional lag.

Are you saying it's (at least partly) #3?

Comment by jameson-quinn on Editor Mini-Guide · 2020-07-04T00:50:07.055Z · score: 2 (1 votes) · LW · GW

This may not be the right place for this. But gotta say it somewhere.

Overall, I like this font. But I hate hate HATE that it makes it impossible to distinguish "1" (one) from "I" (capital-i), except in the amount of horizontal space they take up. The numeral one should have a hook at the top, even in a sans-serif font. This is really important if you're going to be having mathematical discussions. (Of course, "l" (lower-case L) is also indistinguishable here. But letters are either clear from context or, if used as variable names, a choice. So two indistinguishable letters is no big deal; a numeral and a letter is a PROBLEM.)

Comment by jameson-quinn on [ongoing] Thoughts on Proportional voting methods · 2020-07-03T17:52:07.296Z · score: 2 (1 votes) · LW · GW

I meant "𝓮" to be the number whose natural logarithm is 1 — "𝓮" is the closest to the math version of the character that I can type on my keyboard as-is. I don't have any argument for why it should be precisely that value, it's just the most famous number in between 2.5 and 3, which is roughly where I think the correct value will lie. I am aware of the larger literature on people's political preferences, but that's all from a world that's already conditioned by more-or-less non-ideal voting methods, so I don't think this is a question with an easy-to-find empirical answer like that. I may expand the section you reference to talk about these issues a bit more, and to discuss the available data/literature, but when I first wrote it, I was hurrying through to get somewhere else.

Comment by jameson-quinn on [ongoing] Thoughts on Proportional voting methods · 2020-07-03T17:51:47.722Z · score: 2 (1 votes) · LW · GW

Thanks for the close read and the thoughtful comments!

Open:

1. Loss function, future-oriented: interesting questions; I have thoughts on them; I left them out the first time through but may add them later. 

2. gerrymandering; "infinite"; proposal I don't fully grasp: Also interesting but I think beside the points I want to make here.

3. Dimensions.

"The choice of dimensions seems like a bigger deal than the number." I'm implicitly assuming that they're chosen by principal component analysis or something similar. Of course that's not robust to scaling or other monotone transforms, but I think it's close enough to being well-defined to handwave away for my purposes.

But if I'm wrong, and the variation in political opinions/interests that are politically salient in an ideal world is much higher, then the very "republican idea" of representative democracy is problematic.

"whether things interact with each other to greatly affect outcomes": good point. I'll see if I can incorporate it without being too wordy.

4. Weighted legislatures: yes, that's a whole topic I could write an entire section on. For now, can't afford to get that sidetracked, sorry.

"Variance seems like it might be a red herring, given a focus on outcomes/exemplariness."

Um. It's possible I'm not being clear what I mean by "variance". I don't mean variance of ideologies of legislators; I mean variance (meta-variance?) of distributions. That is to say, a 1-dimensional procedure for picking 2 legislators would have higher variance if it sometimes picked {5,5} and sometimes picked {4,6}, than if it reliably picked {0,10}.

I think there may something to your critique here aside from that possible misunderstanding, though. I have some thoughts but I'm not sure how I should or will deal with this issue. Probably, I should respond to that in a separate comment.

5.

"Weird methods:" Maybe I should explain how my existing proposal for rating voting methods would handle that? Because I have thought about it, and it is a tricky case. But I think I'll save that for later; for now, I want to stick with simpler cases.

...

to be continued; probably in a separate comment or comments, for better notification.

Comment by jameson-quinn on [ongoing] Thoughts on Proportional voting methods · 2020-07-03T17:28:51.159Z · score: 2 (1 votes) · LW · GW

I meant "𝓮" to be the number whose natural logarithm is 1 — "𝓮" is the closest to the math version of the character that I can type on my keyboard as-is. I don't have any argument for why it should be precisely that value, it's just the most famous number in between 2.5 and 3, which is roughly where I think the correct value will lie. I am aware of the larger literature on people's political preferences, but that's all from a world that's already conditioned by more-or-less non-ideal voting methods, so I don't think this is a question with an easy-to-find empirical answer like that. I may expand the section you reference to talk about these issues a bit more, but when I first wrote it, I was hurrying through to get somewhere else.

Comment by jameson-quinn on [ongoing] Thoughts on Proportional voting methods · 2020-07-03T15:03:20.193Z · score: 3 (2 votes) · LW · GW

A bit of polishing of existing parts; skeleton headings for remainder

Comment by jameson-quinn on Editor Mini-Guide · 2020-07-03T13:42:21.471Z · score: 2 (1 votes) · LW · GW

How can we add alt-text, so that our articles with images are accessible to visually-impaired readers?

What about captions?

Comment by jameson-quinn on Conditions for Mesa-Optimization · 2020-07-02T14:45:24.929Z · score: 2 (1 votes) · LW · GW

That link is broken, and I'm very interested to read this. Is there a newer link?


ETA: I think I found it: https://arxiv.org/abs/1810.10862

Comment by jameson-quinn on Conditions for Mesa-Optimization · 2020-07-02T14:44:09.683Z · score: 4 (2 votes) · LW · GW

I'm having difficulty with this in a way that "smells" like either "too much jargon" or "not enough effort expended explaining terms at start". That is, I (recent PhD in statistics) can more-or-less parse individual statements, but unable to chunk things into parts that are large enough to seem non-trivial. Note that I'm coming here from the "deep double-descent" article which does seem intuitive for me at first read (that is, definitely interesting, but closer to being too trivial than too complex).

I'll come back and read this again later, but for now: if you're editing this, I'd suggest starting slower and/or working to de-jargonify.

Comment by jameson-quinn on A Personal (Interim) COVID-19 Postmortem · 2020-07-02T14:12:40.518Z · score: 14 (7 votes) · LW · GW

"Rely more on other people's views in the rationality community" is inherently a bias-variance tradeoff. It's also collectively self-defeating, with a potential to become a death spiral. As such, it may be a reasonable thing to do, but it is potentially very dangerous as a principle to state for others to follow. The selfishly-first-order-rational thing to do would be to do more of it yourself, and encourage others to do less of it; of course, this is free riding, and may not be meta-rational.

My point is: this may be wise, but you should beware, and you should definitely not encourage others to do this without cautioning them against the risks.

One way to beware about this is to ask the question: "which of the biggest differences between rationality community conventional beliefs, and general 'smart non-explicit-rationalist' conventional beliefs, have the weakest support?" I have personal answers to that question but in the spirit of this comment, I won't share them here.

Comment by jameson-quinn on [ongoing] Thoughts on Proportional voting methods · 2020-07-02T13:20:31.768Z · score: 2 (1 votes) · LW · GW

If I get three "pings", now would be one. I think this has progressed far enough that a reader would begin to see the big ideas I'm driving at, and certainly far enough that I'd be ready for structural critiques. But if I only get two "pings", then no; I still haven't gotten to the biggest new idea or to either of the fun bits of historical trivia, and I'd want my two pings to be "at least two of those three" and "all done".

Comment by jameson-quinn on [ongoing] Thoughts on Proportional voting methods · 2020-07-02T13:16:33.760Z · score: 3 (2 votes) · LW · GW

Starting to get to the fun ideas! Hooray! But I'm probably still probably only about halfway done with the "theory" section, which means I'm probably only 30-40% done with the entire thing.

Comment by jameson-quinn on [ongoing] Thoughts on Proportional voting methods · 2020-07-01T15:37:28.297Z · score: 3 (2 votes) · LW · GW

Tightened up the "God's MSE decomposition" section. Almost ready for the big reveal of how this connects to a larger philosophy of assigning responsibility, in a way that I think could be of interest far beyond voting.

Comment by jameson-quinn on [ongoing] Thoughts on Proportional voting methods · 2020-06-27T22:11:26.444Z · score: 5 (3 votes) · LW · GW

Began a section on God's MSE decomposition, but it's very very sketchy still.

Comment by jameson-quinn on Covid 6/25: The Dam Breaks · 2020-06-26T17:49:15.702Z · score: 7 (4 votes) · LW · GW

Your comment makes me want to be more explicit about the norm I was using, so that I don't contribute to eroding the overall group norm.

I believe it is OK to post about politics if all of the following are true:

-Some point you make is non-obvious.

-Some point you make is potentially useful.

-Every point you make is beyond reasonable debate in your mind. That is to say, you are so confident that you are right that most readers would agree with you even after seeing someone post the other side, that you'd be willing to leave a contrary response without a public answer.

When I posted the above, I roughly thought that it met all those criteria. On second consideration, I think I was wrong about that in some regards.

Claim 1: The electoral college substantially contributed to the degree of failure of the US with regard to COVID-19. I consider this beyond reasonable debate. By saying so I am not casting aspersions on those who disagree or challenging them to express their disagreement, merely stating my considered opinion.

Claim 2: Gerrymandering and/or Senate malapportionment contribute to hyper-partisanship of the kind that could, at least in theory, in turn contribute to polarized acceptance of basic hygeine measures such as masks. Again, I consider this beyond reasonable debate.

Claim 3: The odds ratio for accepting voting reform between Republicans and Democrats is considerable, such that if a time traveller from the future told me that voting had been reformed before 2030, I would be confident that it had been done by Democrats. In my book, beyond reasonable debate.

Claim 4: Republicans would be able to veto reform by holding just 1 of the 3 loci of power I mentioned: WH, House, and Senate. Beyond reasonable debate.

Implicit claim 5: these are the three bodies that are most key to success. Debatable; should have explicitly disclaimed.

Claim 6: Democrats have non-negligible chance of carrying out reform. Debatable; shouldn't have said this.

Claim 7: Because of all of the above, you, dear USA reader, should vote for Democrats. Highly debatable and should have been out-of-bounds.

Note: by writing all this out, I am NOT encouraging object-level discussion. I'm sure there are people here who'd disagree with one or more of my judgements on what is or isn't debatable. I'm simply trying to be explicit about the interpretation of the rules I was implicitly using to justify writing my original post, and about how in retrospect, I think I was letting myself be too loose about that interpretation. I'd be happy to discuss these issues of norms insofar as it doesn't get bogged down at the object level.

Comment by jameson-quinn on Covid 6/25: The Dam Breaks · 2020-06-26T06:35:40.470Z · score: 1 (10 votes) · LW · GW

There's plenty of blame to go around, but even so, blaming "our civilization" is too broad a brush. US voting methods — the electoral college, gerrymandered single-seats in the House, and the malapportioned Senate — are horrible. They get obviously wrong answers far too often, including 2000 and 2016; and even when they don't, they encourage partisan polarization, which increases the mind-killing potency of politics. Both of these can be directly connected to the level of epidemiological failure in the US.

This is all fixable, without needing constitutional amendments. In practice, Republicans are almost certain to veto a fix if they can, while Democrats may allow a fix if we pitch it right. Thus, one clear and important step in fixing this is to elect Democrats to a trifecta — White House, House, and Senate.

Comment by jameson-quinn on [ongoing] Thoughts on Proportional voting methods · 2020-06-24T15:09:01.865Z · score: 3 (2 votes) · LW · GW

Added a section on God's proportional voting method.

Comment by jameson-quinn on [ongoing] Thoughts on Proportional voting methods · 2020-06-23T19:40:26.346Z · score: 3 (2 votes) · LW · GW

Added second section near top to explain where I'm intending to go with all of this.

Comment by jameson-quinn on [ongoing] Thoughts on Proportional voting methods · 2020-06-22T20:08:54.291Z · score: 3 (2 votes) · LW · GW

More, but overall status same.

Comment by jameson-quinn on [ongoing] Thoughts on Proportional voting methods · 2020-06-21T23:03:16.467Z · score: 5 (3 votes) · LW · GW

Added a few paragraphs, but I still haven't gotten to the interesting stuff; just still laying the groundwork.

Comment by jameson-quinn on [ongoing] Thoughts on Proportional voting methods · 2020-06-20T18:37:33.157Z · score: 2 (1 votes) · LW · GW

This isn't a link-post, it's just incomplete as yet. I mistakenly added a link because I thought that was a way to give a URL to a draft post.

But, how did you see it? I explicitly added Ben Pace, but not you.

Basically, I want to figure out how this works; sorry for the confusion, and eventually I'll fill in actual content here.

Comment by jameson-quinn on Can we use Variolation to deal with the Coronavirus? · 2020-03-18T17:15:37.951Z · score: 9 (5 votes) · LW · GW

I have no specific expertise here; I'm just a statistician.

I believe that if we're in this for the long haul — that is, over a year until a vacciine comes out, with responsible people spending the majority of that year in "suppression" mode according to the terminology of the Imperial College simulation — it would be beneficial for those under 30 without special vulnerabilities to be deliberately infected in a way that does not substantially spread the virus to the wider population. This would require a huge mobilization: facilities and social organization such that a bunch of kids with a few twenty-something caretakers could live (hopefully, happily) for a 6-8 weeks with absolutely minimal physical contact with people outside. That's possible to do well in principle — think summer camps — but doing it at the largest scale possible would involve big challenges, including having health care available for a bunch of sick-but-mostly-not-dying kids.

I suspect there is no way that Western societies will be able to do this at substantial scale. I would NOT suggest attempting it at individual scale. I would not be surprised to see China begin to attempt something like this. I would be surprised if they pulled it off without at least some horror stories. Those bad results might or might not be bad enough to outweigh the net good of improving herd immunity, allowing normal schooling going forward, and creating a population of under-30 immune people able to continue to work without having to worry about their own exposure (that is, about the inside of their body serving as a vector/reservoir for the virus; of course, the outside of their body would still need precautions not to be a vector).

Comment by jameson-quinn on 5 general voting pathologies: lesser names of Moloch · 2020-02-16T16:51:03.777Z · score: 2 (1 votes) · LW · GW

I believe the answers is "yes" to both questions, but I'm not 100% on the second one.

Comment by jameson-quinn on Using vector fields to visualise preferences and make them consistent · 2020-01-30T16:44:33.216Z · score: 4 (2 votes) · LW · GW

Consistency is the opposite of humility. Instead of saying "sometimes, I don't and can't know", it says "I will definitively answer any question (of the correct form)".

Let's assume that there is some consistent utility function that we're using as a basis for comparison. This could be the "correct" utility function (eg, God's); it could be a given individual's extrapolated consistent utility; or it could be some well-defined function of many people's utility.

So, given that we've assumed that this function exists, obviously if there's a quasi-omnipotent agent rationally maximizing it, it will be maximized. This outcome will be at least as good as if the agent is "humble", with a weakly-ordered objective function; and, in many cases, it will be better. So, you're right, under this metric, the best utility function is equal-or-better to any humble objective.

But if you get the utility function wrong, it could be much worse than a humble objective. For instance, consider adding some small amount of Gaussian noise to the utility. The probability that the "optimized" outcome will have a utility arbitrarily close to the lower bound could, depending on various things, be arbitrarily high; while I think you can argue that a "humble" deus ex machina, by allowing other agents to have more power to choose between world-states over which the machina has no strict preference, would be less likely to end up in such an arbitrarily bad "Goodhart" outcome.

This response is a bit sketchy, but does it answer your question?

Comment by jameson-quinn on Using vector fields to visualise preferences and make them consistent · 2020-01-30T14:52:38.923Z · score: 2 (1 votes) · LW · GW

Yes, the utility function is the "one dimension". Of course, it can be as complicated as you'd like, taking into account multiple aspects of reality. But ultimately, it has to give a weight to those aspects; "this 5-year-old's life is worth exactly XX.XXX times more/less than this 80-year-olds' life" or whatever. It is a map from some complicated (effectively infinite-dimensional) Omega to a simple one-dimensional utility.

Comment by jameson-quinn on Using vector fields to visualise preferences and make them consistent · 2020-01-30T14:46:40.315Z · score: 2 (1 votes) · LW · GW

Good point. Here's the raw beginnings of a response:

The idea here would be to resolve such questions "democratically" in some sense. I'm intentionally leaving unspecified what I mean by that because I don't want to give the impression that I think I could ever tie up all the loose ends with this proposal. In other words, this is a toy example to suggest that there's useful space to explore in between "fully utilitarian agents" and "non-agents", that agents with weakly-ordered and/or intransitive preferences may in some senses be superior to fully-utilitarian ones.

I realize that "democratic" answers to the issue you raise will tend to be susceptible to the "Omelas problem" (a majority that gets small benefits by imposing large costs on a minority) and/or the repugnant conclusion ("cram the world with people until barely-over-half of their lives are barely-better-than-death"). Thus, I do not think that "majority rules" should actually be a foundational principle. But I do think that when you encounter intransitivity in collective preferences, it may in some cases be better to live with that than to try to subtract it out by converting everything into comparable-and-summable utility functions.

Comment by jameson-quinn on Using vector fields to visualise preferences and make them consistent · 2020-01-29T23:25:06.414Z · score: 7 (6 votes) · LW · GW

I believe this is precisely the wrong thing to be trying to do. We should be trying to philosophically understand how intransitive preferences can be (collectively) rational, not trying to remove them because they're individually irrational.

(I'm going to pose the rest of this comment in terms of "what rules should an effectively-omnipotent super-AI operate under". That's not because I think that such a singleton AI is likely to exist in the foreseeable future; but rather, because I think it's a useful rhetorical device and/or intuition pump for thinking about morality.)

Once you've reduced everything down to a single utility function, and you've created an agent or force substantially more powerful than yourself who's seeking to optimize that utility function, it's all downhill from there. Or uphill, whatever; the point is, you no longer get to decide on outcomes. Reducing morality to one dimension makes it boring at best; and, if Goodhart has anything to say about it, ultimately even immoral.

Luckily, "curl" (intransitive preference order) isn't just a matter of failures of individual rationality. Condorcet cycles, Arrow's theorem, the Gibbard-Satterthwaite theorem; all of these deal with the fact that collective preferences can be intransitive even when the individual preferences that make them up are all transitive.

Imagine a "democratic referee" AI (or god, or whatever) that operated under roughly the following "3 laws":

0. In comparing two world-states, consider the preferences of all "people" which exist in either, for some clear and reasonable definition of "people" which I'll leave unspecified.

1. If a world-state is Pareto dominated, act to move towards the Pareto frontier.

2. If an agent or agents are seeking to change from one world-state A to another B, and neither of the two pareto dominates, then thwart that change iff a majority prefers the status quo A over B AND there is no third world-state C such that a majority prefers B over C and a majority prefers C over A.

3. Accumulate and preserve power, insofar as it is compatible with laws 1 and 2.

An entity which followed these laws would be, in practice, far "humbler" than one which had a utility function over world-states. For instance, if there were a tyrant hogging all the resources that they could reasonably get any enjoyment whatsoever out of, the "referee" would allow that inequality to continue; though it wouldn't allow it to be instituted in the first place. Also, this "referee" would not just allow Omelas to continue to exist; it would positively protect its existence for as long as "those who walk away" were a minority.

So I'm not offering this hypothetical "referee" as my actual moral ideal. But I do think that moral orderings should be weak orderings over world states, not full utility functions. I'd call this "humble morality"; and, while as I said above I don't actually guess that singleton AIs are likely, I do think that if I were worried about singleton AIs I'd want one that was "humble" in this sense.

Furthermore, I think that respecting the kind of cyclical preferences that come from collective preference aggregation is useful in thinking about morality. And, part of my guess that "singleton AIs are unlikely" comes from thinking that maintaining/enforcing perfectly coherent aggregated preferences over a complex system of parts is actually a harder (and perhaps impossible) problem than AGI.

Comment by jameson-quinn on Have epistemic conditions always been this bad? · 2020-01-28T15:57:59.428Z · score: 2 (1 votes) · LW · GW

I spent the 00s in Guatemala and Chiapas, so I'm probably not the best judge of that question.

As for writing this more deeply... frankly, it's unlikely to make it above the threshold on my to-do list.

Comment by jameson-quinn on Have epistemic conditions always been this bad? · 2020-01-27T14:09:54.371Z · score: 13 (8 votes) · LW · GW

I went to Oberlin College for undergrad; my in-laws are Central American communists; I live in Cambridge, MA, where my daughter goes to high school; and much of my internet activity is on left-leaning political blogs. So I think I have a reasonably broad experience of PC culture.

I'm not interested in getting deeply into this conversation here; it would take pages of writing to say everything I think, and that writing would be relatively slow because I'd have to measure my words in various ways to make it through this minefield. However, I do think that at least from my perspective, this concern is overblown. Yes, there are definitely people who self-righteously try to silence opposing views, and they do have some power; but in my experience, their power is limited in most places, and the capacity for reasonable dissent is still present.

As for the exceptions, I see no reason to believe they're particularly more widespread now than in the past (for instance, my parents have stories of weaponized conformity in EST meetings they briefly attended in the 70s). Furthermore, "dissent is illegitimate here" seems to me more often a symptom than a cause of toxic spaces.

So, sorry I don't have time to show my work on this, but for what it's worth, that's my opinion.

Comment by jameson-quinn on 2018 Review: Voting Results! · 2020-01-25T14:28:31.370Z · score: 4 (2 votes) · LW · GW

There's frequently a tradeoff between "less strategic incentives" and "more-intelligible under honesty". I don't think that you should pick the former every time, but it is certainly better to err a little bit on the side of the former and get good-but-slightly-more-confusing results, than to err on the side of the latter and get results that are neither good nor intelligible (because strategic voting has ruined that, too).

Comment by jameson-quinn on 2018 Review: Voting Results! · 2020-01-25T00:26:04.312Z · score: 4 (2 votes) · LW · GW

Just renormalize votes to be mean-0 before scaling.

Comment by jameson-quinn on 2018 Review: Voting Results! · 2020-01-24T21:05:50.232Z · score: 29 (6 votes) · LW · GW

It's far worse than that.

Even without the off-by-one bug, if you were able to vote on all the nominees, then the most efficient use of points would have been to make your average vote as close to 0 as possible. For instance, imagine there were 50 nominees, and one voter rated 5 of them at 10 to use their 500 points. If instead, they rated each of those 5 at 9, and the other 45 at -1, that would be 405+45=450 points, with exactly the same overall impact on relative standings. (In algebraic terms, this is a simple quadratic decomposition, akin to the fact that mean squared error equals bias squared plus variance).

It's clear from the vote results that the average voter did not ensure mean-0 and thus probably left some voting power on the table. The average of all votes is substantially positive.

This is even more "irrational" for those voters who were also authors of one or more of the works. Such voters were not allowed to vote on their own works, but could have helped their works win by voting all other works down. In other words, the incentive would be to have their average vote be negative, not just zer0.

I myself subtracted 2 from all my default-calculated votes for the above two reasons. Frankly, this 2-point difference (reduced to 1 in many cases by the off-by-one bug correction) was less than I considered selfishly rational, but I didn't want to take too much advantage of such "underhanded" strategy. Looking at the voting results, I don't think it's likely that anybody else but me did this.

Comment by jameson-quinn on 2018 Review: Voting Results! · 2020-01-24T03:47:47.153Z · score: 11 (6 votes) · LW · GW

On a self-interested note: you listed the top 15, but it appears my post, A Voting Theory Primer for Rationalists, was tied for 15th place, and yet was left off of the list. If that reflects some implicit tie-breaker, I'd like to know what it is. And — speaking purely as the author, not as a voting theorist — I'd suggest maybe you should just include my post in the list.

I'll make a separate post with some analysis ideas later.

Comment by jameson-quinn on Improving Group Decision Making · 2020-01-24T02:24:03.089Z · score: 7 (4 votes) · LW · GW

I'm glad to see that EA is engaging with social choice theory. But I don't find this paper particularly interesting. To me, it centers on the sleight-of-hand redefinition of Approval voting into "UNV", which is basically just "approval voting, but we assume the ballots are full expressions of the voters' preferences". That redefinition is necessary to cram approval into the procrustean bed of the Arrovian axioms; but to me, if the axioms don't fit a method without such ridiculous assumptions, we should find new axioms.

This is probably not the place for a more-lengthy engagement with these ideas and/or an attempt to situate them within my larger vision of the state of social choice theory. But I'd be interested to talk to Prasad, especially if I get the grant I've sought to pursue voting theory research over the next months.

Comment by jameson-quinn on Theory of Causal Models with Dynamic Structure? · 2020-01-23T23:42:11.947Z · score: 16 (5 votes) · LW · GW

Pyro can do this for certain technical reasons:

  • because it uses backwards-mode automatic differentiation based on pytorch, which (unlike other major python AD packages) creates its execution graphs on-the-fly.
  • Because pyro is primarily designed for variational inference, not MCMC. The problem with dynamic structure in MCMC is that it's inherently discrete: either the model has this structure, or it has that one. That makes it tough to do MCMC with, especially in high dimensions, because these days it's clear that making high-dimensional MCMC practical requires at least some use of the tricks of HMC, which means differentiating the likelihood with respect to the parameters, which means you need continuous parameters.

However: variational inference doesn't actually work well with complex dynamic structure, because the variational approximations themselves are usually "smooth" in ways that a correct posterior of a dynamic model would not be.

In other words: although it would not be hard to create a syntax / notation for expressing dynamic models, the reason there is not yet a standard syntax/notation for that is that the methods for fitting such models, or, to the same purpose, sampling from their posterior... are still mostly lacking.

It seems to me this is a fundamentally hard problem overall, so I don't expect it will be fully solved even in the next 5-10 years, but I do expect to see significant progress. PPLs are getting better by leaps and bounds now, and I'd expect something like Gen.jl to usefully cover at least some special cases of dynamic models, and to have a clearer syntax for at least talking about those models it still struggles to cover, in even just the 1-2 year time frame.

Comment by jameson-quinn on Moloch Hasn’t Won · 2020-01-18T09:00:07.988Z · score: 4 (2 votes) · LW · GW

Exactly, thank you.