[not ongoing] Thoughts on Proportional voting methods

post by Jameson Quinn (jameson-quinn) · 2020-07-22T02:46:27.490Z · LW · GW · 53 comments

Contents

  About this post
    Status
    Epistemic status
    Audience
    Terminology note
    Goal/endpoint of this piece
    TLD̦R (Summary)
  What does it mean to say a system is "democratic"?
    Digression: dimensionality and representation
  God's proportional voting method
    God's MSE decomposition?
  Allocating responsibility: a tricky (and important) problem
    Pascal's Other Wager? (The Problem of Points)
    Interlude: Am I barking up the wrong tree? Is it even clear to you what tree I'm barking up?
    Tentative answer
  Equalizing voters' retrospective power (RP)
    Phragmén and Thiele
  Measuring "Voter Choice", separate from power
  Average Voter Effective Choice: a combined proportionality metric
  Where to, next?
    AVEC in single-winner plurality
    Real-world multi-winner voting methods (under construction)
  Sections I'd originally planned to write, but have now decided not to:
  Real-world proportional voting reform (empty)
    Gerrymandering and "Fair Maps" (empty)
    Problems with proportionality: Israel, Italy, etc. (empty)
    Novel methods (empty)
    Prospects for change in US (empty)
    Canada, UK, India... (empty)
  Post-Mortem on this article (incomplete)
None
56 comments

About this post

This post was a work-in-progress, I (Jameson) updated it over the course of weeks/months, and the reposted it when it got substantial updates. I have now decided to leave it as it is for historical interest, as the next step would be a ground-up rewrite. I'm not sure when I'll have the time to finish that ground-up rewrite but I'd say there's a >80% chance something like that will happen within the next year; that is, before September 2021.

Status

I think I'm going to stop making large edits to the article as it stands. I'll add a longer "post-mortem" at the end. In sum: I think that the process of writing this article has helped me sharpen my thinking and make several breakthroughs, but that making an article that best conveys these insights to others requires rewriting from scratch. I still think that it would be worthwhile to eventually write the unfinished parts of my initial outline, but that attaching them to this thinking-it-through prologue would not do them justice.

Epistemic status

This is largely a work of political philosophy with a mathematical bent. That is to say, it is working through the implications of assumptions; more specifically, it is an attempt to find a way to estimate the implications of certain normative assumptions in certain real-world situations. As such, it rests on a larger context of thought about those assumptions — whether they are realistic, whether they are desirable — which I cannot possibly fully explore here.

But leaving aside those connections to wider ideas and taking the assumptions I make here as axioms, how confident am I of the train of thought that is contained within this essay? Reasonably so. At various times I gesture towards or sketch an idea of a mathematical proof for a claim, but I do not and have not worked out those proofs fully, so I'm aware that they may not "compile". Still, as somebody who has a PhD and so has the experience of following an idea more of the way to the ground, I would be pretty surprised (<10% subjective chance) if there were some mistake here so large that it would invalidate the main ideas (rather than just requiring minor adjustments). I would of course be a lot less surprised if someone found a better or simpler way to reach the same goals I'm pursuing here.

Last 3 updates

V0.9.1: Another terminology tweak. New terms: Average Voter Effectiveness, Average Voter Choice, Average Voter Effective Choice. Also, post-mortem will happen in a separate document. (This same version might later be changed to 1.0)

V0.9: I decided to make a few final edits (mostly, adding a summary and a post-mortem section, which is currently unfinished) then freeze this as is.

V0.7.3: Still tweaking terminology. Now, Vote Power Fairness, Average Voter Choice, Average Voter Effectiveness. Finished (at least first draft) of closed list/Israel analysis.

V 0.7.2: A terminology change. New terms: Retroactive Power, Effective Voting Equality, Effective Choice, Average Voter Effectiveness. (The term "effective" is a nod to Catherine Helen Spence). The math is the same except for some ultimately-inconsequential changes in when you subtract from 1. Also, started to add a closed list example from Israel; not done yet.

V 0.7.1: added a digression on dimesionality, in italics, to the "Measuring "Representation quality", separate from power" section. Finished converting the existing examples from RF to VW.

Audience

I have been thinking about proportional voting methods, and have some half-baked thoughts. "Half-baked" means that I think the basic ingredients of some worthwhile new ideas are in there, but they're not ready for general consumption yet. By the time you read these words, the thoughts below may well be more than half-baked.

I think it's worthwhile to begin write these thoughts out with a real-world audience in mind. I'm choosing to do that here. The readership of this site is not the ideal audience, but it's close enough.

Here are some assumptions/standpoints I'm going to take:

...

Terminology note

I will discuss both democratic "systems" and voting "methods", and these may appear to be interchangeable terms. They are not. "Methods" are specific algorithms which define an "abstract ballot" (set of valid vote types), and map from a tally of such votes to an outcome. "Systems" are broader; they include "methods", but also rules for when and how to hold elections, who gets to vote, etc.

Here's some simple notation/terminology I'll use:

...

Goal/endpoint of this piece

My goal here is to begin to put the comparative study of proportional (multi-winner) voting methods on a footing that's as solid as that of single-winner methods. In the case of single-winner methods, that solid footing is provided primarily by having a clear, unified metric: Voter Satisfaction Efficiency (VSE), which I'll explain briefly below. Although this metric is in principle sensitive to assumptions about the distribution of electorates, it is in practice robust enough to provide at least some useful guidance as to which methods are better or worse, and as to which pathologies in given methods are likely to be common or rare. That makes it much more possible to have a useful discussion about the tradeoffs involved in voting method design.

This is not to say that any metric will ever be the be-all-and-end-all of mechanism design. In the single-winner case, even though we have VSE, we still have to answer questions like: "Is it worth advocating for a method with better VSE that's more complex and/or harder to audit, or should we stick with something with a slightly worse VSE that's simpler?" But without having a metric, that debate would very quickly devolve into self-serving bias on both sides; everybody would tend to weight the scenarios where their preferred method performed well more heavily and discount those where it performed poorly. In other words, without a clearly-defined metric, it's almost impossible to properly "shut up and multiply".

Defining a good metric for proportional methods is harder than it is for single-winner methods. I am not the first person to attempt this, but I find prior attempts fall short of what is needed. As I write these words, I'm not finished defining my metric, but I do have a clear enough idea that I think I can do better than what's gone before. I also have a clear enough idea to know that, even when I'm done with this, my multi-winner metric will not be even as "mathematically clean" as VSE is. In fact, I suspect it may be provably impossible to create a mathematically clean metric with all the characteristics you'd like it to have; I suspect that it will be necessary to make some simplifying assumptions/approximations even though they are clearly violated. I hope it will be possible to do so in a way such that the measured quality of practical voting methods is robust to the assumptions, even if there might be methods that could intentionally exploit the assumptions to get unrealistic quality measures.

Once I've defined a metric, I'll discuss practicalities: which proportional methods I believe may be better in which real-world contexts, and how one might get those adopted.

TLD̦R (Summary)

It's well-known how to connect single-winner voting methods to utilitarian consequentialism. Doing the same for multi-winner voting methods has, until now, been harder. I provide a scheme for estimating a multi-winner voting method's utilitarian loss, retrospectively, using only the ballots and knowledge of the method. The metric I propose is a number from 0% (bad) to 100% (ideal; in practice, unattainable). I call it Average Voter Effective Choice (AVEC), and it's the product of Average Voter Effectiveness (AVE) times Average Voter Choice (AVC). I begin to give examples of AVEC calculations in a few situations, but stop before I finish all the voting methods one might look at.

What does it mean to say a system is "democratic"?

The archetypical case of democracy is a referendum. A new law is proposed; "all" the people get to vote "yes" or "no"; and "majority rules". In practice, "all" tends to be limited by some eligibility criteria, but I'd say that most such criteria make the system less democratic, other than maybe age and residency. The normative case that this kind of democracy is a good idea tends to rest on the Condorcet jury theorem: that is, if the average voter is more likely to be "right" than "wrong" (for instance, more likely to choose the option which will maximize global utility), then the chances the referendum outcome will be "right" quickly converge towards 100% as the number of voters increases.

But of course, not every collective decision naturally reduces to just two options, and not "all" the people have time to vote on all the collective decisions. (Which decisions should be made collectively and which should be made individually is out of scope for these musings, but I believe there are at least some cases that fall clearly on each side.) So not all "democracy" will be like that archetypal case. Still, it's pretty clear that any more-complex democratic system must be "similar" to the archetypical case in some sense.

When you're choosing between a finite number of options that are known at the time of voting, you need a single-winner voting method. It's pretty easy to define a way to measure "how good" such a method is: we can just use utilitarianism. (That doesn't mean you have to entirely buy in to utilitarianism as an ethical/philosophical system; even if you have reservations about it in a broader sense, I think it's pretty clear that it's a useful model for the purposes of comparing voting methods in the abstract, in a way that other ethical systems aren't.) A voting method is good insofar as it maximizes expected utility. To actually evaluate this expected utility, you'd need a distribution of possible electorates, where each possible electorate has a set of options and a set of voters. Each voter has some utility for each option, some knowledge about the other voters, and some rule for turning their utility and knowledge into a vote. As you might imagine, actually working this all out can get complex, but the basic idea is simple.

Choosing a single leader can be seen as a special case of choosing between various options. But electing a legislature is something different. How do you define the utility of a legislature that has a 60/40 split between two parties, versus one that has 70/30 split? There is (to me at least) an obvious intuition that a legislature is more democratic if it's more representative, and more representative if it's more proportional. With some additional assumptions, the drive for proportionality might be made more rigorous; if voters would be 60/40 on X vs Y but legislators are 70/30, there must probably be some X' and Y' such that voters would be 49/51 but legislators would be at least 51/49.

Still, "representative" or "proportional" are still just words; it turns out that, outside of simple special cases where there are a few pre-defined parties and voters care only about partisanship, political scientists don't tend to define these ideas rigorously. But we can do better. One way we might define "representative" comes from statistical sampling. Imagine that the legislature's job is merely to reproduce, as faithfully as possible, direct democracy, without ordinary voters having to waste all that time choosing and voting. In a statistical sense, that would mean the loss function would be something like: the expected squared difference between the percent a proposal gets in the legislature, and the percent it would have gotten in a direct referendum, given that the proposal is drawn from some predefined distribution over possible proposals. Note that we can still use the Condorcet jury theorem to argue that this would maximize utility, but now not only are we piling assumption on top of assumption, we're also making approximations of approximations. The legislative outcome is an approximation of the popular outcome which is an approximation of the ideal outcome; if the first step of approximating goes wrong, there's no reason to believe the second step will fix it.

(This loss function is effectively [the expectation of] a metric between distributions: measuring a kind of distance between the voter distribution and the legislator distribution, using the proposal distribution as an underlying probability measure. To help get an intuition for what this means, imagine that ideology is a single dimension, with the voters somehow distributed along it, and each proposal is a cut point. Assuming the proposal distribution is continuous, we could without loss of generality rescale the ideology so that the proposal distribution is Uniform(0,1), and then graph those proposals on the x axis with the proportion of voters to the left of the cut on the y axis. We could superimpose a graph with the proportion of legislators to the left of the cut on the y axis; this loss function would then be the integral of the square of the vertical distance between those graphs.)


 

A sketch to help visualize the above

This is a rigorous definition of "representative", but a deeply unsatisfying one. By this definition, one of the "best" possible voting methods would be sortition; just choose representatives uniformly at random from the population. We know from statistical sampling theory that in this case, the loss function stated above, for literally any possible question, is asymptotically consistent; it falls off as the inverse square root of the size of the legislature. (Such a simple random sample isn't actually the best; you can improve it by doing variance-minimizing tricks such as stratifying on self-declared party or other salient political variables. But that's a minor detail which changes the error by a roughly constant factor, not by changing the basic limiting behavior.)

But in the single-winner case, sortition (even stratified sortition) just means randomly choosing a dictator, which is clearly nowhere close to being among the most-democratic single-winner methods. Clearly, there's some democratic ideal or ideals missing from the above loss function.

There are at least two characteristics that good representatives should have, besides having similar views collectively to those whom they represent: they should be exemplary, and they should be accountable.

"Exemplary" means that in some regards, you want them to be systematically different from the average of the people they represent; smarter, less ignorant, better at positive-sum negotiation, etc. For instance, a linguistic minority would want to be represented by somebody who was fluent in the language of business in the legislature, even if they themselves mostly were not. Sortition is not a good solution here.

"Accountable" means that there is at least some mechanism to help better align incentives between representatives (agents) and their "constituents" (principals). This generally means that voting rules work to aggregate preferences in a way that considers all votes equally. Again, sortition fails this badly.

Note that by adding "exemplary" and "accountable" to our list of goals, we've opened up the possibility for legislative outcomes to be not just as-good-as direct democracy, but actually better, in several ways. First, it may even be that there is some optimum size for reasoned debate and group rationality; too small and there is not enough diversity of initial opinions, too large and there is not enough bandwidth for consensus-seeking discussion. Second, exemplary representatives may be better than ordinary voters at understanding the impact of each decision, or at negotiating to find win-win compromises. It's difficult to imagine formalizing this without making huge arbitrary assumptions, but it's similar to the idea of "coherent extrapolated volition"; that is, exemplary representatives may be able to extrapolate voters' volition in a way that the voter might not exactly agree with a priori but would see in retrospect to be better. Which brings us to the third way better outcomes might be possible: by making representatives accountable primarily not on a moment-by-moment basis, but on a once-per-election time cycle, we might nudge them to think from a slightly more future-oriented perspective.

This idea — that representative democracy doesn't just save voters' time; that it, in principle, can actually give better outcomes than direct democracy — is so important, it already has a name: republicanism. (I'll be using that word as I continue this article; to avoid confusion, I'll refer to the main US political parties as "GOP" and "Dems".)

It's clear that all three of these ideals — representativity, exemplariness, and accountability — have to do with democracy. One can imagine pairs of situations that differ on only one of these three axes, and still, one member of the pair will seem (in my strong intuition) more democratic than the other. For example, imagine the fictional planet mentioned in The Hitchhikers' Guide to the Galaxy, where people vote to elect lizards. Though the complete lack of representativity makes this a poor democracy, it would clearly be even less democratic if the losing lizard committed electoral fraud.

...

Digression: dimensionality and representation

Above, we've defined "representativeness" as some kind of difference-in-distributions between the voters (that is, their support for any given proposal drawn randomly from the distribution of possible proposals) and the legislature (that is, the expected support for any given proposal, where the expectation is taken across any randomness in the voting method). ((Note that in some cases, when considering the legislature's "expected" support for proposals, it may be convenient to condition, not on the ballots, but only on the voter's views on all proposals; that is, to consider the possible variability in voters' strategic behavior as both "random" and "part of the voting method".))

Since the number of voters is usually quite large compared to the size of the legislature, we might imagine/approximate the voter distribution as a continuous "cloud", and any given legislature as a set of points intended to represent that cloud. Since we're interested in how well those points can possibly represent that cloud, one key variable is the effective dimension of that cloud.

To be specific: we know that it's possible to represent the cloud spatially, with any possible proposal as one of the dimensions. This is potentially an infinite-dimensional space, but in practice, it will have some dimensions with high variance, and some dimensions with negligible variance.

You've probably seen "political compass" diagrams which project political ideologies onto two dimensions:

But of course, those aren't the only two dimensions you could use to categorize voters:

If we use the dimensions that tend to align well with how voters would divide up on high-probability legislative proposals (the principal components), how many dimensions would we need? ((OK, "effective dimension" isn't exactly that; it measures not only how many "relatively big" dimensions there are, but also how "relatively small" the rest are. I'm being deliberately vague about how precisely I'd define "effective dimension" because I suspect that unless you ignore variation below a certain noise threshold the ED is actually infinite in the limit of infinite voters.))

If the "effective dimension" is low, then a relatively small legislature can do quite a good job at representing even an infinite population. If the effective dimension is high, though, then the ideological distance between a randomly-chosen voter and the closest member of the legislature, will tend to be almost as high as the distance between any two randomly-chosen voters. In other words, there will be so many ways for any two people to differ, that the very idea that one person could "represent" another well begins to break down (unless the size of the legislature grows exponentially in the number of effective dimensions)

What is the effective dimension of political ideology in reality? It depends how you measure it. If you look at US incumbent politicians' voting records using a methodology like DW-NOMINATE, you get a number between 1 and 2. If you look at all people's opinions on all possible issues, you'd surely get a much higher number. I personally think that healthy political debate would probably settle at about 𝓮 effective dimensions — that is, a bit less than 3. For getting a sense of this, 5^𝓮≈80, 10^𝓮≈500, 20^𝓮≈3500; so, if I'm right, a well-chosen reasonably-sized legislature could do a reasonably good job representing an arbitrarily-large population.

But if I'm wrong, and the variation in political opinions/interests that are politically salient in an ideal world is much higher, AND it's impossible to deal well with "one issue at a time" (that is, tradeoffs due to high-dimensional structure are meaningful), then the very "republican idea" of representative democracy is problematic. A large-enough legislature could still serve as a random polling sample, reproducing the opinions of the larger population on any arbitrary single question with low error. But as the legislature debated multiple issues, considering the balance of possible tradeoffs between several related choices, the chances that there would be some combination of points of view that exists in the population but not in the legislature would go to 100%.

Understanding dimensionality is especially important for analyzing voting methods that involve delegation. If politically-salient ideologies are relatively low-dimensional, then your favorite candidate's second-choice candidate is likely to be someone you'd like. If they're high-dimensional, that two-steps-from-you candidate is not significantly better from your point of view than a random choice of candidates. I'll talk more about delegation methods, which can help simplify ballots and thus allow broader overall choice, later.

God's proportional voting method

Say that an omniscient being with infinite processing power wants to select a legislature of size S to represent a population 𝒱 of V chosen ones. God knows the distribution of proposals that this legislature will vote on (and for simplicity, that distribution is independent of who is in the legislature), and she knows the distribution of utility that each outcome option will give to each person v in 𝒱. (For technical reasons, let's assume that each chosen one's distribution of utility for each option is absolutely continuous with regard to some common interval. If you don't know what that means, you can safely ignore it.) Her goal is to use a process for choosing a legislature that will:

Of course, since this is God we're talking about, she could come up with selection rules that are better than anything I could come up with. But if I had to advise her, here's the procedure I'd suggest:

1. Divide the population up into equal-sized groups, which we'll call "constituencies", in a way that minimizes the total expected within-group variance for an option that's randomly-chosen from the distribution of votable proposals. In essence: draw maximally-compact equal-population districts in ideology space.

2. Within each constituency, give each voter a score that is their piety, divided by the expected piety of a person in that group with their utilities. (I'm intentionally being a bit vague on defining that expectation; there are tighter or looser definitions that would both "work". Tighter definitions would do less to optimize piety but would be unbiased in a stricter sense; vice versa for looser definitions.)

3. Choose one representative from each group, using probabilities weighted by score.

In other words: stratified sortition, using probabilities weighted by piety and "inverse piety propensity".

What's the point of even discussing this case? Obviously, if God were choosing, she wouldn't need a legislature at all; she could just choose the best option each time. But I bring this up to illustrate several points:

God's MSE decomposition?

Let's say God's busy, so she asks the archangel Uriel to choose the legislature. Uriel isn't omniscient, so he doesn't know everyone's utility for each proposal option, and even if he did, he couldn't perfectly integrate that over the high-dimensional distribution of proposals the legislature might vote on. So he merely asks each chosen one to write down how they feel about each candidate on a little piece of paper, then, using the papers, does his best to choose a representative body. In other words, he holds an election.

When God returns, how will she judge Uriel's election? First off, let's say that she doesn't just judge the particular outcome of one election, but the expected outcome of the method across some multiversal distribution of possibilities. She's omniscient, so she directly knows the expected mean squared error (MSE) between the percentage supporting each proposal in an idealized version of direct democracy (correcting towards coherent extrapolated volition, as much as republicanism allows), and the percent supporting within the chosen legislature. But to help Uriel understand how he's being judged, I expect she'd decompose this MSE into various components, following the same step-by-step logic as I outlined in my suggestion above for her own selection method.

Classically, MSE can be decomposed into squared bias plus variance. In many cases, each of these can be decomposed further:

So let's go through the steps of God's process; suggest a way to establish a correspondence between that and an arbitrary voting method; and see what components of MSE might have arisen at each stage.

))(I bet you missed the big reveal! That is to say, I think the most novel and philosophically-meaty idea in this essay was buried in clause 2 of the sentence just above. Establishing a step-by-step correspondence between an arbitrary voting method, and a single unrealizable "perfect" method, is, I believe, an important idea, as I'll discuss further below. I would have said so earlier, but I didn't see a way to get there directly without laying all the groundwork above.)((

God's first step was to divide voters into S equal-sized "constituency" groups, minimizing the variability within each constituency. Multi-winner voting methods do not necessarily divide voters into S groups. However, they do, by definition, elect S winners. So if we want to assign voters into S (possibly-overlapping) constituencies, we have merely to trace the chain of responsibility backwards: for each of the winners, which voters were responsible for helping them to win?

In more quantitative terms: for each candidate c and voter v, we want to assign a quantity of responsibility r(c,v), such that the sum over v for any given c is always 1. I'll be saying a LOT more about how this might work below.

There are several kinds of errors that could happen at this step:


 

Rough picture of the above issues. Not perfect; don't @ me. Note that the space of the diagram is intended to be ideological, not geographical.

God's second step was to choose one representative for each of the equal-sized groups, using an algorithm that balanced the goals of exemplary "piety" and ideological unbiasedness. Note that in rating a voting method, we're doing this "backwards": we constructed the "set of responsible voters" from the candidate, not vice versa. Still, we can use this "backwards" procedure to consider the inherent biases and variances of whatever arbitrary "forwards" voting method that Uriel actually used. These might include:

I've listed various different possible problems, and suggested that they could be the basis of an MSE decomposition. That is to say, God could begin grading the outcome by using the above problems as a rubric; subtracting points separately for problems of each type. But of course, that kind of MSE decomposition only works if the different kinds of problem are orthogonal/independent. So in practice, God might end up adding or subtracting a less-effable "fudge factor" at the end.

Allocating responsibility: a tricky (and important) problem

A key step in "God's MSE decomposition" for evaluating a multi-winner voting method is assigning responsibility; deciding, using the ballots, which voters should be considered "constituents" of which winning representative. That's not at all as simple as it might seem. Even if you know the ballots, the voting method, and the winners, it's not necessarily easy to make a function r(c,v) to say which ballots helped elect each of the winners and by how much. You know not only exactly the outcome in reality, but also the counterfactual outcomes for every other possible combination of ballots; still, how do you turn all that into the numbers you need?

Before going on, we should note that although we are interested in using the answer to this problem as one sub-step in evaluating multi-winner voting methods, the problem itself applies equally-well to single-winner voting methods: given that candidate X won, which voters are responsible, and by how much? Thus, until further notice, most of the concrete cases we'll be considering come from the simpler domain of single-winner methods.

With single-winner plurality, it's easy to allocate responsibility using symmetry: all the N people who voted for the winner did so equally, so each of them has 1/N responsibility. This is independent of how many voters there are or how many voted for other candidates; N could be 99%, or 50.01% in a two-way race, or even 1% in a 101-way race.

But that kind of symmetry argument doesn't work for complex voting methods with many more valid ways that each voter could vote than there are candidates. In such cases, even briefly considering edge cases such as Condorcet cycles or impossibility results such as Arrow's or Gibbard-and-Satterthwaite's theorems suggests that allocating responsibility will be no simple task.

The problem of allocating responsibility has been considered before in the context of single-winner voting methods; there are existing formulas for "voter power" indices, and my eventual answer is similar to one of them. But this issue actually goes well beyond voting. There are many situations when an outcome has occurred, but even though the inputs and counterfactuals are more-or-less agreed-upon, people can still argue as to which of those inputs are responsible.

Here's my plan for discussing this question:

Consider the problem of legal liability. There are many cases of laws that where one of the criteria for a person being liable is that their actions "cause" a certain outcome. One traditional legal definition for "cause" in this sense is "sine qua non", Latin for "without this, not"; in other words, X causes Y iff Y wouldn't have happened without X. But what if two people independently set separate fires on the same day, and as a result my house burns down? Under a "sine qua non" standard, person A isn't liable because even without them fire B would have burned my house, and vice versa.

This merely shows that the traditional legal standard of "sine qua non" is silly, not that it's not easy to find a good standard. I know; the fact that lawyers and judges haven't figured out how to allocate responsibility, doesn't prove that doing so is truly hard. But it does at least show that it's not trivial.

If all you care about is whether or not someone was partially responsible for an outcome, and not how much responsibility they deserve, there's a more-or-less simple, satisfying answer (and the lawyers are just beginning to discuss it, though it's not a settled matter): a factor is causal if it's a necessary element of a sufficient set (NESS). (Actually, in the legal situation, there's an additional complication needed for a good definition of responsible causes. If something it was "established/known" "after" a sufficient set already existed, it is not eligible to be a cause; and defining those terms in a way that gives satisfying answers is tricky. For instance, a fire set after my house burned down cannot be a cause, even if it is a NESS. But discussing this issue is out of scope here, because in the case of voting methods, only ballots matter, and all ballots are considered to be cast simultaneously.)

NESS answers the question of who shares responsibility, but it doesn't answer that of how much responsibility they have. For instance, imagine that a group of people made stone soup with 1 stone, 1 pot, water, fire, and ingredients; and that in order to be a soup, it needed a pot, water, and at least 3 ingredients. NESS tells us that the person who brought the stone was not responsible for the soup, and that everyone else was; but how do we divide responsibility among the others? Simple symmetry shows that each ingredient gets the same responsibility, but there are many facially-valid ways you could set relative responsibility weights between an ingredient and the pot.

There's also another flaw with NESS: similar outcomes. Consider a trolley problem: you just pulled the lever, saving 5 people by diverting the trolley to crush 1. By NESS, you are now a murderer; a necessary element of the set of actors leading to that one guy's death. But, unlike the fools who let the trolley run away in the first place, you are not in a NESS for the event "somebody dies here today". In proportional voting terms, we will need to create such composite events — "at least one of this subset of candidates is elected" — because otherwise, frequently, a sufficient set will not be complete until there are so few voters outside the set that they couldn't tip the balance between two similar candidates.

Pascal's Other Wager? (The Problem of Points)

This is beginning to look a lot like another famous problem from the history of mathematics: the "Problem of Points". This question was originally posed by Luca Pacioli (most famed for inventing double-entry bookkeeping): say that two people are gambling, and they have agreed that a pot of money will go to the first player to win a total of N games. But for some reason they are interrupted after neither player has won; say that one of them has won P and the other Q, with P>Q. How should the pot be divided fairly? Clearly, player P should get more, because they were closer to winning; but how much more?

Like the retroactive allocation of responsibility that we're considering, this is a case of allocating a fixed resource between various agents in a well-defined situation (one without any "unknown unkowns"). And similar to our case, solving it is deceptively hard. Several smart mathematicians of the time—such as Tartaglia and Cardano, rival co-discoverers of the cubic formula—proposed "solutions", but there was something unsatisfying about all of them. For instance, Pacioli suggested that the two players should split the pot in proportion to the number of games they'd won; but this would give the entire pot to somebody who'd won just 1 game to 0 out of a tournament to 100, a tiny lead in context.

The problem was definitively solved in letters between Fermat and Pascal. Fermat got the right answer, and Pascal refined the argument to support it; Pascal's letter of August 24, 1654 is considered one of the founding documents of a branch of math. I'll tell that answer in a moment, but in case you don't already know it, I'll give you a moment to pause and see how you'd solve the problem.

....

The answer is: the pot should be split in proportion to the probability that each player will win it.

This answer may seem almost obvious and trivial to us, but to Fermat and Pascal, it required a leap of insight. This was arguably the first time people used math (probability) to reason about events in an explicitly counterfactual future, as opposed to simply generalizing about future events that closely resemble oft-repeated past ones. This key idea — that in order to better describe the universe as it is, we must sometimes rigorously analyze states that are not and will never be — is the foundation of risk analysis, which is itself key to modern society. (Here's a video which tells this story, and makes this argument for its importance, at length.)

In that case, Pascal and Fermat were considering how to "justly" allocate a fixed resource: a pot of money. The problem had been posed without a clear definition of what "justly" would mean, but the solution they came up with was so mathematically beautiful that once you've understood it, it's hard to imagine any other definition of justice in that case.

It's clear that this is an interesting story, but why do I spend so much time on it, when the problem at hand is allocating retrospective responsibility, not prospective odds? Am I about to argue that odds and risk analysis are key to answering this problem, too?

No! In fact, quite the opposite. I think this example is relevant in part because the very arguments that pushed Pascal and Fermat to think probabilistically, should push us to avoid basing our answer primarily in probabilities. A key argument they used against earlier proposed "solutions" such as Pacioli's was that Pacioli's answer would always give the same share for given values of P and Q, independent of N; but clearly a player who leads by 7 games to 4 is in a much better position if the threshold to win the pot is 8 games than if the threshold is 100 games. However, in order to allocate responsibility retroactively, we already saw using symmetry arguments that in the case of plurality, responsibility is inversely proportional only to winning vote total, independent of the threshold it would have taken to win.

Interlude: Am I barking up the wrong tree? Is it even clear to you what tree I'm barking up?

So far in this essay, I've been explaining things I believe I understand pretty well. That is, most of what I've said so far, I think that if you asked me to, I could explain at greater length and with more rigor. This is about to change. This section, in particular, deals with ideas that I am not yet done chewing over. So I'll make some claims here that I can't entirely back up with concrete arguments.

But that doesn't mean this is pure speculation, either. I've spent literally decades of my life thinking about this stuff, and that pursuit has included getting a doctorate in statistics. So if you, reading this, think you see a fatal flaw in my ideas, you may well be right; but if you think that the flaw is obvious and that I haven't at least considered it, you're very probably wrong. My counter-counter-argument may be handwavy and ultimately incorrect, but it at least exists.

If you look at previous attempts to solve retroactive responsibility problems, most rely implicitly or explicitly on the kind of probability-based arguments that you use for proactive problems like the problem of points. For instance, voter power indices might be based on the probability that a given (set of) actor(s) is pivotal, given some distribution of actions for the other actors. I believe that this logic is ultimately a dead end. Once you have a retroactive scenario, where actual voters each cast an actual ballot, reasoning that's based on "what was the probability that those people would have acted differently" will inevitably be sensitive to assumptions in a way that undermines it. Furthermore, this problem is exponentially compounded by the need to consider more than two possible outcomes, and the issues that result from that — that is, Condorcet cycles, Arrow's theorem, the Gibbard-Satterthwaite theorem, etc.

But I didn't tell the story of the Problem of Points just to say what the answer to retroactively allocating responsibility isn't. It was also a good example of a problem specified in loose practical terms (who "deserves" what share of the pot), where just translating into rigorous math terms turned out to be more than half the work of finding a solution that pretty much anyone can agree is "correct". So, if the goal is to solve a similar loosely-defined problem, our first step should be to restate why we want a solution and what characteristics we know that solution should have.

We're trying to estimate the expected mean squared error (squared bias plus variance) of a voting method, by looking at an outcome of that voting method. All we necessarily have is the ballots and the winners; we imagine that came from some kind of decision process that might be represented as voters and candidates located in an ideological space where voters tend to prefer candidates who are closer, but we don't have enough information to reliably place voters and candidates in that space. So we model the voting system as having had two steps: grouping voters into (possibly-overlapping) weighted constituencies, and choosing a winner in each constituency. Assigning responsibility means giving each voter a weight in each constituency — how much power they exercised to help elect that winner. If we do a good job assigning responsibility, then it should be easy to find counterfactual outcomes where changing responsible votes changed the winner, but harder to find counterfactuals where changing non-responsible votes did so.

When I put it like that, it may sound as if I'm solving the wrong problem. I'm trying to find an "allocated responsibility" measure as a one-dimension-per-candidate approximation to a more-complex causal structure of the actual voting method; so that I can then combine that with some approximation of how candidates are selected within the constituencies; so I can use that to approximate the expected MSE between the voters' (exemplariness-weighted) ideological distribution and that of the legislators; so I can use that to approximate the utility lost between those two things; when, to start out with, the voters' ideological distribution was itself an approximation to the utility-maximizing outcome, justified by the Condorcet Jury Theorem. There's already 5 different levels/types of approximation in there, and I'm finished yet; surely, there must be a better way?

I believe there probably isn't. That is to say: I believe that each of the approximation steps so far is plausibly the lowest-error way to accomplish its task. Specifically, all of them are "neutral" in terms of ideological bias in a way that alternate approximation schemes often are not; and all of them are, plausibly, among the lowest-variance possible approximation strategies that have such neutrality. Just as in Churchill's famous dictum that "democracy [that is, the last 2-3 approximation steps in that chain] is the worst form of Government except for all those other forms that have been tried from time to time", I believe that responsibility-allocation may be the worst way to evaluate multiwinner voting methods except for all the other forms that have been tried (or have occurred to me over my years of thinking about this) from time to time.

Furthermore, while this concept of numerically allocating responsibility is used in this case as one step in a larger strategy of evaluating voting methods, I think that numerically-allocated responsibility could be useful in other contexts. I've already mentioned that it relates to legal liability. I also think it could be useful to individual voters in considering voting tactics or even the decision of whether it's worth voting in the first place. I'll say a bit more about that latter issue later.

Tentative answer

Thanks to Thomas Sepulchre's comment below, I have a solution to this problem that I consider "correct". I'll copy his explanation here:

I think there's actually one way to set relative responsibility weight which makes more mathematical sense than the others. But, first, let's slightly change the problem, and assume that the members of the group arrived one by one, that we can order the members by time of arrival. If this is the case, I'd argue that the complete responsibility for the soup goes to the one who brought the last necessary element. 

Now, back to the main problem, where the members aren't ordered. We can set the responsibility weight of an element to be the number of permutations in which this particular element is the last necessary element, divided by the total number of permutations. 

This method has several qualities : the sum of responsibilities is exactly one, each useful element (each Necessary Element of some Sufficient Set) has a positive responsibility weight, while each useless element has 0 responsibility weight. It also respects the symmetries of the problem (in our example, the responsibility of the pot, the fire and the water is the same, and the responsibility of each ingredient is the same)

In a subtle way, it also takes into account the scarcity of each ressource. For example, let's compare the situation [1 pot, 1 fire, 1 water, 3 ingredients] with the situation [2 pots, 1 fire, 1 water, 3 ingredients]. In the first one, in any order, the responsibility goes to the last element, therefore the final responsibility weight is 1/7 for each element. The second situation is a bit trickier, we must consider two cases. First case, the last element is a pot (1/4 of the time). In this case, the responsibility goes to the seventh element, which gives 1/7 responsibility weight to everything but the pots, and 1/14 responsibility weight to each pot. Second case, the last element is not a pot (3/4 of the time), in which case the responsibility goes to the last element, which gives 1/6 responsibility weight to everything but the pots, and 0 to each pot. In total, the responsibilities are 1/56 for each pot, and 9/56 for each other element. We see that the responsibility has been divided by 8 for each pot, basically because the pot is no longer a scarce ressource.

Anyway, my point is, this method seems to be the good way to generalize on the idea of NESS : instead of just checking whether an element is a Necessary Element of some Sufficient Set, one must count how many times this element is the Necessary Element of some permutation.

I believe that TS probably independently "invented" this idea (as a competitive programmer, I expect he'd have seen lots of problems and have a good toolbox of solutions and strategies for re-adapting those solutions to new problems). But this solution is actually a version of the classic (1954) Shapley-Shubik voter power index.

((Technical note, feel free to skip: the key difference is that originally S-S was made for measuring prospective voter power in single-winner elections; in the retroactive responsibility, multi-winner case, we only consider counterfactuals in which ballots support a given candidate "less" than they did in reality. This involves defining this concept of "less support"; ideally this would be done in such a way that the voting method was monotonic over ballots, but in cases of complex voting methods, it may be worthwhile to accept some non-monotonicity in order to get stricter and more-useful definitions of "less" support. ))

Interestingly, the ordinary Shapley-Shubik index can be interpreted as a Bayesian probability, by using the right prior (Straffin, P. D. The Shapley-Shubik and Banzhaf power indices as probabilities. in The Shapley value: essays in honor of Lloyd S. Shapley). From an individual voter's perspective, what Straffin calls the "homogeneity assumption" can be stated as a Polya(1,1) prior; in practical terms, this could mean that the voter expects that their own decision is acausally related to all other voters' decisions, and is using some acausal decision theory. (For more discussion and links on voting and acausal decision theories, see here [LW · GW].)

((The Polya parameters 1,1 there mean a relatively high correlation between the ego voters' decision and that of other voters, though it is slightly weaker than with the Jeffreys prior of Polya(.5, .5). A strictly functional-decision-theory agent would probably expect a weaker correlation with other voters, as they would only consider correlations that came from other voters using very similar decision procedures. That would probably lead to priors more like Polya(total_polled_yesses_across_reputable_polls, total_polled_nos_across_reputable_polls), which would assign much lower power to most voters — more similar to the Banzhaf power index. Personally, I find Polya(1,1) to be a reasonable prior; aside from having an easy-to-calculate algorithm as given by Sepulchre, it feels "about right" for my personal decision theory.))

For the modified, retroactive Shapley-Shubik index we're using here, I'm not entirely sure what prior it's equivalent to, or even if there is such a prior. However, intuitively, it seems to me that it's likely to be something like "you're correlated to all the voters who are at least as supportive as you are of any candidate you help elect, using a Polya(1,1) prior". This idea — that for purposes of decision theory, you should give more consideration to your acausal links to other voters if those other voters hold similar underlying preferences to you — is intuitively appealing to me.

Shorter "solution" statement

Say you have a full set of ballots ℬ and an outcome O. For each individual ballot b_i in ℬ, and each set of candidates 𝒸 containing exactly one winner c_j for O, take ℒ(b_i, 𝒸) to be the set of possible legal ballots which support all candidates in 𝒸 "less" than or equal to what ballot b_i does. Take k_1...k_V to be a random permutation of the integers 1...V; this is the only randomness here so all expectations are with respect to it. Take K(i) to be that integer n such that k_n = i. E(...) denotes expectation and I(...) denotes an indicator function of an event; that is, 1 if that event is true and 0 if it is false.

Voter i's "retrospective power" (RP) to elect a member of the set 𝒸 is:

(V/S)*E( I(some candidate in 𝒸 wins for all elections such that b_{k_1} to b_{k_{K(i)}} are held constant, and b_{k_{K(i)+1}} to b_{k_V} are allowed to be replaced by ℒ(..., 𝒸) ) - I(some candidate in 𝒸 wins for all elections such that b_{k_1} to b_{k_{K(i)-1}} are held constant, and b_{k_{K(i)}} to b_{k_V} are allowed to be replaced by ℒ(..., 𝒸) ).

(To do: define a bit more notation up front so I can write that in clean LaTeX.)

When calculating the RP to elect an individual candidate c_j, use the power to elect a set, choosing 𝒸 as follows: of all the sets possible sets 𝒸 which include c_j, choose the one with the smallest measure among the ones which assign RP to the fewest voters.

The constant V/S (number of voters divided by number of seats) is added up front so that the average RP for each voter will, by construction, always be 1. Since the average is constant, minimizing the standard deviation will be the same as minimizing the mean of squares.

However, this mean of squares, while it is naturally relevant as an estimator of the utilitarian loss, is hard to explain as-is; for instance, its natural units are completely abstract. Thus, in the overall "Average Voter Effective Choice" metric below, I'll find a (monotone) transformation from this into a more-interpretable version.

Equalizing voters' retrospective power (RP)

Now that we have a defninition of retrospective voter power, we can use it as an ingredient in a metric of multi-winner voting methods. That is, all else equal, we might want to avoid the following:

I'm going to focus for now on just the first of these two. We'll deal with "choice" (which allows voters to ensure their reps are "close" to them ideologically) separately. In particular, it's possible that "cake-cutting-like" algorithms could simultaneously improve exemplariness, have little cost in terms of choice, and cause high overlap in measured voter power; so directly avoiding overlap is not necessarily a good thing.

Phragmén and Thiele

In fact, there is already a proportional voting method that is designed to ensure voters have roughly equal power, and that defines that power in a way that's entirely compatible with, though less generally-applicable than, our definition above. And in fact, it was first considered in the 1890s by two of the earliest theorists of proportional voting, Scandinavian mathematicians Edvard Phragmén and Thorvald Thiele. However, since neither of them ultimately settled on this method as their favorite, it fell into obscurity; occasionally reconsidered or reinvented (by Saint-Laguë in 1910, Ebert in 2003, and Mora in 2016).

The method was invented in a Swedish context of weak parties and approval-style ballots. In the case of such approval ballots, there's an obvious way to calculate each voter's responsibility for electing each winner: 1/p, where p is the proportion of ballots that approved that winner. Thus, one of the methods Phragmén considered was to explicitly try to minimize the variance/standard deviation of voters' total power. Since the sum of total power is constant, this is equivalent to minimizing the sum of each voters' squared power; known as the "squared load".

Note that this is an optimization problem, and a naive algorithm for finding the optimum might have to check a combinatorial explosion of possible winner sets. Thus, in practice, the method would probably done sequentially: at each step, elect one new candidate, minimizing the sum of squared loads.

I mention this method not because I think it's better than other proportional methods, but because it shows that the idea of "minimizing variance in voter power" goes all the way back to the origins of proportional voting methods. Overall, I think that approval-ballot-based proportional methods tend to lead to too much "overlap" and thus to poor average voter choice, unless voter strategy is implausibly precise. Also, as Phragmén himself pointed out, it often isn't "optimal" using this rule to pick a candidate who is approved by literally all the voters, as it leads to apparent "imbalance" in voter power.

Measuring "Voter Choice", separate from power

I've already mentioned the idea of "choice" several times, in connection with voter power overlap.

Imagine two raw pizzas, where all the curls of shredded cheese are voting on which 3 pepperonis should represent them. On one pizza, the voting method gives each slice of 1/3 the power to elect one pepperoni from around the center of the slice. The cheese curls all have equal voting power, and the average distance between a cheese curl and its representative pepperoni is about as low as it can possibly be.

Now, on the other pizza, one of the 1/3 slices gets to elect its pepperoni, as before. But the other two slices combine to elect two pepperonis, both close to their combined center of gravity. Since those two pepperonis are both farther from the center of the individual slices, the average distance between a cheese curl and its representative pepperoni is higher. Clearly, this is a worse outcome, even though the RP is still perfectly even.

Clearly, what I'm driving at is that a voter's degree of choice is somehow related to the average "distance" between a voter and their representative. But just using average distance would mean that the quality of elections would vary between a small pizza and a large one, even if the outcome was exactly homologous. So instead, we'll define the voter choice for one voter i using a normalized/standardized quantity: the portion rho(v_i, r_j) of other voters who are farther from i's representative than i is.

That's all very well and good for artificial examples like the pizza, where we can somehow know exactly where each voter and candidate is located in ideology space. In the real world, though, that's impossible, for numerous reasons. So we'll have to use the ballots to infer distances, or at least, to infer the expected/average distance.

In order to make such inferences, we will have to make assumptions about the voter distribution. Perhaps the most fundamental of those assumptions, the one which will inform all the others, is our belief about the effective dimensionality of the voter space. In this case, I believe that it's best to assume one-dimensionality; this is not necessarily realistic, but unlike any other dimensionality, it leads to reasonably-obvious and easy-to-visualize ways to make the rest of the assumptions. Also, it tends to be "friendly" to voting methods, giving a rough "best-case" analysis; I think this is fine.

Note that we've defined a voter's choice as the percent of voters that are farther from the candidate, not closer, so that it's 1 at best and 0 at worst.

Adding a digression: The definition we've given, in terms of "proportion of voters who are farther from the candidate", can be thought of as candidate-centric; that is, assuming affinity between candidates and voters is symmetric, it prioritizes situations where every candidate gets "their favorite voters". But that is not precisely what we'd want, in an ideal world. For instance, in a one-dimensional ideological space with voters distributed uniformly from 0 to 100, it does not distinguish between a two-winner election where the winners are at 0 and 100, from one where they're at 25 and 75. Note that in this example, the problem arises when the candidates align with the most-extreme fraction of voters. With more winners, the number of winners in this potentially-extreme position would be lower.

There are two obvious ways we might try to address this, but either one has issues, so by default we'll use neither.

The first way would be to make some rough "parametric" dimensional assumptions to back out from "number of voters that are farther from the given candidate than the given voter is" to "distance between the given candidate and the given voter". For instance, if ideology space is 2-dimensional, we might assume the distance is (1-rho(v_i,r_j))^(1/2), because the number of voters who can fit in the space closer to the candidate is proportional to the square of the distance. In general, then, for d dimensions, and subtracting from 1 so that 100% is good, this would be 1-(1-rho(v_i,r_j))^(1/d). We could also use d=0.5, not to represent "half of a dimension", but merely to get a kind of squared-distance metric that best favors finding a representative at the average ideology for their constituents. Still, all of this relies on some strong assumptions, and so we'll use d=1 unless otherwise stated.

The second way would be to try to get a voter-centric measure; instead of "proportion of voters who are farther from the given candidate", to measure "proportion of candidates who are farther from the given voter". The problem with this is, it depends on the distribution of candidates. This makes it completely unworkable for real-world elections; but for in-silicio monte carlo experiments, we can simply create a candidate for each voter, ensure that all those candidates are ideal in terms of any non-ideological dimensions that all voters agree on, and then count. Note that while this counting itself will not be affected by the dimensionality of the ideological space, in order to do it, we will have had to create an entire in silicio election, with its own ideology space of some arbitrary dimension. Also note that higher-dimensional ideological spaces will require exponentially more seats in the legislature to get the effective choice above a given fixed threshold, even using an ideal voting method; this is not true for the candidate-centric view, as voronoi diagrams are possible in any dimensionality of space.

So, define VC_d(v_i, r_j) = 1-rho(v_i, r_j)^(1/d).

For example, say we had an IRV (aka single-winner ranked choice voting) election with 4 candidates and the following votes:

30 ballots: A>B>C>D

10 ballots: B>A>C>D

5 ballots: B>C>A>D

5 ballots: B>C>D>A

9 ballots: C>B>A>D

11 ballots: D>C>A>B

30 ballots: D>C>B>A

The outcome is that A wins, after C and B are eliminated. To find the average VC_1 of each group, we'd begin by lining up the groups from "closest to A" to "farthest from A". Using the simple assumption that voters are closer to A the higher preference they give, and on average the same distance when the preference level is the same, gives the following order:

A>B>C>D (30 ballots total; so the group has an average VC_1 of 85%)

B>A>C>D (10 total, so group average VC_1 of 65%

B>C>A>D, C>B>A>D, and D>C>A>B (25 total, so cross-group average VC_1 of 47.5%)

B>C>D>A and D>C>B>A (35 total, average VC_1 of 17.5%)

Average Voter Effective Choice: a combined proportionality metric

We now have all the ingredients necessary for making an overall metric for multi-winner outcomes, which I'm dubbing "Average Voter Effective Choice" (AVEC).

To begin with, note that "standard deviation of RP" is monotonically related to "average squared load", which in turn could be viewed as "weighted average lobbying power". That is to say, average voting power, where the average is itself weighted by voting power.

Imagine a representative who was unsure how to vote on some question, so she decided to "ask a constituent". She draws a voter at random, with probability proportional to that voter's retrospective power to elect her, then listens to them. Furthermore, say that the legislator as a whole let a randomly-chosen representative have dictatorial say over this issue. So ultimately, one voter will be deciding, and the question is: what is the expected total retrospective voting power of that voter?

For instance, if half the voters split the voting power evenly, then the "average squared load"/"weighted average lobbying power" is 2²/2=2. The people with power, have twice as much power on average as if everybody were equal. And in this simple case, the reciprocal of that is the number of wasted votes.

What I'm saying is: we can think of the reciprocal of the average squared load as being the "Average Voter Effectiveness" (AVE). If AVE is, say, 60%, that about the same amount of overall inequality as if 40% of votes had been wasted.

We can then combine this with another form of vote wastage: for those votes that did have an impact, how much choice did they really have? How of the ideology space were they able to effectively screen out? This is VC. To avoid double-counting wasted votes, we can take the average VC, weighted by total voting power.

Thus, the overall metric is Average Voter Effective Choice (AVEC), which is the product from multiplying Average Voter Effectiveness (AVE) times Average Voter Choice (AVC).

For instance, in a single-winner plurality election won by a bare majority, half the voters split the voting power equally, so average voter power is just over 50%; and those whose votes do matter, on average like the winner ideologically more than 75% of the other voters, so that's the average voter choice. Multiplying, we get 37.5% for the overall average vote effectiveness.

Where to, next?

So far, I've spent about 10,000 words to (mostly) define a feasible metric for evaluating the quality of voting methods. Next, I'm going to look at some actual multi-winner voting methods and try to estimate how they'd do by that metric. Note that this will involve some rough-and-ready approximations, but I believe it will at least be possible to class methods into at least two quality levels (with gerrymandered first-past-the-post far behind most alternatives). I also hope it will be possible to give numbers that undermine some specific bad arguments that people use in this area.

AVEC in single-winner plurality

To warm up, let's calculate AVEC for some simple scenarios under single-winner plurality.

First off, the simplest possible case: a 2-candidate election. The winner will have X>.5 (that is, over 50%) of the votes. Retrospective voting power will be of course be 0 for the losing votes. For each of the winning ones it will be 1/X; that is, between 1 (for a unanimous outcome) and just under 2 (for the barest majority). That means average voter power is between .5 and 1. Average voter choice for represented voters will be between 50% (unanimous) and 75% (bare majority). All told, average voting effectiveness will be between 37.5% (bare majority) and 50% (unanimous).

Now, consider a 3-way race in which no candidate has a majority, and the margin between 2nd and 3rd place is more than twice the difference between 1st place and 50%. For an example, you could imagine a simplified version of the US 2000 presidential election, with national popular vote and Gore 49%, Bush 48%, and Nader 3% (thus, Gore wins). Imagine all the votes start out "face down", and we turn them "face up" one at a time in a random order. As we do so, set variables G/B/N to be the face-up Gore/Bush/Nader votes as a percent of the total, and D to be the face-down votes. Gore is guaranteed to win iff G>B+D. Turning over a Gore vote increases the left hand side of that inequality by 1 (LHS+1) and similarly RHS-1. Turning over a Nader vote is LHS= and RHS-1. Turning over a Bush vote is LHS= and RHS=.

So, obviously, the Bush voters have no retrospective power to elect Gore. I still have to do the math more carefully, but I'm pretty sure that as the total number of voters grows, the retrospective power of the Nader voters converges to precisely one half that of the Gore voters, just as their power to move the difference betwen the LHS and RHS totals is half as big. (I've worked out one simple example fully but I still don't have an inductive proof of this). Thus a Gore voter would have retrospective power 1/.505, and a Nader voter, 1/1.01. This leads to an AVE of 51.26%. (1/(.49 * 1/.505^2 + .03 * 1/1.01^2)).

The effective choice depends on how often a Bush voter is closer to Gore than a Gore voter is. In the "median voter theorem" case, where Gore and Bush were both ideologically close to the threshold between them, the average voter choice for Gore voters would be .53 and .06 for Nader voters. If Gore were ideologically like his median supporter, it would be .755 and .48. And if Nader voters were all "left wing" and Gore was left of his median supporter, it would be .765 and .505. Using the second (middle) assumptions, the overall AVEC would be 38.49% (1/(.49 * 1/.505^2 + .03 * 1/1.01^2)) * (.49*1/.505^2*.755 + .03 * 1/1.01^2 * .48)/(.49 * 1/.505^2 + .03 * 1/1.01^2).

None of this is at all surprising, but it's comforting to be able to put it on a rigorous footing.

And some of the conclusions even for single-winner plurality might be a bit more surprising than that. For instance, in a 3-way election with totals 48%, 26%, 26%, voters for both of the losing candidates count as having had retrospective power to elect the winner; 1/2.44, or 1/4 as much as the winning voters, to be exact. (When the losers are nearly but not precisely tied, the power of those voters depends on the overall size of the electorate in a way that makes calculating exact powers tedious. The larger losing group will range somewhere between 1/4 and 1/2 the power per voter as the winning group.)

Real-world multi-winner voting methods (under construction)

I'm going to come back to all the ideas above. But it's finally time to discuss some specific multi-winner voting methods.

Single-seat plurality (aka separate districts, as in US House, or parliament in Canada/UK/India):

This is not a proportional method, but we can still approximate its AVE.

Let's say that the two-way vote share by district is drawn from a beta(3,3); this is a rough by-eye approximation to a few years of recent US data. That means that the average winning two-way margin is about 30 points, with a standard deviation of 10 points; that is, most districts are safe, with winning vote totals of around 65% or more. That would mean that an average district has 65% of voters with 1/.65=1.54 retrospective power, and 35% wasted votes with 0 voting power. At best, those 65% have an average 67.5% effective choice, distributed uniformly from 100% down to 35%. This multiplies out to an overall AVEC of 44%. (Note that it turns out that all the main steps in this example are linear, so the AVEC of the average precinct is the same as the average AVEC of all precincts).

Single transferrable vote (STV):

Each voter ranks the candidates in preference order. We define a quota, the number of votes you must beat to win; this may be either the Hare quota (tends to break ties in favor of smaller parties, because larger parties use up more votes on their early seats), or the Droop quota (opposite of above). Votes are always tallied for their top non-eliminated choice.

If a candidate has at least one quota of votes, they are elected (and thus "eliminated" from further consideration), and 1 quota worth of their tallied votes are "exhausted" (removed from all later tallies). This has the effect of transferring their excess votes.

If no candidate has a quota, and there are still seats left to fill, then the candidate with the lowest tally is eliminated (and so their votes are transferred).

To understand how the components of average vote effectiveness would work with STV, let's take a simple scenario, with S=9, and using the Droop quota (10% in this case) as is most customary. Say the candidates are the capital letters, and we have the following ballots:

55%: A>B>C>D>E>F>G>H>.... (alphabetical)

36%: Z>Y>X>W>V>U>... (reverse alphabetical)

9%: E>T>A>O>I>N>.... (in order of the Linotype keyboard, based on letter frequency in English)

STV elects A and Z immediately, since both have more than a quota. Excess votes spill over to B and Y, also elected; then C and X, also elected; then W and D, of whom only D has a full quota; then E; and finally F. At this point, ABCDE and ZYX have been elected (5 seats from the beginning of the alphabet and 3 from the end), and F, W, and T have 5%, 6%, and 9%, respectively. At this point, all other letters are eliminated because their tallies are 0. Then F is eliminated, and those votes transfer to the next surviving candidate in alphabetical order, T, who then has a quota and gets the last seat. So the final winner set is ABCDETXYZ.

For candidate A, any 10% from the alphabetical voters would be a sufficient set, so those voters all had 1/(9*.55) power to elect. For candidate B, any 20% of that group would be a sufficent set, so those voters had the same 1/(9*.55) power to elect. Similarly for candidates C and D (with 30% and 40% being the SS magnitude). So far, those voters have 4/9 * 1/.55 RP.

For candidate E, any set of 50% from the combined alphabetical and frequentist voters would be sufficient, so those voters each get 1/(9 * .64).

For candidate T, characterizing sufficient sets is a bit more complex. To elect T alone, a sufficient set would be simultaneously 60% from the combined alphabetical and frequentist voters, and 40% from the combined anti-alphabetical and frequentist voters; for instance, 52% alpha, 32% anti-alpha, and 8% freq.

But those sets are at least 91% in total magnitude. Can we get a smaller number of sufficient voters by expanding the set of "similar" candidates? Yes! To elect some member of the set {F...U} (that is, to ensure W doesn't win the last seat), it suffices to have one quota (10%) of votes from the alphabetical or frequentist camps remaining after 5 alphabetical candidates have been elected. That means a total of 60% from that 64% of voters. Similarly, to elect some member of {G...V}, it would suffice to have 40% from the 45% combined anti-alphas and freqs. Since those are the smallest "sufficient" voter sets for a candidate set including T, we use that for our power calculation.

Assigning RP to the anti-alphabetical voters for ZYX is straightforward as for ABCD; those voters get a total of 3/9 * 1/.36.

This leaves the following RPs:

55% Alphabetical: 4/9*1/.55 + 1/9 * 1/.64=0.98

36% Anti-alphabetical: 3/9*1/.36 + 1/9 * 1/.45=1.17

9% Frequentist: 1/9*1/.64 + 1/9 * 1/.45=0.42

I think that, given these votes, this method, and this outcome, this assignment of voter power is not quite ideal; I'd prefer an algorithm which gives the frequentists more credit for getting their second-choice T than it gives to the anti-alphabeticals for ensuring F loses. I can even imagine how to tweak the RP definition to improve that by explicitly reducing overlap in assigned voter power whenever possible. But I think this is close enough; the extra complexity of such tweaks (both in terms of definition, and, more crucially, in terms of computability) would not be worth it.

So this is 84.8% AVE (1/(.55*.98^2 + .36*1.17^2 + .9*0.42^2)).

Including the average voter choice, we get 1/(.55*.98^2 + .36*1.17^2 + .9*0.42^2) * (.55*.98^2*(1-.55/2) + .36*1.17^2*(1-.36/2) + .9*0.42^2*(1-.09/2))/(.55*.98^2 + .36*1.17^2 + .9*0.42^2) = 67.4% AVEC overall. (The formula I gave there took some shortcuts, neglecting to separately count the EC for the small slice of voting power that the anti-alphabeticals had to elect T, but this will not materially change the result.) This is already substantially better than the 44% we saw for single-seat districts. And note that, for simplicity, we used a scenario with large blocks of party-line voters; if, instead, the voters had a more-realistic variety in their first-choice preferences while still grouping into the same three overall blocs, AVEC might be as high as 80.5%.

Closed list proportional (a la Israel)

In this voting method, voters just choose a party. Each party which gets more than some minimum threshold of votes is assigned seats proportional to their votes. To decide who gets those seats, each party has pre-registered an ordered list, so if they get four seats, those go to the first four people on their list.

Let's use the April 2019 Israeli legislative election as an example. I chose this one because it shows the effect of the threshold more than the two more-recent "rematch" elections; so the estimate of vote wastage that I get will be higher than it could have been. Here's a spreadsheet calculating Average Voting Effectiveness (lower right cell) for all three of those elections.

The results are:

Several notes:

Phragmen's method:

Ebert/Phragmen method:

Warren Smith's "phi voting":

MMP:

DMP:

Sections I'd originally planned to write, but have now decided not to:

Real-world proportional voting reform (empty)

Gerrymandering and "Fair Maps" (empty)

Problems with proportionality: Israel, Italy, etc. (empty)

Israel: Too Many Parties (empty)

Italy: Unstable rules (empty)

Wales: strategic voting / clone parties (empty)

Poland: high (party) thresholds (empty)

Godwin violation: not actually relevant (empty)

Novel methods (empty)

Modified Bavarian MMP (empty)

PLACE (empty)

Broad Gove (empty)

Prospects for change in US (empty)

Medford, MA (empty)

How you can help (empty)

Canada, UK, India... (empty)

Post-Mortem on this article (incomplete)

... This is the one part I still haven't finished.

... and I've decided it will happen in a separate document.

53 comments

Comments sorted by top scores.

comment by Thomas Sepulchre · 2020-07-17T12:32:47.309Z · LW(p) · GW(p)
NESS answers the question of who shares responsibility, but it doesn't answer that of how much responsibility they have. For instance, imagine that a group of people made stone soup with 1 stone, 1 pot, water, fire, and ingredients; and that in order to be a soup, it needed a pot, water, and at least 3 ingredients. NESS tells us that the person who brought the stone was not responsible for the soup, and that everyone else was; but how do we divide responsibility among the others? Simple symmetry shows that each ingredient gets the same responsibility, but there are many facially-valid ways you could set relative responsibility weights between an ingredient and the pot.

I think there's actually one way to set relative responsibility weight which makes more mathematical sense than the others. But, first, let's slightly change the problem, and assume that the members of the group arrived one by one, that we can order the members by time of arrival. If this is the case, I'd argue that the complete responsibility for the soup goes to the one who brought the last necessary element.

Now, back to the main problem, where the members aren't ordered. We can set the responsibility weight of an element to be the number of permutations in which this particular element is the last necessary element, divided by the total number of permutations.

This method has several qualities : the sum of responsibilities is exactly one, each useful element (each Necessary Element of some Sufficient Set) has a positive responsibility weight, while each useless element has 0 responsibility weight. It also respects the symmetries of the problem (in our example, the responsibility of the pot, the fire and the water is the same, and the responsibility of each ingredient is the same)

In a subtle way, it also takes into account the scarcity of each ressource. For example, let's compare the situation [1 pot, 1 fire, 1 water, 3 ingredients] with the situation [2 pots, 1 fire, 1 water, 3 ingredients]. In the first one, in any order, the responsibility goes to the last element, therefore the final responsibility weight is 1/7 for each element. The second situation is a bit trickier, we must consider two cases. First case, the last element is a pot (1/4 of the time). In this case, the responsibility goes to the seventh element, which gives 1/7 responsibility weight to everything but the pots, and 1/14 responsibility weight to each pot. Second case, the last element is not a pot (3/4 of the time), in which case the responsibility goes to the last element, which gives 1/6 responsibility weight to everything but the pots, and 0 to each pot. In total, the responsibilities are 1/56 for each pot, and 9/56 for each other element. We see that the responsibility has been divided by 8 for each pot, basically because the pot is no longer a scarce ressource.

Anyway, my point is, this method seems to be the good way to generalize on the idea of NESS : instead of just checking whether an element is a Necessary Element of some Sufficient Set, one must count how many times this element is the Necessary Element of some permutation.

Replies from: jameson-quinn, jameson-quinn
↑ comment by Jameson Quinn (jameson-quinn) · 2020-07-17T18:13:00.364Z · LW(p) · GW(p)

Nice. Thank you!!!

This corresponds to the Shapley-Shubik index. I had previously discounted this idea but after your comment I took another look and I think it's the right answer. So I'm sincerely grateful to you for this comment.

↑ comment by Jameson Quinn (jameson-quinn) · 2020-07-20T21:05:55.734Z · LW(p) · GW(p)

I rewrote the article to incorporate your contribution. I think you'd be interested to read what I added afterwards discussing this idea.

Replies from: Thomas Sepulchre
↑ comment by Thomas Sepulchre · 2020-07-26T08:19:55.407Z · LW(p) · GW(p)

Thanks a lot!

comment by Ben Pace (Benito) · 2020-06-20T20:19:25.950Z · LW(p) · GW(p)

This is a great idea. I'd be excited for you to put together a post over time in public.

My initial suggestions for how to do this:

  • Change the title to say something like "Ongoing thoughts on proportional voting methods" or "[In progress] Proportional Voting Methods". Also add a note to the top in italics that says something like "This post is a stub, I (Jameson) am updating it over the course of many weeks/months, if you read it and have questions please leave a comment."
  • When you make substantive edits, leave a comment with a few bullets saying what you did, which will appear in recent discussion.
  • Once or twice, when you've finished a substantial addition of work (a whole new section, or a big restructuring), you can ping me and I'll make it re-appear on the frontpage like a new post, for people to look over anew.

Here are my thoughts on your assumptions:

  • I think the first assumption is pretty good for writing down your thoughts. I'll add to it that I think you should update a bit in response to discussion. If someone says "What exactly are you using consistency to show here?" answer them but don't make a big edit to the post to explain it from first principles. If 5 people ask then add a paragraph or two of intuitive explanation before using it.
  • While democracy has clearly been an incredibly powerful invention, I'm not ultimately wedded to it, and can imagine many variants being better (futarchy is a nearby example). But as you say, I still expect to find this very interesting and useful.
Replies from: jameson-quinn, jameson-quinn, jameson-quinn
↑ comment by Jameson Quinn (jameson-quinn) · 2020-07-20T20:57:06.928Z · LW(p) · GW(p)

Ping 2.5 of 3; that is, unexpectedly, I got input that superceded the old ping 2 of 3, and I've now incorporated it.

Replies from: Benito
↑ comment by Ben Pace (Benito) · 2020-07-20T21:37:28.829Z · LW(p) · GW(p)

I think that the essay is big enough and was very recently resubmitted such that people who re-read it will want a guide to the bits updated most recently, and otherwise will bounce off.

My guess is that for future resubmissions it would be good at the top to have a brief list of the main changes since it was last resubmitted. Something like:

---

List of updates since this was last submitted to the frontpage:

  • Section B: Redrew the diagram and gave a better explanation
  • Section D: Minor edits to the framing
  • Section E: Total rewrite
  • Section F: new section added!

--break--

Essay begins.

---

My current guess is that the main change is where you quote Thomas's solution and your discussion of it? That only needs a short thing added to the top, but I think it's probably important.

Can you try that?

Replies from: jameson-quinn
↑ comment by Jameson Quinn (jameson-quinn) · 2020-07-21T13:43:28.285Z · LW(p) · GW(p)

Done.

Replies from: Benito
↑ comment by Ben Pace (Benito) · 2020-07-22T02:47:04.209Z · LW(p) · GW(p)

Resubmitted. Pardon the delay.

↑ comment by Jameson Quinn (jameson-quinn) · 2020-07-16T14:07:47.052Z · LW(p) · GW(p)

Ping 2 of 3

Replies from: Benito
↑ comment by Jameson Quinn (jameson-quinn) · 2020-07-02T13:20:31.768Z · LW(p) · GW(p)

If I get three "pings", now would be one. I think this has progressed far enough that a reader would begin to see the big ideas I'm driving at, and certainly far enough that I'd be ready for structural critiques. But if I only get two "pings", then no; I still haven't gotten to the biggest new idea or to either of the fun bits of historical trivia, and I'd want my two pings to be "at least two of those three" and "all done".

Replies from: Benito
↑ comment by Ben Pace (Benito) · 2020-07-03T06:10:20.944Z · LW(p) · GW(p)

I have "pinged" it. I also added a note to the top that I feel will help people orient, though you're welcome to edit it.

comment by Jameson Quinn (jameson-quinn) · 2020-06-21T23:03:16.467Z · LW(p) · GW(p)

Added a few paragraphs, but I still haven't gotten to the interesting stuff; just still laying the groundwork.

Replies from: jameson-quinn
↑ comment by Jameson Quinn (jameson-quinn) · 2020-06-22T20:08:54.291Z · LW(p) · GW(p)

More, but overall status same.

Replies from: jameson-quinn
↑ comment by Jameson Quinn (jameson-quinn) · 2020-06-23T19:40:26.346Z · LW(p) · GW(p)

Added second section near top to explain where I'm intending to go with all of this.

Replies from: jameson-quinn
↑ comment by Jameson Quinn (jameson-quinn) · 2020-06-24T15:09:01.865Z · LW(p) · GW(p)

Added a section on God's proportional voting method.

Replies from: jameson-quinn
↑ comment by Jameson Quinn (jameson-quinn) · 2020-06-27T22:11:26.444Z · LW(p) · GW(p)

Began a section on God's MSE decomposition, but it's very very sketchy still.

Replies from: jameson-quinn
↑ comment by Jameson Quinn (jameson-quinn) · 2020-07-01T15:37:28.297Z · LW(p) · GW(p)

Tightened up the "God's MSE decomposition" section. Almost ready for the big reveal of how this connects to a larger philosophy of assigning responsibility, in a way that I think could be of interest far beyond voting.

Replies from: jameson-quinn
↑ comment by Jameson Quinn (jameson-quinn) · 2020-07-02T13:16:33.760Z · LW(p) · GW(p)

Starting to get to the fun ideas! Hooray! But I'm probably still probably only about halfway done with the "theory" section, which means I'm probably only 30-40% done with the entire thing.

Replies from: jameson-quinn
↑ comment by Jameson Quinn (jameson-quinn) · 2020-07-03T15:03:20.193Z · LW(p) · GW(p)

A bit of polishing of existing parts; skeleton headings for remainder

Replies from: jameson-quinn
↑ comment by Jameson Quinn (jameson-quinn) · 2020-07-04T02:43:16.632Z · LW(p) · GW(p)

Began the "allocating responsibility" section.

Replies from: jameson-quinn
↑ comment by Jameson Quinn (jameson-quinn) · 2020-07-06T17:54:31.300Z · LW(p) · GW(p)

Fleshing out "allocating responsibility" and beginning "Pascal's other wager".

Replies from: jameson-quinn
↑ comment by Jameson Quinn (jameson-quinn) · 2020-07-15T03:27:10.764Z · LW(p) · GW(p)

Beginning "a proposed solution" but it's slow going — haven't actually gotten to the solution part of the solution yet. Also wrote a bit more above on what "MSE" means in this context — it's less intiutive than you might think when you first read the acronym.

Replies from: jameson-quinn
↑ comment by Jameson Quinn (jameson-quinn) · 2020-07-16T14:05:46.176Z · LW(p) · GW(p)

OK, I think I'm "over the hump". That is, there's still plenty left to write and plenty of details left to work out, but I think that an ideal reader could begin to see the light at the end of the tunnel, could begin to grasp the overall scale and shape of what I'm trying to do here. I'm going to ping again.

Replies from: jameson-quinn
↑ comment by Jameson Quinn (jameson-quinn) · 2020-07-20T20:55:48.321Z · LW(p) · GW(p)

Rewritten to reflect Thomas Sepulchre's contribution. Which is awesome, by the way.

Or in other words...

V 0.5.1: the main changes since the previous version 0.5.0 are a complete rewrite of the "Tentative Answer" section based on a helpful comment by a reader here, with further discussion of that solution; including the new Shorter "Solution" Statement subsection. I also added a sketch to visualize the loss I'm using.

Replies from: jameson-quinn
↑ comment by Jameson Quinn (jameson-quinn) · 2020-07-22T00:50:17.783Z · LW(p) · GW(p)

Version 0.5.2: Added "Equalizing voter power" and "Measuring "Representation quality", separate from power" sections.

Replies from: jameson-quinn
↑ comment by Jameson Quinn (jameson-quinn) · 2020-07-22T17:23:15.027Z · LW(p) · GW(p)

V 0.5.3: Added "Bringing it all together" (85% complete)

Replies from: jameson-quinn
↑ comment by Jameson Quinn (jameson-quinn) · 2020-07-23T15:13:51.553Z · LW(p) · GW(p)

V 0.5.4: Meaningful rewrite to "Shorter "solution" statement", which focuses not on power to elect an individual, but power to elect some member of a set, of whom only 1 won in reality.

Replies from: jameson-quinn
↑ comment by Jameson Quinn (jameson-quinn) · 2020-07-25T13:13:14.441Z · LW(p) · GW(p)

V 0.5.5: wrote a key paragraph about NESS: similar outcomes just before Pascal's Other Wager? (The Problem of Points). Added the obvious normalizing constant so that average voter power is 1. Analyzed some simple plurality cases in Retrospective voting power in single-winner plurality.

Replies from: jameson-quinn
↑ comment by Jameson Quinn (jameson-quinn) · 2020-07-27T15:26:20.119Z · LW(p) · GW(p)

V 0.6.0: Coined the term "Representational Fairness" for my metric. Did a worked example of Single transferrable vote (STV), and began to discuss the example. Bumping version because I'm now beginning to actually discuss concrete methods instead of just abstract metrics.

Replies from: jameson-quinn
↑ comment by Jameson Quinn (jameson-quinn) · 2020-07-30T15:51:50.216Z · LW(p) · GW(p)

V 0.7.0: Switched from "Representational Fairness" to the more-interpretable "Vote Wastage". Wrote enough so that it's possible to understand what I mean by VW, but this still needs revision for clarity/convincingness. Also pending, change my calculations for specific methods from RF to VW.

Replies from: jameson-quinn
↑ comment by Jameson Quinn (jameson-quinn) · 2020-08-01T21:39:06.893Z · LW(p) · GW(p)

V 0.7.1: added a digression on dimesionality, in italics, to the "Measuring "Representation quality", separate from power" section. Finished converting the existing examples from RF to VW.

Replies from: jameson-quinn
↑ comment by Jameson Quinn (jameson-quinn) · 2020-08-04T21:43:50.878Z · LW(p) · GW(p)

V 0.7.2: A terminology change. New terms: Retroactive Power, Effective Voting Equality, Effective Choice, Average Voter Effectiveness. (The term "effective" is a nod to Catherine Helen Spence). The math is the same except for some ultimately-inconsequential changes in when you subtract from 1. Also, started to add a closed list example from Israel; not done yet.

Replies from: jameson-quinn
↑ comment by Jameson Quinn (jameson-quinn) · 2020-08-08T16:26:16.602Z · LW(p) · GW(p)

V0.7.3 Still tweaking terminology. Now, Vote Power Fairness, Average Voter Choice, Average Voter Effectiveness. Finished (at least first draft) of closed list/Israel analysis.

Replies from: jameson-quinn
↑ comment by Jameson Quinn (jameson-quinn) · 2020-10-01T20:16:45.785Z · LW(p) · GW(p)

V0.9: I decided to make a few final edits (mostly, adding a summary and a post-mortem section, which is currently unfinished) then freeze this as is.

(This was done some time ago but I forgot to post it here.)

Replies from: jameson-quinn
↑ comment by Jameson Quinn (jameson-quinn) · 2020-10-01T20:17:09.295Z · LW(p) · GW(p)

V0.9.1: Another terminology tweak. New terms: Average Voter Effectiveness, Average Voter Choice, Average Voter Effective Choice. Also, post-mortem will happen in a separate document. (This same version might later be changed to 1.0)

comment by ChristianKl · 2020-07-03T17:21:12.593Z · LW(p) · GW(p)

In practice, it's impossible to create a voting method that elects S equally-weighted candidates without wasting some votes; typically at least somewhere between 1/2S and 1/(S+1) of them. 

This is a bit hard to read. Maybe write instead of 1/2S and 1/(S+1) of them a formula that depends on the V the number of voters and that's V/(2S) and V/(S+1). 

Replies from: jameson-quinn
↑ comment by Jameson Quinn (jameson-quinn) · 2020-07-06T17:55:53.041Z · LW(p) · GW(p)

Thanks, fixed. (A numerator of 1 gives an answer in dimensionless units of "fraction of voters"; using V gives units of voters. I tend to prefer the former but I agree that at first read the latter is more intuitive.)

comment by Pattern · 2020-07-03T15:30:49.499Z · LW(p) · GW(p)

(Currently complete, but subject to change with post - unless different versions are separate posts?)

Content:


Contents:

Content

  • Open
  • Addressed by document's later contents

Style


(Numbering is within sections and only reflects order within text (appearance), but doesn't cross categories.)


Open:

1.

In a statistical sense, that would mean the loss function would be something like: the expected squared difference between the percent a proposal gets in the legislature, and the percent it would have gotten in a direct referendum, given that the proposal is drawn from some predefined distribution over possible proposals.

I'm curious how this loss function differs from the results being the same, rather than the ratios. I'd guess that it would get smoother results, but the main issue would probably be interactions. (Systematic rather than random variance.)


Which brings us to the third way better outcomes might be possible: by making representatives accountable primarily not on a moment-by-moment basis, but on a once-per-election time cycle, we might nudge them to think from a slightly more future-oriented perspective.

The question of how to get a future oriented perspective seems an empirical one.


2.

For example, imagine the fictional planet mentioned in The Hitchhikers' Guide to the Galaxy, where people vote to elect lizards. Though the complete lack of representativity makes this a poor democracy, it would clearly be even less democratic if the losing lizard committed electoral fraud.

Obligatory mention that gerrymandering might be detectable by measuring variance in outcomes (of elections).


If the "effective dimension" is low, then a relatively small legislature can do quite a good job at representing even an infinite population.

But the election would take forever.


If the effective dimension is high, though, then the ideological distance between a randomly-chosen voter and the closest member of the legislature, will tend to be almost as high as the distance between any two randomly-chosen voters. In other words, there will be so many ways for any two people to differ, that the very idea that one person could "represent" another well begins to break down (unless the size of the legislature grows exponentially in the number of effective dimensions)

Unless people are organized into groups based around a group that literally negotiate over the tradeoffs, possibly reversing the 'usual' direction of voters -> legislature. With the last step being the one that makes the Laws.


3.

What is the effective dimension of political ideology in reality? It depends how you measure it. If you look at US incumbent politicians' voting records using a methodology like DW-NOMINATE

The choice of dimensions seems like a bigger deal than the number.


But if I'm wrong, and the variation in political opinions/interests that are politically salient in an ideal world is much higher, then the very "republican idea" of representative democracy is problematic.

And the question of whether things (like the factors captured by the dimensions, as well as 'representability') interact with each other to greatly affect outcomes (laws or whatever the means in question is that itself leads to outcomes).


4.

Having individual representation, where each voter can point to the representative they helped elect, and each representative was elected by an equal number of voters, is good from several perspectives.

Why? Getting a more diverse set of perspectives seems doable via:

Electing one or more officials with different numbers of votes, but giving them power based on the number of votes they received. (Officials with less power may have less of an impact most of the time.* But if they're able to bring new ideas to the table that work...)


*Ignoring the possibility of officials with powers based on circumstance, or within certain domains.

For instance, in this case, we could optimize for piety even while also ensuring unbiasedness and minimizing variance.

Variance seems like it might be a red herring, given a focus on outcomes/exemplariness.


5.

3a from the next section seems like it might be relevant.

In practice, it's impossible to create a voting method that elects S equally-weighted candidates without wasting some votes; typically at least somewhere between 1/2S and 1/(S+1) of them.

Imprecision: instead of dividing voters into S separate groups with one representative per group, the method might divide them into fewer groups with more than one representative each. This almost certainly would lead to higher bias and/or variance at the next step.

Weird methods:

Representatives are chosen from Group A, but chosen by Group B voters. (There's also changing power amounts as mentioned previously.)


6.

Note that in rating a voting method, we're doing this "backwards": we constructed the "set of responsible voters" from the candidate, not vice versa.

In some ways there seemed to be a similar circularity around the use for "utilitarianism" whcih seemed to mean 'representativeness of democracy' in context.


.... (I'm making good progress writing this but I've gotta go now; to be continued)

I look forward to seeing more in this section.


Addressed by document's later contents:

1.

When you're choosing between a finite number of options that are known at the time of voting, you need a single-winner voting method. It's pretty easy to define a way to measure "how good" such a method is: we can just use utilitarianism.

utilitarianism

Based on the rest of this document, you mean a utility function*, or something else very different from utilitarianism.


This (later) part makes it more clear:

The legislative outcome is an approximation of the popular outcome which is an approximation of the ideal outcome; if the first step of approximating goes wrong, there's no reason to believe the second step will fix it.

2.

But electing a legislature is something different. How do you define the utility of a legislature that has a 60/40 split between two parties, versus one that has 70/30 split?

Depends on the system - The utility is based on the actions of the legislature (and also it's cost and thus size, especially if the people/voters have to pay for it).


(Later) Addressed here:

The legislative outcome is an approximation of the popular outcome which is an approximation of the ideal outcome; if the first step of approximating goes wrong, there's no reason to believe the second step will fix it.

3.

In practice, "all" tends to be limited by some eligibility criteria, but I'd say that most such criteria make the system less democratic, other than maybe age and residency.
The normative case that this kind of democracy is a good idea tends to rest on the Condorcet jury theorem: that is, if the average voter is more likely to be "right" than "wrong" (for instance, more likely to choose the option which will maximize global utility), then the chances the referendum outcome will be "right" quickly converge towards 100% as the number of voters increases.

a) Voters could also be selected to 'create/represent' conflict, and see which side wins. (If the remaining population that hasn't voted isn't big enough to overturn the current win/result, the voting stops, or the result is passed. (An error rate/margin could also be included.))


b) This could be reversed; the voters are selected in order to create a body in which

the average voter is more likely to be "right" than "wrong"

and where additional voters improve this probability.


c)

which will maximize global utility

The question of cost is also a matter of global utility.


d)

If instead of voters voting, a body that deliberates on the question/s this can allow for more options being evaluated, and perhaps more "right" being found - though the bigger the body (and the search space), the longer this might take, and costs probably should be taken into account.


Later addressed here:

First, it may even be that there is some optimum size for reasoned debate and

(And it may be different for "exemplary representatives".)



Style:

on a fooding

on a footing


((OK, "effective dimension" isn't exactly that; it measures not only how many "relatively big" dimensions there are, but also how "relatively small" the rest are. I'm being deliberately vague about how precisely I'd define "effective dimension" because I suspect that unless you ignore variation below a certain noise threshold the ED is actually infinite in the limit of infinite voters.))
...
))(...)((

The multiple parenthesis are an interesting style choice.


The "ongoing" nature of this post is interesting and I like it. Having read it as it is, I think it worked really well, and some of the empty sections could fit together in a different post. Though since they don't exist I don't know how important the connections between them in one document are. (Maybe having this version as is might be useful.)

Replies from: jameson-quinn, jameson-quinn, jameson-quinn
↑ comment by Jameson Quinn (jameson-quinn) · 2020-07-15T14:45:33.503Z · LW(p) · GW(p)

Key to strange punctuation I use: "((...))" are technical notes that you can safely skip if you don't understand; the extra parens are to emphasize the skippability. "))(...)((" are digressions that break the narrative flow, but are actually important; the inner normal parens are to mark digression and the doubled outer reverse parens are to mark importance.

Also I wrote TLD̦R instead of TL;DR (too long; didn't read) because on my keyboard the semicolon is a dead key that combines with space or enter to be an ordinary semicolon but combines with letters to add accents. I think making TLD̦R one character shorter is in the spirit of TL;DR.

↑ comment by Jameson Quinn (jameson-quinn) · 2020-07-04T01:31:34.605Z · LW(p) · GW(p)

I've switched over to responding to your comments/suggestions one-by-one.

On your point 1, regarding loss functions: I agree that a strictly utilitarian/consequentialist PoV would care about which side won, not about vote totals. I think there are three reasons to nevertheless build a loss function around vote totals.

1. Mathematically more well-behaved. For instance, the whole "MSE decomposition" idea I bring in later would be much much messier with a binary-outcome-based loss function.

2. I believe that in practice, if there's a question where the target support would be, say, 70%, but the legislature supports it at 90%, you can probably use it to construct some other at-least-somewhat-reasonable question where the target support would be 40% but the legislature supports it at 60%. That is, in practice, errors in vote totals go hand-in-hand with errors in outcomes, even if this is not a logical necessity (at least, not without additional assumptions about convex lotteries and stuff).

3. Some of the votes/decisions of the legislature may be made in a non-majority-rules fashion. For instance, you could have some situations where each legislator gets to allocate a share of some resource. In such cases, the vote-total-based loss function is clearly correct even from a consequentialist standpoint. (This might be seen as a special case of 2, but it's different enough to list separately.)

I think that saying 1 and 2 in the main article would be too much of a digression, but I will think further about whether there's a way to include point 3.

↑ comment by Jameson Quinn (jameson-quinn) · 2020-07-03T17:51:47.722Z · LW(p) · GW(p)

Thanks for the close read and the thoughtful comments!

Open:

1. Loss function, future-oriented: interesting questions; I have thoughts on them; I left them out the first time through but may add them later. 

2. gerrymandering; "infinite"; proposal I don't fully grasp: Also interesting but I think beside the points I want to make here.

3. Dimensions.

"The choice of dimensions seems like a bigger deal than the number." I'm implicitly assuming that they're chosen by principal component analysis or something similar. Of course that's not robust to scaling or other monotone transforms, but I think it's close enough to being well-defined to handwave away for my purposes.

But if I'm wrong, and the variation in political opinions/interests that are politically salient in an ideal world is much higher, then the very "republican idea" of representative democracy is problematic.

"whether things interact with each other to greatly affect outcomes": good point. I'll see if I can incorporate it without being too wordy.

4. Weighted legislatures: yes, that's a whole topic I could write an entire section on. For now, can't afford to get that sidetracked, sorry.

"Variance seems like it might be a red herring, given a focus on outcomes/exemplariness."

Um. It's possible I'm not being clear what I mean by "variance". I don't mean variance of ideologies of legislators; I mean variance (meta-variance?) of distributions. That is to say, a 1-dimensional procedure for picking 2 legislators would have higher variance if it sometimes picked {5,5} and sometimes picked {4,6}, than if it reliably picked {0,10}.

I think there may something to your critique here aside from that possible misunderstanding, though. I have some thoughts but I'm not sure how I should or will deal with this issue. Probably, I should respond to that in a separate comment.

5.

"Weird methods:" Maybe I should explain how my existing proposal for rating voting methods would handle that? Because I have thought about it, and it is a tricky case. But I think I'll save that for later; for now, I want to stick with simpler cases.

...

to be continued; probably in a separate comment or comments, for better notification.

comment by Raemon · 2020-06-20T18:23:50.490Z · LW(p) · GW(p)

I think the link-post url is somehow malformed. (Also I initially missed this was a linkpost, you might want to link it again at the end to be clear what my next action is supposed to be when I get to the bottom of the post)

Replies from: jameson-quinn
↑ comment by Jameson Quinn (jameson-quinn) · 2020-06-20T18:37:33.157Z · LW(p) · GW(p)

This isn't a link-post, it's just incomplete as yet. I mistakenly added a link because I thought that was a way to give a URL to a draft post.

But, how did you see it? I explicitly added Ben Pace, but not you.

Basically, I want to figure out how this works; sorry for the confusion, and eventually I'll fill in actual content here.

Replies from: DanielFilan, Raemon
↑ comment by DanielFilan · 2020-06-20T21:29:25.556Z · LW(p) · GW(p)

FYI this was visible to me on the front page.

↑ comment by Raemon · 2020-06-20T21:16:46.136Z · LW(p) · GW(p)

Ah, cool. I think you intended the post to be a draft, while sharing it with Ben Pace.

[edit: oh, but an unfortunate property of draft posts is you can't leave regular comments on them. Ben should probably make it an unlisted post]

comment by ChristianKl · 2020-07-03T17:44:31.473Z · LW(p) · GW(p)

There are multiple desired properties of a voting system for a general selection.  Most systems are designed with an idea that there are legislative matters that have different effect on different geographical locations and thus the system not only wants to empower representation along the e-political dimensions but also representation along meaningful geographic areas. 

That principle seems to be important enough for people to have decided for example for elections to the European parliament that Malta gets one seat per ~100,000 citizens while Germany gets one seat per  ~865.000 citizens. 

Replies from: Dagon
↑ comment by Dagon · 2020-07-03T23:20:20.000Z · LW(p) · GW(p)

I _really_ want to believe that posting this comment multiple times was an intentional demonstration of how opinions can be unevenly weighted. If so, brilliant!

Replies from: ChristianKl
↑ comment by ChristianKl · 2020-07-08T13:20:15.087Z · LW(p) · GW(p)

It wasn't intentional, but that's a nice reading ;) I now deleted the doublicates to make discussion easier. 

comment by ChristianKl · 2020-07-03T17:07:22.309Z · LW(p) · GW(p)

If you look at US incumbent politicians' voting records using a methodology like DW-NOMINATE, you get a number between 1 and 2. If you look at all people's opinions on all possible issues, you'd surely get a much higher number. I personally think that healthy political debate would probably settle at about 𝓮 effective dimensions — that is, a bit less than 3. 

https://www.civicscience.com has data about political preferences of large numbers of citizens. It might be good to talk with them to get an idea about what e actually happens to be.

Replies from: jameson-quinn, jameson-quinn
↑ comment by Jameson Quinn (jameson-quinn) · 2020-07-03T17:52:07.296Z · LW(p) · GW(p)

I meant "𝓮" to be the number whose natural logarithm is 1 — "𝓮" is the closest to the math version of the character that I can type on my keyboard as-is. I don't have any argument for why it should be precisely that value, it's just the most famous number in between 2.5 and 3, which is roughly where I think the correct value will lie. I am aware of the larger literature on people's political preferences, but that's all from a world that's already conditioned by more-or-less non-ideal voting methods, so I don't think this is a question with an easy-to-find empirical answer like that. I may expand the section you reference to talk about these issues a bit more, and to discuss the available data/literature, but when I first wrote it, I was hurrying through to get somewhere else.

comment by Jameson Quinn (jameson-quinn) · 2020-07-29T15:25:16.009Z · LW(p) · GW(p)

I am rewriting the overall "XXX: a xxx proportionality metric" section because I've thought of a more-interpretable metric. So, where it used to be "Representational fairness: an overall proportionality metric", now it will be "Vote wastage: a combined proportionality metric". Here's the old version, before I erase it:


Since we've structured RQ_d as an "efficiency" — 100% at best, 0% at worst — we can take each voter's "quality-weighted voter power" (QWVP) to be the sum of their responsibility for electing each candidate, times their RQ_1 for that candidate. Ideally, this would be 1 for each voter; so we can define the overall "quality-weighted proportionality" (QWP) of an outcome as the average of squared differences between voters' QWVP and 1, shifted and scaled so that no difference gives a QWP of 100% and uniform zeros gives a QWP of 0. (Note that in principle, a dictatorship could score substantially less than 0, depending on the number of voters).

(To do: better notation and LaTeX)

Since realistic voting methods will usually have at least 1 Droop quota of wasted votes (or, in the case of Hare-quota-based methods, just over half a Hare quota of double-powered votes and just under half of wasted votes; which amounts to much the same thing in QWP terms), the highest QWP you could reasonably expect for a voting method would be S/(S+1).

(show the math for the QWP of the IRV example above. Key point: the D>C>A>B voters have zero responsibility for electing A, so all they do is lower the average RQ of the B>C>A>D and C>B>A>D voters)

Note that this QWP metric, in combining the ideas of overall equality and representation quality, is no longer perfectly optimizing either of those aspects in itself. That is to say, in some cases it will push methods to sacrifice proportionality, in search of better representation, in a way that would tend to hurt the MSE from God's perspective. I think those cases are likely to be rare enough, especially for voting methods that weren't specifically designed to optimize to this metric, that I'm OK with this slight mis-alignment. That is to say: I think the true ideal quality ordering would be closer to a lexical sort with priority on proportionality ("optimize for proportionality, then optimize RQ only insofar as it doesn't harm proportionality`); but I think most seriously-proposed, practically-feasible voting methods are far enough from the Pareto frontier that "optimize the product of the two" is fine as an approximation of that ideal goal.

One more note: in passing, this rigorous framework for an overarching proportional metric also helps define the simple concept of "wasted vote"; any vote with 0 responsibility for electing any winner. Although "wasted votes" are already commonly discussed in the political science literature, I believe this is actually the first time the idea has been given a general definition, as opposed to ad-hoc definitions for each voting method.