Less Wrong Polls in Comments

Vote up for YES.

Vote up for NO.

Karma balance.

Vote up for NO.

Vote up for YES.

Karma balance.

I love this poll

After one month and 120 responses, I'm considering this poll closed. The results are:

1) The third one: 21%
2) The fifth one: 15%
3) The second one: 14%
4) The first one: 32%
5) The fourth one: 18%

A chi-squared test says that these results are non-uniform, with a p-value of 0.02.

The correct answer, #5 "the fourth one", was chosen by 18% of respondents. The most common answer was #4, "the first one".

This poll idea was taken from a gamefaqs poll which was linked on LW last year. The results of that poll (which had a much larger sample size) were:

1) The third one: 17%
2) The first one: 24%
3) The last one: 21%
4) The second one: 26%
5) The fourth one: 11%

My hypothesis about that poll was:

The first option is most salient, by virtue of being first. Level 0 players will tend to choose option 1. Level 1 players will realize that this is what level 0 players will do, so they will tend to choose option 2 ("the first one"). Level 2 players will realize that this is what level 1 players will do, so they will tend to choose option 4 ("the second one"). Level 3 players will realize that this is what level 2 players will do, so they will tend to choose option 5 ("the fourth one"). Apparently there are lots of level 1 & 2 players, but very few level 3 players.

That hypothesis predicts that the most common responses on the LW poll would be the level 1 response, #4 "the first one", and the level 2 response, #5 "the fourth one". The data provide partial confirmation of this hypothesis; in terms of levels the most common responses to the LW poll were:

32% Level 1 (#4 "the first one")
21% Level 0 (#1 "the third one")
18% Level 2 (#5 "the fourth one")
15% Level 3 (#2 "the fifth one")
14% Level 4 (#3 "the second one")

And for the gamefaqs poll:
26% Level 2 (#4 "the second one")
24% Level 1 (#2 "the first one")
21% Level 4 (#3 "the last one")
17% Level 0 (#1 "the third one")
11% Level 3 (#5 "the fourth one")

Some people don't pick very good schelling points.

In the meantime, the effect can be simulated by proper selection of options. Example:

Which is your favourite superhero? [pollid:37]

This is a problem. Ideally there would be a separate button next to the "vote" button (a forum that I read has that feature, with the button labeled "View Results (Null Vote)"). Second-best would be to allow people to submit a blank vote (which is not as good, since it's not obvious to people that they have that option), but it currently does not work that way (even though the OP seems to say that it does).

I see a 'Total 123' but the table and chart only show 2 votes. The raw data also have 123 entries.

Hey everyone, I just voted, and so I can see the correct answer. The average is 19.2, so you should choose 17%!

A function to automatically compute the averages should be implemented.

java PollStat < poll.csv.txt

import java.io.*;
import java.util.*;

public class PollStat {
  public static void main(String[] args) {
    try {
      BufferedReader br = new BufferedReader(new InputStreamReader(System.in));
      String line = br.readLine();
      int[] counts = new int[12]; // PUT NUMBER OF POLL OPTIONS HERE
      int end, begin;
      int linen = 0;
      while (line != null && line.charAt(0) == '#') {
        line = br.readLine();
        linen++;
      }
      while (line != null) { // GATHER LOOP
        linen++;
        end = line.lastIndexOf("\",\"");
        begin = line.lastIndexOf("\",\"", end-1);
        try {
          line = line.substring(begin+3, end);
        } catch (Exception e) {
          System.out.println("At line number "+linen+":");
          e.printStackTrace();
          System.exit(1);
        }
        Integer c = Integer.parseInt(line);
        counts[c]++;
// For arbitrary numeric responses, don't use counts.
// just continuously gather your statistics. Alternately, make a list of Doubles or something.

        line = br.readLine();
      }
      br.close();


      int total = 0;
      int resp = 0;

      int[] numbers = new int[]{100, 64, 41, 26, 17, 11, 7, 4, 3, 2, 1, 0}; // PUT POLL OPTIONS HERE
      for (int i = 0; i < c.length; i++) {
        System.out.println(numbers[i]+": "+counts[i]);
        total += numbers[i] * counts[i];
        resp += counts[i];
      }
      System.out.println("total: "+total);
      System.out.println("average: "+((double)total)/resp);

      } catch (IOException ioe) {
        ioe.printStackTrace();
        System.exit(2);
      }
  }
}

Ahrgh, if only everyone was running TDT…

seconded

(Or even it remembering whether you chose to vote non-anonymously in the last poll you took.)

The poll seems designed for very short answers.

Feature, not bug.

Confirmed. The issue is in an interaction between the polling code and Reddit's custom ORM which causes vote-totals to be cached, but not persisted to the database correctly. I have a fix, which I'm testing now. All polls created before the fix is applied will be affected; it'll be possible to restore them, but it'd take some work which isn't a priority for me.

Here's a screenshot of that with my poll:

I still have access to the raw poll data.

I used random.org to generate my answer.

But, when I submitted it, I got the following:

First Answer 0 (0%)
Second Answer 0 (0%)
Third Answer 0 (0%)
Fourth Answer 1 (2%)
Fifth Answer 0 (0%)
Total 58 (100%)

The raw data contained all the 58 rows, however. Seems like there might be a bug in the result-rendering code.

To anyone thinking this is not random, with 42 votes in:

• The p-value is 0.895 (this is the probability of seeing at least this much non-randomness, assuming a uniform distribution)

• The entropy is 2.302bits instead of log(5) = 2.322bits, for 0.02bits KL-distance (this is the number of bits you lose for encoding one of these votes as if it was random)

If you think you see a pattern here, you should either see a doctor or a statistician.

Ha. I fail at random. In my defence, the universe is probably deterministic anyway.

After one month and 118 responses, I'm considering this poll closed. The results are:

1) 17%
2) 21%
3) 20%
4) 24%
5) 18%

A chi-squared test says that these results do not differ significantly from uniform random responding, with a p-value of 0.78.

The main reason why I ran this poll was because I thought it might have implications for the trickier poll above. It is interesting the option #4 was the most common response in this poll, that poll, and the gamefaqs poll which that poll was based on. #4 may seem especially random, and some respondents in the other polls may have just been trying to answer at random. But this poll ended up not providing much information about that; to test it we'd need a larger sample size, and preferably a poll where respondents did not use external sources of randomness.

For convenience: http://www.random.org/ or in Bash, echo $(($RANDOM % 5 + 1))

Question: what's a reasonable prior over the probability distribution of poll answers? Because I downloaded the raw data, and it says:

1. 15
2. 22
3. 21
4. 24
5. 18

...and I'm not sure what would constitute reasonable priors for the uniform distribution hypothesis versus the "aversion toward First Answer" hypothesis versus the "aversion toward First Answer and Fifth Answer" hypothesis.

Is (the seconds' figure in my watch) mod 5 random enough?

I think this would be even more interesting as "pick at random, without an external source of randomness". Sure you can get random numbers from random.org, your computer or the seconds on your watch (a nice idee), but those just blur the effect of mind-generated random numbers.

I rolled 1d6, intending to reroll any 6s.

It correctly interpreted ۲ as 2. :)

After my vote:

" Mean 43.5 Median 75.0 Total votes 2"

Well this is mathematically impossible... My guess is the median isn't properly calculated for even numbers of votes.

Current results: Red: 0%; Green, 33%; Blue: 67%; Other: 0%.

I'm gladdened to see that even though we don't discuss politics on LW, the green scum are in the minority here!

What kind of question is, “Red is a nice color”? Some shades of red are nice and some aren't. Duh. (Also, who the hell has had a favourite colour for six dozen million times the age of the Universe?)

I got an error the first time because I put a percent instead of a fractional probability. Upon correcting this, I now see the following erroneous result: http://imgur.com/IASqy

The solution on Mathematics Stack Exchange

This doesn't look right: http://screencast.com/t/qpRGihBG

The raw data says there are 13 votes for "0" and 20 votes for "1".

As of my vote, I count 28 winners.

While simulating this with the current poll structure is possible, it quickly becomes cumbersome with an increased number of options; the number of options in the poll would be two to the power of the number of choices.

Example: Which of the following superheroes' powers would you prefer to duplicate, if you could do so safely?

A: Spiderman B: Mr. Fantastic C: Aquaman

[pollid:44]

The largest number is about 45,000,000,000, although mathematicians suspect that there may be even larger numbers. (45,000,000,001?)

According to my I Ching calculator, beyond 4 is a suffusion of yellow.

This appears not to be a valid response. Curious.

Median 17.0

....

I'll just leave Mr. Show's "highest number" sketch here.

According to the poll, my understanding of what qualifies as an integer is very, very wrong. 1e+19=the universal integer limit. NO EXCEPTIONS!

It does not seem to accept 'inf' or 'infinite'.

NaN

Doesn't accept "⌊∞⌋".

The results so far (only showing answers with > 1 responder):

11 "0.0"
 8 "-1.0"
 7 "2147483647.0"
 5 "3.0"
 4 "42.0"
 4 "1e+19"
 3 "9.0"
 3 "8.0"
 3 "1.0"
 2 "666.0"
 2 "32767.0"
 2 "24.0"
 2 "2.0"
 2 "1e+17"

To regenerate this, run grep -v "#" poll.csv | awk -F , '{ print $3 }' | sort | uniq -c | sort -nr.

I'm not surprised by the number of votes for 2^31-1. It was the first number to pop into my head when I saw the poll.

So according to the raw data, 100 people voted, but what I see displayed is 3 votes for answer 4 of 5, one for 5 of 5, and no total.

Trying to do a poll with only one option fails gracefully. Example: What kind of book did you read last? [poll]{a book}

Modifying the number of periods modifies the number of options available: [pollid:12]

|poll:True............False|

One period is well-defined: Nope. |poll:True.False| throws an "invalid poll type" error.

What about leaving off one of the names? |poll:True...| throws an "invalid poll type" error as well.

Polls are stored by their id, which makes it so they can't be edited after the fact. (Probably wise.) But what happens when you refer to an old poll by its id?

This is |pollid:10|, which refers to the poll from this comment.

[pollid:10]

What about a future poll?

[pollid:999999]

Trying to put two kinds of polls together:

How many bugs will I find? [pollid:11]

The code for that was |poll:number|{all of them}{none of them}, with the pipes replaced by square brackets. It looks like the interior poll type takes precedence, which is probably what should happen, but it might be better to complain instead.

Testing the spaces to make sure I'm doing it right:

Yep, [I](http://lesswrong.com/lw/ekw/less_wrong_polls_in_comments/7gqv) am.

Now let's try a well-formatted poll:

Is this commented? [pollid:10]

Gray is darker than grey.

It's a shame that Randall Munroe collated the two spellings together in the results of his survey. (And he even kept “periwinkle” and “perrywinkle” separate.)

Should I do a weighted sum over descendant entities I deem fractionally me, or just over entities I deem "me"?

How to aggregate across the distribution of possibilities? Average? Median? Most likely range?

I'm 33, so it wouldn't take too much life extension to get me to 133, but a fair amount... I'd rate the probabilities as roughly 40%, 30%, 10%, 20%. So, each of the three answers is different.

The bug concerning reporting of results is still present: currently the counts for the four categories are displayed as 9, 0, 0, 0, with a total of 39. According to the downloaded data, the counts are 28, 4, 1, 6 = 39.

The FAI hidden deep in the poll code logic refuses to run stupid and trollish polls.

Thank you for creating an off-topic test reply to reply to.

[pollid:748]

What about fuligin?

Won't you also ask about my favourite colour?

Psh, of course rationalists think Twilight Sparkle is the best pony.

I think it's "Pinkie".

Best background pony [pollid:23]

Rainbow Dash is the official spokespony of #lesswrong. She's been in the /topic for several months, now.

Geeks and goths, man.

Currently it has to be in comments. Since comments and articles have entirely different markup and formatting systems (comments use Markdown, articles use constrained HTML), supporting them in both places at once is nontrivial.

If someone wrote the code to make the inclusion of an "Other" option a default, opt-out behavior of LW polls, I would not object if that code were added.

I can vote in a poll even after it's retracted...

Can you still vote on retracted polls?

Testing...

Answer: Yes.

Can you be more specific about what you mean?

This is an especially good point because you're currently able to change the question after the results are in, allowing you to prank the poll takers by making their answers seem to support anything you feel like.

I don't know if it's specifically addressed anywhere in the Terms of Use, but free use of polls can have some very hurtful results; it might be helpful to somewhere post a guide to what type of polls are appropriate and tolerated.

What? About the same as the what you could write in comments already but prettier.

"Don't be an asshole" covers it. If you need a guide to tell you that, a guide will not help you.

Please don't do that.

Either the first or the second depending on how busy I am.