Posts
Comments
I'm curious what disagree votes mean here. Are people disagreeing with my first sentence? Or that the particular questions I asked are useful to consider? Or, like, the vibes of the post?
(Edit: I wrote this when the agree-disagree score was -15 or so.)
I think that people who work on AI alignment (including me) have generally not put enough thought into the question of whether a world where we build an aligned AI is better by their values than a world where we build an unaligned AI. I'd be interested in hearing people's answers to this question. Or, if you want more specific questions:
- By your values, do you think a misaligned AI creates a world that "rounds to zero", or still has substantial positive value?
- A common story for why aligned AI goes well goes something like: "If we (i.e. humanity) align AI, we can and will use it to figure out what we should use it for, and then we will use it in that way." To what extent is aligned AI going well contingent on something like this happening, and how likely do you think it is to happen? Why?
- To what extent is your belief that aligned AI would go well contingent on some sort of assumption like: my idealized values are the same as the idealized values of the people or coalition who will control the aligned AI?
- Do you care about AI welfare? Does your answer depend on whether the AI is aligned? If we built an aligned AI, how likely is it that we will create a world that treats AI welfare as important consideration? What if we build a misaligned AI?
- Do you think that, to a first approximation, most of the possible value of the future happens in worlds that are optimized for something that resembles your current or idealized values? How bad is it to mostly sacrifice each of these? (What if the future world's values are similar to yours, but is only kinda effectual at pursuing them? What if the world is optimized for something that's only slightly correlated with your values?) How likely are these various options under an aligned AI future vs. an unaligned AI future?
Yeah, there's definitely value in experts being allowed to submit multiple times, allowing them to update on other experts' submissions. This is basically the frame taken in Chapter 8, where Alice and Bob update their estimate based on the other's estimate at each step. This is generally the way prediction markets work, and I think it's an understudied perspective (perhaps because it's more difficult to reason about than if you assume that each expert's estimate is static, i.e. does not depend on other experts' estimates).
Thanks! I think the reason I didn't give those expressions is that they're not very enlightening. See here for l = 2 on (0, 1/2] and here for l = 4 on [1/2, 1).
Thanks! Here are some brief responses:
From the high level summary here it sounds like you're offloading the task of aggregation to the forecasters themselves. It's odd to me that you're describing this as arbitrage.
Here's what I say about this anticipated objection in the thesis:
For many reasons, the expert may wish to make arbitrage impossible. First, the principal may wish to know whether the experts are in agreement: if they are not, for instance, the principal may want to elicit opinions from more experts. If the experts collude to report an aggregate value (as in our example), the principal does not find out whether they originally agreed. Second, even if the principal only seeks to act based on some aggregate of the experts' opinions, their method of aggregation may be different from the one that experts use to collude. For instance, the principal may have a private opinion on the trustworthiness of each expert and wishes to average the experts' opinions with corresponding weights. Collusion among the experts denies the principal this opportunity. Third, a principal may wish to track the accuracy of each individual expert (to figure out which experts to trust more in the future, for instance), and collusion makes this impossible. Fourth, the space of collusion strategies that constitute arbitrage is large. In our example above, any report in [0.546, 0.637] would guarantee a profit; and this does not even mention strategies in which experts report different probabilities. As such, the principal may not even be able to recover basic information about the experts' beliefs from their reports.
For example, when I worked with IARPA on geopolitical forecasting, our forecasters would get financial rewards depending on what percentile they were in relative to other forecasters.
This would indeed be arbitrage-free, but likely not proper: it wouldn't necessarily incentivize each expert to report their true belief; instead, an expert's optimal report is going to be some sort of function of the expert's belief about the joint probability distribution over the experts' beliefs. (I'm not sure how much this matters in practice -- I defer to you on that.)
It's surprising to me that you could disincentivize forecasters from reporting the aggregate as their individual forecast.
In Chapter 4, we are thinking of experts as having immutable beliefs, rather than beliefs that change upon hearing other experts' beliefs. Is this a silly model? If you want, you can think of these beliefs as each expert's belief after talking to the other experts a bunch. In theory(?) the experts' beliefs should converge (though I'm not actually clear what happens if the experts are computationally bounded); but in practice, experts often don't converge (see e.g. the FRI adversarial collaboration on AI risk).
It seems to me that under sufficiently pessimistic conditions, there would be no good way to aggregate those two forecasts.
Yup -- in my summary I described "robust aggregation" as "finding an aggregation strategy that works as well as possible in the worst case over a broad class of possible information structures." In fact, you can't do anything interesting in the worse case over all information structures. The assumption I make in the chapter in order to get interesting results is, roughly, that experts' information is substitutable rather than complementary (on average over the information structure). The sort of scenario you describe in your example is the type of example where Alice and Bob's information might be complementary.
Great questions!
- I didn't work directly on prediction markets. The one place that my thesis touches on prediction markets (outside of general background) is in Chapter 5, page 106, where I give an interpretation of QA pooling in terms of a particular kind of prediction market called a cost function market. This is a type of prediction market where participants trade with a centralized market maker, rather than having an order book. QA pooling might have implications in terms of the right way to structure these markets if you want to allow multiple experts to place trades at the same time, without having the market update in between. (Maybe this is useful in blockchain contexts if market prices can only update every time a new block is created? I'm just spitballing; I don't really understand how blockchains work.)
- I think that for most contexts, this question doesn't quite make sense, because there's only one question being forecast. The one exception is where I talk about learning weights for experts over the course of multiple questions (in Chapter 5 and especially 6). Since I talk about competing with the best weighted combination of experts in hindsight, the problem doesn't immediately make sense if some experts don't answer some questions. However, if you specify a "default thing to do" if some expert doesn't participate (e.g. take all the other experts' weights and renormalize them to add to 1), then you can get the question to make sense again. I didn't explore this, but my guess is that there are some nice generalizations in this direction.
- I don't! This is Question 4.5.2, on page 94 :) Unfortunately, I would conjecture (70%) that no such contract function exists.
(Note: I work with Paul at ARC theory. These views are my own and Paul did not ask me to write this comment.)
I think the following norm of civil discourse is super important: do not accuse someone of acting in bad faith, unless you have really strong evidence. An accusation of bad faith makes it basically impossible to proceed with discussion and seek truth together, because if you're treating someone's words as a calculated move in furtherance of their personal agenda, then you can't take those words at face value.
I believe that this post violates this norm pretty egregiously. It begins by saying that hiding your beliefs "is lying". I'm pretty confident that the sort of belif-hiding being discussed in the post is not something most people would label "lying" (see Ryan's comment), and it definitely isn't a central example of lying. (And so in effect it labels a particular behavior "lying" in an attempt to associate it with behaviors generally considered worse.)
The post then confidently asserts that Paul Christiano hides his beliefs in order to promote RSPs. This post presents very little evidence presented that this is what's going on, and Paul's account seems consistent with the facts (and I believe him).
So in effect, it accuses Paul and others of lying, cowardice, and bad faith on what I consider to be very little evidence.
Edited to add: What should the authors have done instead? I think they should have engaged in a public dialogue with one or more of the people they call out / believe to be acting dishonestly. The first line of the dialogue should maybe have been: "I believe you have been hiding your beliefs, for [reasons]. I think this is really bad, for [reasons]. I'd like to hear your perspective."
To elaborate on my feelings about the truck:
- If it is meant as an attack on Paul, then it feels pretty bad/norm-violating to me. I don't know what general principle I endorse that makes it not okay: maybe something like "don't attack people in a really public and flashy way unless they're super high-profile or hold an important public office"? If you'd like I can poke at the feeling more. Seems like some people in the Twitter thread (Alex Lawsen, Neel Nanda) share the feeling.
- If I'm wrong and it's not an attack, I still think they should have gotten Paul's consent, and I think the fact that it might be interpreted as an attack (by people seeing the truck) is also relevant.
(Obviously, I think the events "this is at least partially an attack on Paul" and "at least one of the authors of this post are connected to Control AI" are positively correlated, since this post is an attack on Paul. My probabilities are roughly 85% and 97%*, respectively.)
*For a broad-ish definition of "connected to"
I don't particularly see a reason to dox the people behind the truck, though I am not totally sure. My bar against doxxing is pretty high, though I do care about people being held accountable for large scale actions they take.
That's fair. I think that it would be better for the world if Control AI were not anonymous, and I judge the group negatively for being anonymous. On the other hand, I don't think I endorse them being doxxed. So perhaps my request to Connor and Gabriel is: please share what connection you have to Control AI, if any, and share what more information you have permission to share.
(Conflict of interest note: I work at ARC, Paul Christiano's org. Paul did not ask me to write this comment. I first heard about the truck (below) from him, though I later ran into it independently online.)
There is an anonymous group of people called Control AI, whose goal is to convince people to be against responsible scaling policies because they insufficiently constraint AI labs' actions. See their Twitter account and website (also anonymous Edit: now identifies Andrea Miotti of Conjecture as the director). (I first ran into Control AI via this tweet, which uses color-distorting visual effects to portray Anthropic CEO Dario Amodei in an unflattering light, in a way that's reminiscent of political attack ads.)
Control AI has rented a truck that had been circling London's Parliament Square. The truck plays a video of "Dr. Paul Christiano (Made ChatGPT Possible; Government AI adviser)" saying that there's a 10-20% chance of an AI takeover and an overall 50% chance of doom, and of Sam Altman saying that the "bad case" of AGI is "lights out for all of us". The back of the truck says "Responsible Scaling: No checks, No limits, No control". The video of Paul seems to me to be an attack on Paul (but see Twitter discussion here).
I currently strongly believe that the authors of this post are either in part responsible for Control AI, or at least have been working with or in contact with Control AI. That's because of the focus on RSPs and because both Connor Leahy and Gabriel Alfour have retweeted Control AI (which has a relatively small following).
Connor/Gabriel -- if you are connected with Control AI, I think it's important to make this clear, for a few reasons. First, if you're trying to drive policy change, people should know who you are, at minimum so they can engage with you. Second, I think this is particularly true if the policy campaign involves attacks on people who disagree with you. And third, because I think it's useful context for understanding this post.
Could you clarify if you have any connection (even informal) with Control AI? If you are affiliated with them, could you describe how you're affiliated and who else is involved?
EDIT: This Guardian article confirms that Connor is (among others) responsible for Control AI.
Social graces are not only about polite lies but about social decision procedures on maintaining game theoretic equilibria to maintain cooperation favoring payoff structures.
This sounds interesting. For the sake of concreteness, could you give a couple of central examples of this?
There were 14 -- but they did so well that it's unlikely to have been by chance: the p-value is 0.0002 (i.e. the probability of IQ >150 people having gotten such a large percentile conditioned on their true skill levels being distributed like the entire population is only 0.02%).
Hi! I just wanted to mention that I really appreciate this sequence. I've been having lots of related thoughts, and it's great to see a solid theoretical grounding for them. I find the notion that bargaining can happen across lots of different domains -- different people or subagents, different states of the world, maybe different epistemic states -- particularly useful. And this particular post presents the only argument for rejecting a VNM axiom I've ever found compelling. I think there's a decent chance that this sequence will become really foundational to my thinking.
Note that this is just the arithmetic mean of the probability distributions. Which is indeed what you want if you believe that P is right with probability 50% and Q is right with probability 50%, and I agree that this is what Scott does.
At the same time, I wonder -- is there some sort of frame on the problem that makes logarithmic pooling sensible? Perhaps (inspired by the earlier post on Nash bargaining) something like a "bargain" between the two hypotheses, where a hypothesis' "utility" for an outcome is the probability that the hypothesis assigns to it.
The aggregation method you suggest is called logarithmic pooling. Another way to phrase it is: take the geometric mean of the odds given by the probability distribution (or the arithmetic mean of the log-odds). There's a natural way to associate every proper scoring rule (for eliciting probability distributions) with an aggregation method, and logarithmic pooling is the aggregation method that gets associated with the log scoring rule (which Scott wrote about in an earlier post). (Here's a paper I wrote about this connection: https://arxiv.org/pdf/2102.07081.pdf)
I'm also exited to see where this sequence goes!
Thanks for the post! Quick question about your last equation: if each h is a distribution over a coarser partition of W (rather than W), then how are we drawing w from h for the inner geometric expectation?
How much should you shift things by? The geometric argmax will depend on the additive constant.
Thanks for the post -- I've been having thoughts in this general direction and found this post helpful. I'm somewhat drawn to geometric rationality because it gives more intuitive answers in thoughts experiments involving low probabilities of extreme outcomes, such as Pascal's mugging. I also agree with your claim that "humans are evolved to be naturally inclined towards geometric rationality over arithmetic rationality."
On the other hand, it seems like geometric rationality only makes sense in the context of natural features that cannot take on negative values. Most of the things I might want to maximize (e.g. utility) can be negative. Do you have thoughts on the extent to which we can salvage geometric rationality from this problem?
I wonder if the effect is stronger for people who don't have younger siblings. Maybe for people with younder siblings, part of the effect kicks in when they have a younger sibling (but they're generally too young to notice this), so the effect of becoming a parent is smaller.
"My probability is 30%, and I'm 50% sure that the butterfly probability is between 20% and 40%" carries useful information, for example. It tells people how confident I am in my probability.
I often talk about the "true probability" of something (e.g. AGI by 2040). When asked what I mean, I generally say something like "the probability I would have if I had perfect knowledge and unlimited computation" -- but that isn't quite right, because if I had truly perfect knowledge and unlimited computation I would be able to resolve the probability to either 0 or 1. Perfect knowledge and computation within reason, I guess? But that's kind of hand-wavey. What I've actually been meaning is the butterfly probability, and I'm glad this concept/post now exists for me to reference!
More generally I'd say it's useful to make intuitive concepts more precise, even if it's hard to actually use the definition, in the same way that I'm glad logical induction has been formalized despite being intractable. Also I'd say that this is an interesting concept, regardless of whether it's useful :)
The Bayesian persuasion framework requires that the set of possible world states be defined in advance -- and then the question becomes, given certain utility functions for the expert and decision-maker, what information about the world state should the expert commit to revealing?
I think that Bayesian persuasion might not be the right framework here, because we get to choose the AI's reward function. Assume (as Bayesian persuasion does) that you've defined all possible world states.[1] Do you want to get the AI to reveal all the information -- i.e. which particular world state we're in -- rather than a convenient subset (that it has precommitted to)? That seems straightforward: just penalize it really heavily if it refuses to tell you the world state.
I think the much bigger challenge is getting the AI to tell you the world state truthfully -- but note that this is outside the scope of Bayesian persuasion, which assumes that the expert is constrained to the truth (and is deciding which parts of the truth they should commit to revealing).
- ^
"World states" here need not mean the precise description of the world, atom by atom. If you only care about answering a particular question ("How much will Apple stock go up next week?" then you could define the set of world states to correspond to relevant considerations (e.g. the ordered tuple of random variables (how many iPhones Apple sold last quarter, how much time people are spending on their Macs, ...)). Even so, I expect that defining the set of possible world states to be practically impossible in most cases.
For personal reasons it made sense for me to calculate the percentage of Londoners who will have COVID this Thursday, the 16th. The number I got was much higher than I intuitively expected: 10%. Please point out any errors you see!
- Among specimens collected in London 5 days ago, about 8000 were positive. This is relative to 4000 before the recent rise in cases, suggesting about 4000 are Omicron. Source
- Omicron doubles at a rate of 2.5 days in the UK. Source
- So among specimens collected Monday, we’d expect ~16k Omicron cases. Among specimens collected Thursday the 16th that should be ~35k.
- As a ballpark guess, we might guess that about half of cases are caught, so that’s ~70k.
- The typical time period between someone catching COVID and getting tested is 5 days. So the number of Londoners who will catch COVID on Thursday is ~280k, since they’ll typically get tested 5 days (two doublings) after that. That’s about 3% of the population of London.
- Omicron grows by a factor of ~1.3 per day, so (3/1.3)% will catch COVID on Wednesday, and so on. The total percentage of Londoners who will have COVID on Thursday is thus ~10% (summing the appropriate geometric series).
Thoughts?
I think we're disagreeing on semantics. But I'd endorse both the statements "violence is bad" and "violence is sometimes good".
I'm not sure. The strongest claim I make in that direction is that "Many in the rationalist sphere look down on tribalism and group identity." I think this is true -- I bet each of the people I named would endorse the statement "The world would be better off with a lot less tribalism."
To be clear, I'm agreeing with Eliezer; I say so in the second paragraph. But for the most part my post doesn't directly address Eliezer's essay except in passing. Instead I point out: "Yeah, the 'bad for reasoning about tribe affiliated subjects' is a drawback, but here's a benefit, at least for me."
It's true that I didn't draw a distinction between tribalism and group identity. My reason for doing so was that I thought both terms applied to my three examples. I thought a bit about the distinction between the two in my mind but didn't get very far. So I'm not sure whether the pattern I pointed out in my post is true of tribalism, or of group identity, or both. But since you pressed me, let me try to draw a distinction.
(This is an exercise for me in figuring out what I mean by these two notions; I'm not imposing these definitions on anyone.)
The word "tribalism" has a negative connotation. Why? I'd say because it draws out tendencies of tribe members to lose subjectivity and defend their tribe. (I was going to call this "irrational" behavior, but I'm not sure that's right; it's probably epistemically irrational but not necessarily instrumentally irrational.) So, maybe tribalism can be defined as a mindset of membership in a group that causes the member to react defensively to external challenges, rather than treating those challenges objectively.
(I know that I feel tribalism toward the rationalist community because of how I felt on the day that Scott Alexander took down Slate Star Codex, and when the New York Times article was published. I expect to feel similarly about EA, but haven't had anything trigger that emotional state in me about it yet. I feel a smaller amount of tribalism toward neoliberalism.)
(Note that I'm avoiding defining tribes, just tribalism, because what's relevant to my post is how I feel about the groups I mentioned, not any property of the groups themselves. If you wanted to, you could define a tribe as a group where the average member feels tribalism toward the group, or something.)
Identity is probably easier to define -- I identify with a group if I consider myself a member of it. I'm not sure which of these two notions is most relevant for the sort of pattern I point out, though.
Good point! You might be interested in how I closed off an earlier draft of this post (which makes some points I didn't make above, but which I think ended up having too high of a rhetoric to insight ratio):
"I don’t endorse tribalism in general, or think it’s a net positive. Tribalism strikes me as a symmetric weapon, equally wieldable by good and evil. This alone would make tribalism net neutral, but in fact tribalism corrupts, turning scouts into soldiers, making people defend their side irrespective of who’s right. And the more tribal a group becomes, the more fiercely they fight. Tribalism is a soldier of Moloch, the god of defecting in prisoner’s dilemmas.
This is somewhat in tension with my earlier claim that my tribalism is a net positive. If I claim that my tribalism is net positive, but tribalism as a whole is net negative, then I’m saying that I’m special. But everyone feels special from the inside, so you’d be right to call me out for claiming that most people who feel that their tribalism is good are wrong, but I happen to be right. I would respond by saying that among people who think carefully about tribalism, many probably have a good relationship with it. I totally understand if you don’t buy that — or if you think that I haven’t thought carefully enough about my tribalism.
But the other thing is, tribalism’s relationship with Moloch isn’t so straightforward. While on the inter-group level it breeds discord, within a tribe it fosters trust and cooperation. An American identity, and a British identity, and a Soviet identity helped fight the Nazis — just as my EA identity helps fight malaria.
So my advice on tribalism might be summarized thus: first, think carefully and critically about who the good guys are. And once you’ve done that — once you’ve joined them — a little tribalism can go a long way. Not a gallon of tribalism — beyond a certain point, sacrificing clear thinking for social cohesion becomes negative even if you’re on the good side — but a teaspoon."
Thanks for mentioning Asch's conformity experiment -- it's a great example of this sort of thing! I might come back and revise it a bit to mention the experiment.
(Though here, interestingly, a participant's action isn't exactly based on the percentage of people giving the wrong answer. It sounds like having one person give the right answer was enough to make people give the right answer, almost regardless of how many people gave the wrong answer. Nevertheless, it illustrates the point that other people's behavior totally does influence most people's behavior to quite a large degree, even in pretty unexpected settings.)
Yeah -- to clarify, in the last section I meant "select how radical you'll be for that issue at random." In the previous section I used "radical" to refer to a kind of person (observing that some people do have a more radical disposition than others), but yeah, I agree that there's nothing wrong with choosing your level of radicalism independently for different issues!
And yeah, there are many ways this model is incomplete. Status quo bias is one. Another is that some decisions have more than two outcomes. A third is that really this should be modeled as a network, where people are influenced by their neighbors (and I'm assuming that the network is a giant complete graph). A simple answer to your question might be "draw a separate curve for 'keep camera on if default state is on' and 'turn camera on if default state is off'", but there's more to say here for sure.
I'm not conditioning on any configuration of points. I agree it's false for a given configuration of points, but that's not relevant here. Instead, I'm saying: number the intervals clockwise from 1 to n + 2, starting with the interval clockwise of Z. Since the n + 2 points were chosen uniformly at random, the interval numbered k1 is just as likely to have the new point as the interval numbered k2, for any k1 and k2. This is a probability over the entire space of outcomes, not for any fixed configuration.
(Or, as you put it, the average probability over all configurations of the probability of landing in a given interval is the same for all intervals. But that's needlessly complicated.)
For what it's worth, the top three finishers were three of the four most calibrated contestants! With this many strings, I think being intentionally overconfident as a bad strategy. (I agree it would make sense if there were like 10 or 20 strings.)
Yours was #104 -- you did well!
Two things, I'm guessing. First, there's the fact that in baseball you get thousands of data points a year. In presidential politics, you get a data point every four years. If you broaden your scope to national and state legislative elections (which wasn't Shor's focus at the time), in some sense you get thousands per election cycle, but it's more like hundreds because most races are foregone conclusions. (That said, that's enough data to draw some robust conclusions, such as that moderate candidates are more electable. On the other hand, it's not clear how well those conclusions would translate to high-profile races.)
Second, electoral politics is probably a way harder thing to model. There are many more variables at play, things shift rapidly from year to year, etc. Meanwhile, baseball is a game whose rules of play -- allowable actions, etc. -- are simple enough to write down. Strategy shifts over the years, but not nearly as much as in politics. (I say that without having much knowledge of baseball, so I could be... um, off base... here.)
That's true. My response is that if I recall correctly, people didn't seem to react very strongly to what was happening in Italy a couple weeks before it was happening here. So I'm not sure that a surge in the UK would inform the US response much (even though it should).
Yeah, you're right. 1.3 was the right constant for Covid in March because of a combination of not being locked down and having more and more tests. This was my attempt to make a direct comparison, but maybe the right way to make that comparison would be to say "if R=1.65 (which I'll trust you that it leads to a constant of 1.08), we will react about X days slower than if we started from scratch."
What is X? The answer is about 60 (two months). On the one hand, that's a lot more than the 2-3 weeks above; on the other hand, it's less scary because R is lower.
Thanks! I've changed the title to "Great minds might not think alike".
Interestingly, when I asked my Twitter followers, they liked "Alike minds think great". I think LessWrong might be a different population. So I decided to change the title on LessWrong, but not on my blog.
I like logarithmic better in general, but I decided to use Brier for the pseudorandomness contest because I decided I really cared about the difference between a 60% chance (i.e. looks basically random) and a 40% chance (kind of suspect). The log rule is better at rewarding people for being right at the extremes; Brier's rule is better at rewarding people for being right in the middle.
Regarding bets: I'm willing to make bets, but won't have a blanket policy like "I'll take a bet with anyone who disagrees with me by 10% or more", because that opens me up to a ton of adverse selection. (E.g. I wouldn't bet with Zvi on COVID.) So... feel free to message me if you want to bet, but also be aware that the most likely outcome is that it won't result in a bet.
(Also, the better I know you, the more likely I am to be willing to bet with you.)
Puzzle 3 thoughts: I believe I can do it with
1
coins, as follows.
First, I claim that for any prime q, it is possible to choose one of q + 1 outcomes with just one coin. I do this as follows:
- Let p be a probability such that (Such a p exists by the intermediate value theorem, since p = 0 gives a value that's too large and p = 1/2 gives a value that's too small.)
- Flip a coin that has probability p of coming up heads exactly q times. If all flips are the same, that corresponds to outcome 1. (This has probability 1/(q + 1) by construction.)
- For each k between 1 and q - 1, there are ways of getting exactly k heads out of q flips, all equally likely. Note that this quantity is divisible by q (since none of 1, ..., k are divisible by q; this is where we use that q is prime). Thus, we can subdivide the particular sequences of getting k heads out of q flips into q equally-sized classes, for each k. Each class corresponds to an outcome (2 through q + 1). The probability of each of these outcomes is which is what we wanted.
Now, note that 2021*12 - 1 = 24251 is prime. (I found this by guessing and checking.) So do the above for q = 24251. This lets you flip a coin 24251 times to get 24252 equally likely outcomes. Now, since 24252 = 2021*12, just assign 12 of the outcomes to each person. Then each person will have a 1/2021 chance of being selected.
Conjecture (maybe 50% chance of being true?):
If you're only allowed to use one coin, it is impossible to do this with fewer than 24251 flips in the worst case.
Question:
What if you can only use coins with rational probabilities?
Thanks for the feedback. Just so I can get an approximate idea if this is the consensus: could people upvote this comment if you like the title as is (and upvote mingyuan's comment if you think it should be changed)? Thanks!
Also, if anyone has a good title suggestion, I'd love to hear it!
Yeah, I agree that the post isn't quite sequential. Most of Section II isn't necessary for any of the translator stuff -- it's just that I thought it was an interesting possible explanation of "alike minds think great" bias. (This somewhat disconnected logical structure was a hangup I had about the post; I was considering publishing it as two separate posts but decided not to.)
But, what I was trying to say about Shor and Constance and their need for a translator is: regardless of whether Shor underestimated Constance and vice versa because of this bias, they weren't in a position to understand each other's arguments. A translator's job is to make them understand each other (convert their thoughts into a language that's easily understandable by the other). This allows for reconciliation, because instead of Shor seeing his own argument and "black box Constance belief which I should update on even though I don't understand it", he sees his own argument and "Constance argument translated into Shor language", which he now has a much better idea what to do with. (And likewise symmetrically for Constance.)
Thanks for the suggestion. I might do that later; in the meanwhile, the following should work for pasting the strings as text (at least on Windows).
- Format the cells you are planning to paste the strings in as "text". (Right click -> format -> text)
- Copy the strings
- Right click -> paste special -> values only (if you hover over the options you should be able to find that one) -> text.
I've changed the rules to get rid of the normalization of probabilities clause, because it was pointed out to me that if someone says 0 to everything in an attempt to do well in Round 1, their Round 2 submission will receive a weight of 0 for Round 1 scoring anyway. It's still possible that there are some incentives issues here, but I doubt there's anything major, and I don't want to mess too much with what people submit.
Yeah, this is okay. But, something that wouldn't be okay is writing some math or whatever on the side as part of calculating more bits. You can look at bits you've typed, but any computation you do with them must be in your head.
I'm not sure I understand; could you clarify? If you're saying that the number of characters you've typed is displayed, that's okay -- that's why I recommended that one. (I suppose this makes my latest comment not quite accurate but the point of that is just so you know when to stop.)
My guess is that your are not breaking the rules as I intended to state them. Or if you think you did, perhaps do it again without breaking the rules and submit with a comment saying to count the later entry?
Yes.
Yup, these are all okay!
It is okay for you to go back and change or insert bits. Ideally you are in a uniform room with the buttons 0, 1, switch to a point of your choice in the string you have typed, and backspace. Thanks for clarifying!
No, sorry -- any thinking about strategy in advance must be your own thoughts. No external resources for that either.
Question to the LW community: I made this linkpost by copy-pasting from my my blog and then correcting any bad formatting. Is there an easier way to do this? Also, is there a way to do in-line LaTeX equations here?