Evaluating 2022 ACX Predictions

post by Zvi · 2023-02-14T12:20:01.148Z · LW · GW · 3 comments

Contents

  Big Picture
  Question By Question
  Wait Till Next Year
None
3 comments

Previously: 2022 ACX Predictions Buy/Sell/Hold, Evaluating 2021 ACX Predictions

As is the yearly tradition, let’s see how we did. Scott noted in his results post that on pure numbers I did pretty well, coming in 54th out of 508 entries.

My own evaluation, by my own standards, is I did a mediocre job.

Big Picture

2022 was the first low-effort year, no looking at prediction markets and five minute time limits. For 2023, I put even less effort into things – I didn’t spend the five minutes even once, and mostly put down numbers on instinct. I modestly regret not trying harder, but am confused as to whether or not that regret makes sense.

Pure numbers is not how I evaluate predictions. They are one input, but it is more important to do deliberate practice. Look back, and ask whether your prediction made sense, whether your logic was good, what would have been the right odds. A Briar score is quite a crude measure, as is looking at calibration. They can tell you vaguely where you are at in some sense, but it won’t much help you improve otherwise.

Then one can look at the big picture, and see what patterns and errors emerge.

There are very clear patterns of error in my predictions from 2022.

1. Too much anchoring on Scott’s (and Vox’s and Matt’s) answers. The degree of anchoring I did makes sense for market prices, or for predictions informed by a market price. Instead I did it for what Scott pulled out of the air. I also did it for Vox and Matt, who did not follow Scott’s restrictions but also did not seem to care at all about looking at markets, including financial markets.
2. Failure to do the research. There was a five minute timer and you couldn’t look at prediction markets, but it is very obvious I frequently paid a price for not taking the full five minutes, and not asking questions I could have asked.
3. Bad inflation takes. A bunch of my dumb value loss was being too much on Team Transitory. I think it’s fair to say that at the time it was clear the market was being dumb this way, and that I repeated that mistake and even went further, for no good reason. This kind of exercise is good at highlighting things like that. Of course, one reason for that was…
4. Skepticism of Russia invading Ukraine. I had world model errors underlying this, which I hope I have addressed. I correctly predicted that any invasion was overdetermined to be disastrous for Russia. Then somehow I assumed that Putin would be able to figure this out with his superior information and focus, as opposed to being in the ultimate SNAFU situation and blind to reality. Whoops.

How should we think about prediction markets outscoring all but one person’s five minute efforts? I think we should be disappointed in prediction markets here, given the asymmetries. I agree that the one person who beat them outright in five minutes got lucky. It still seems clear that anyone thinking well would be able to defeat the prediction markets ‘playing offense’ rather soundly, and that weighing smarter opinions more improves them a lot.

Question By Question

So we go through each question in turn, and ask if good decisions got made. Keep in mind we weren’t allowed to look at prediction markets (both Scott and myself).

1. Biden approval rating (as per 538) is greater than fifty percent: 40% → Sold to 30%.

Did not happen or come close.

Predictor average was 36%. On reflection I didn’t sell quite enough here. Biden had a bunch of good and bad news and none of it mattered much at all. Market was 14%, so if I’d known that I’d have sold down to where it was.

2. At least $250 million in damage from a single round of mass protests in US: 10% → Buy to 15%.

Did not happen or come close.

This was despite Roe vs. Wade being overturned, and not a peep. My logic was that $250 million was not very much in damages, and the bae rate of this wasn’t so low. Hard to know what the base rate truly is in a case like this. I don’t hate my buy here, especially with market recorded as 36%, but I do think 36% is insanely high when you look at the historical record. If Trump was president, maybe, but with Biden?

3. PredictIt thinks Joe Biden is most likely 2024 Dem nominee: 80% → Hold.

Happened and was never close to not happening.

It seems like Biden would have to credibly have said he wasn’t running for this to happen, or a combination of a clear alternative with a truly horrendous midterm. He even got a war against Russia by proxy and nothing much moved. So this was damn unlikely and if anything should have been higher.

4. …thinks Donald Trump is most likely 2024 GOP nominee: 60% → Buy to 70%.

Did not happen, could have gone either way, him and DeSantis flip every so often.

Donald Trump had a maximally bad midterm and generally things went quite poorly for him, while they went extremely well for DeSantis, and this still almost came in. There was a lot of resilience here along the baseline scenario, such that this seems like it starts out at 80%+, as we essentially needed a trifecta of (Trump failure, DeSantis success, and a coin flip). So the question is how often we get a different Trump failure mode, of failing health or severe legal trouble. The buy here seems good to me on reflection, even though it lost.

5. Beijing Olympics happen successfully on schedule: 99% → 98%.

Not on the compilation page but I’d say it happened. Presumably ambiguity was the issue here. Given the level of precautions, I’ll take a 2% shot at a hitch here every time on any reasonable definition.

6. Major flare-up (worse than past 5 years) in Russia/Ukraine conflict: 50% → Sell to 30%.

Very much happened.

I was a big skeptic on this and I was wrong big, with only my anchoring keeping me from making a very bad prediction on likely the most important question. As with many others, I assumed that something this stupid and pointless was highly unlikely, and wow was that logic wrong. There are many lessons here for sure, including ‘authoritarian information environments and incentives are even more warped and SNAFU than I realized’ and ‘no really, a person can’t escape their nature and will do crazy things’ and also some things about Russian culture.

This is also one case where being right about one big thing (that Ukraine could win and the Russian attack was really dumb) and then letting that make me very wrong about an even more important thing, because I thought others would understand that. Whoops.

7. Major flare-up (worse past 10 years) in Israel/Palestine conflict: 5% → Hold.

Did not happen.

On reflection this was probably a little low given how such distributions work. I like the superforecaster average of 7%.

8. Major flare-up (worse than in past 50 years) in China/Taiwan conflict: 5% → Sell to 3%.

Did not happen.

I said ‘even if there is a shooting war in Ukraine this won’t happen.’ There were some scary moments given how bad this would be if it happened, and we will never know for sure how close Xi came to trying something deeply stupid. Given that Ukraine happened, I’m going to say this sale was pretty bad, both in terms of the Ukraine war making the situation more volatile than other years (as did the party congress) and also reflecting a world model where this is more likely. So I don’t mind the sale in 2021 but in 2022 I think you couldn’t do that. Which is super scary. Going into 2023, knowing what we know now and being passed the windows, I’m back to being more of a seller than a buyer.

9. Honduran ZEDEs legally crippled to the point where no reasonable person would invest in them further: 5% → Buy to 7%.

Did not happen. I suppose ‘no reasonable person’ is a high bar, but they did repeal the law and are clearly hostile. Superforecasters had this at 16%, average predictor was at 13%, and I think I didn’t buy enough here, you have to go at least to 10% even with my lack of information and the anchoring. Which, in turn, means that no reasonable person should have invested before, as I noted. If you have a 15% yearly chance of things becoming non-viable, that’s not a good investment, even if it doesn’t tank everything automatically on the spot. Even 5% is quite a lot.

It is often said that all progress depends on the unreasonable man.

10. New ZEDE approved in Honduras: 30% → Hold due to lack of information

Did not happen, and was never going to happen given they outright repealed the law. Likely this should have been sold down a lot if one understood the situation. Seems reasonable to defer when totally in the dark. Would have been a good place to actually spend five minutes doing research, if I’d been going all out to win the competition.

11. GameStop stock price still above $100: 30% → Hold.

Did not happen, was way down instead.

Started year at 112. I think the market is getting short end of the stick being called 50 here because there is a long tail of super crazy and also who exactly is going to be shorting this for a year? Hard to know what one is supposed to think about GameStop levels of irrational behavior. On reflection I think this should have been more like 20%, and yes that makes GameStop a clear sale at the time but… that seems fine, no?

12. Bitcoin above 100K: 20% → sell to 18%.
13. Ether above 5K: 20% → buy to 25%.

Neither happened, both were down instead.

At the time Ether was ~2.5K as my notes indicate, still seems like these are reasonable market-neutral positions to take. One could argue that there were signs one should have been bearish, but it seems fine to pass on such questions. Market is marked as being more skeptical than this and I think that is the market pricing options badly – if I wanted to be long crypto, didn’t have to worry about taxes and was happy to do complex weird things and not worried about counterparty risk, I think buying far-out-of-the-money call options on crypto is a better buy than crypto – party exactly because of all those reasons not to do it.

14. Ethereum above 0.05 BTC: 90% → Sell to 80%.

Happened, it’s 0.07 today.

Crypto was indeed down and this held, which suggests that holding BTC instead of ETH was an error. I do think the 90% is expressing a rather strong opinion on major financial instruments here.

15. Bored Ape floor price here below current price of $203K: 40% → Sell to 30% (I interpreted this as chance of below price at the time, which had gone up to $325k).

It is now 107K and was clearly lower.

Wasn’t graded due to several issues.

The other problem is that the question says ‘below’ rather than ‘above.’ It is very bad to misread markets like that. You can lose quite a lot. What I clearly meant here was that I wanted to buy this, and I don’t think I would have cared too much about Scott’s opinion here so likely to have bought to 65% and maybe the full 70%. Could the NFT craze have sustained itself for the whole year, at least for the ‘top stuff’? Yeah, maybe it had a 25% or so chance, you never know. And I wouldn’t rule these out as potentially super valuable in 20 years, consider what people pay for basically anything from our childhoods that’s rare.

16. Dow above 35K: 90% → Sell to 70%

Did not happen.

90% was clearly very nuts. I said ‘check the market’ and that was marked as 54% here, whereas superforecasters were still at 75% and average predictor was 80%. I still can’t see how it is as low as 54% if you assume market is efficient, I’m willing to accept 70% was too high and reflected me actually being bullish. Hey, I was and still am long.

17. …above 37.5K: 40% → Hold

Did not happen.

Market is marked at 30%. A 24% chance of landing in between these numbers seems fine, I decided 30% was reasonable given my decision not to look at option prices and that all seems fine. Given the 70% was too high, this was too high as well, but it reflects a copy of the same mistake.

18. Inflation for the year below five percent: 90% → Sell to 80%

Did not happen, finished 6.5%.

I fell pretty stupid here for letting Scott’s number trick me into staying that high. There is simply no way this should have been anywhere near 80%. Market here is marked at 45%. Part of the difference is that I didn’t expect the Ukraine War, and with that I think 45% is too low, but anything above ~65% is still crazy talk. In general, it seems like I anchored far too much.

19. Unemployment below five percent: 50% → Buy to 70%.

Rate was about 3.7%, well below 5%. Market is listed at 73%, superforecaster average 68%. Given it is clear I was overinfluenced by Scott’s numbers, I’m going to say I was modestly too low, and 75% or even 80% would have been better.

20. Google widely allows remote work, no questions asked: 50% → Sell to 40%, assuming this was graded as true in 2021, as low as 20% if graded false in 2021.

Was graded as not happening.

Market is listed as 14%, but superforecasters said 52%. Not sure where market comes from here and the ambiguity in what counts as what is large. I noted that the 50% here threw me, which is likely another case of me anchoring too much on not-so-considered numbers. Didn’t sell enough.

21. Starship reaches orbit: 90% → Sell to 89% on no information, respect market.

Did not happen.

It seems most everyone did expect this to happen, market is listed as 94%. It does feel like an awful lot of confidence for something like this where any number of things can go wrong. Market being that high gives me pause but I’d like to have sold this more, given I did not know that. And on reflection if I’d known market was 94% I should have sold a tiny bit there too.

22. Fewer than 10K daily average official COVID cases in US in December 2022: 20% → Buy to 25%.

Did not happen or get anywhere close.

I was right that a lot of people would stop testing and reporting and caring about Covid. There were still a lot of Covid cases. If Omicron hadn’t kept evolving would this have had a chance? I think yeah, but December is a rough month and 10k is a low number. The buy here is at best aggressive.

23. Fewer than 50K daily average COVID cases worldwide in December 2022: 1% → Hold

Quote is ‘yeah, not going to happen.’ Seems fine to not try and sell sub-1% on principle, but here is a place one could have thought about it.

24. >66% US population fully vaccinated (by current standards) against COVID: 70% → Buy to 75%.

Happened, despite the vaccination campaign in 2022 being widely seen as disastrous.

Whenever things go pretty badly and you win anyway like this, in a place where the variance seems reasonably low, it means the uncertainty was in your thinking. I said I wasn’t excited, and presumably that means the calculation was off a bit.

25. India’s official case count is higher than US: 5% → Buy to 10%.

Did not come anywhere near happening.

I interpreted this as cumulative, still not sure if that was intended, it doesn’t happen either way. It still seems like a mystery why this very much did not happen, and I am fine with having bought a bit. Superforecaster average was also 10%.

26. Medical establishment reverses course and officially says any of Vitamin D, HCQ, or ivermectin is actually effective against COVID: 1% → Buy to 2%.

Did not happen.

I still think that I was pretty crazy to only buy this to 2% – only this past week I saw at least one doctor reverse course on Vitamin D and say it was unethical not to recommend it. And if I was worried about Covid, I would be damn sure to take my Vitamin D. Was a reversal here unlikely? Sure, but 5%-style not 2%-style. The other two were sub-1% shots.

27. FDA approves a COVID indication for fluvoxamine: 60% → Sell to 50%.

Sadly did not happen, in a situation where it obviously should have happened and also on reflection obviously was a long shot to have happened. The FDA is very much about not approving things and I feel silly still being as high as 50% even if others were mostly higher.

28. Some new variant not currently known is greater than 25% of cases: 60% → Hold.

Did not happen, there were new variants that displaced old ones but they were classified as sub-variants of Omicron. Given all the near misses in that sense, 60% still seems fine.

29. Most people I see in the local grocery store 12/31/22 are wearing masks: 60% → Sell to 40%.

I was too anchored here, and too worried (well, too worried in this particular context) that San Francisco can stay irrational longer than you can stay solvent. This was never going to happen and I should have been more confident.

30. Masks still required on domestic flights: 60% → Sell to 35%.

This does seem like exactly the type of place we could plausibly have stayed irrational, but one needs to justify 35% as being high rather than as being low. Market was still at 51%, so this was likely appropriately low.

31. CDC recommends that triple-vaxxed people get at least one more vax: 70% → Hold (with a conditional action to sell to 50% that I do not believe applies here).

Graded as happening due to the Omicron boosters.

This is more too low than too high, but it seems mostly inevitable that there would be an Omicron booster and that they’d tell everyone to get it. So if that alone counts, as I think it would have, then yeah, probably a bit low.

32. China has fewer than 100,000 COVID cases this year (official estimate): 30% → Hold.

This is graded as not happening. I think it’s pretty ambiguous. Yes the official statistics simply started ignoring reality, but there were various officials giving out ‘estimates’ that were several orders of magnitude higher than the threshold.

The reality could also have gone either way.

I think if you interpret the scenario that happened as resolving to no, then 30% is too high, because there is too big a risk of the official statistics simply ignoring reality, so now you need to win quite the parlay. If you don’t need that, 30% is too low, the dam broke rather late if anything.

Skipping the personal predictions because I don’t have grades for them, and I mostly didn’t trade them anyway.

88. No new real-money prediction market becomes bigger than Polymarket: 70% → Buy to 75%.

Happened.

Prediction markets are a hard business. There are a lot of people trying. There is a lot of value to the world in making them succeed. The problem is it is highly unclear how you the prediction market website makes real money. People want to bet on sports and politics, where there are clearly defined high profile outcomes without too much insider information, quick resolution, good drama. That doesn’t mean there is great appetite for weirdness that mostly happens over long period and also yes everything is probably a security (not investment advice). And then there’s all the Hansonian issues that people don’t want accurate information that is visible to all because it disrupts elite power and decision making. I am very happy to keep working with prediction markets to try and make them work, but the problem increasingly seems super hard to commercialize successfully where it would count.

In any case, yeah, if anything I didn’t buy enough here, Polymarket filled the niche of ‘crypto prediction market company actually trying’ and there was no reason to expect a new entrant that would be bigger.

89. Manifold Markets is still alive and active: 30% → Hold

Happened. Market is listed here as 91%, I am guessing that was a Manifold Market, which for reasonably obvious reasons is kind of cheating? Still, I do think that 30% chance for a company with no huge costs or reason to fold up shop keeping their doors open a year was on reflection super low and I feel dumb here not going to 60% or higher. Wow I anchored too much last year.

90. New legal US real-money prediction market at least half as big as Kalshi: 5% → Sell to 3%.

Market is listed as 12% and the superforecasters averaged 6% but I am happy with my call here. It indeed seems super difficult to get these things approved and off the ground quickly, market seems saturated, even if something does arrive it’s not going to get to half Kalshi size within a year of getting on our radar.

91. New illegal but easy-to-use market satisfying the above: 20% → Sell to 15%.

Yep, I doubt anything new is on the horizon, and if it is it’s going to take a while to ramp up most of the time. Still, could have happened, who knows, seems OK to stop at 15%. The market 42% here seems crazy.

92. I post my scores on these predictions before 2/1/23: 80% → Sell to 75%.

January 23, and I’m guessing what happened counts. My theory was Scott might choose to slightly miss, which did not happen. On reflection no, Scott mostly wants to meet his obligations, and I shouldn’t have sold.

Now the Vox predictions, so numbers reset.

1. Democrats will lose their majorities in the House and Senate: 95% → Scott sold to 90%, I sell to 90%.

Yeah, 95% was completely insane, should have known the market would never be anything like that high, 90% still insanely high. I did bet on Republicans and on net lost a bit at the market odds, which I mostly stand by given where there was uncertainty. It took a lot to have this not happen.

2. Inflation in the US will average under three percent: 80% → Scott holds, I sell to 60%.

Yeah, once again even my level of confidence in Team Transitory here was crazy and this was dumb.

3. Unemployment in the US will fall below four percent by November: 80%

Seems like this already happened and I was confused so I ignored it. Then in the spreadsheet this gets listed as under 3%. Super weird, in any case essentially a repeat where I make the same mistakes.

4. Supreme Court will overturn Roe v. Wade (65%) → Scott sells to 60%, I sell to 55%.

This did famously happen.

It’s hard to know how inevitable it was. Feels like it could have gone either way, and the leak reflected that this was up in the air no matter who or which side did it. Everyone was in the 50%-65% range which is actually reasonably narrow. I’m pretty much fine with it.

5. Stephen Breyer will retire from the Supreme Court (55%) → Whoops.

Yeah, that escalated quickly, whoops.

6. Emmanuel Macron will be reelected president of France (65%) → Scott Holds, I sell to 55%.

Seems I was on the wrong side of this, it was hard for him not to make the finals and hard for him to lose a heads-up against whoever it was because they were likely to either be hard right or hard left, despite his Covid policies. I didn’t think enough about how the dynamics work in France. It felt touch and go but not coin-flip level touch and go.

7. Jair Bolsonaro will be reelected president of Brazil (55%): Scott sells to 50%, I buy to 60%.

Did not happen, was close, market was highly skeptical so presumably I was missing things here. I haven’t looked into details enough to know what to think.

8. Bongbong Marcos will be elected president of the Philippines (55%): Scott buys to 60%, I split difference at 57% and essentially passed.

Happened. Both sources here are clearly not making good predictions so it’s not clear who to favor if I have no idea. If I had to guess I’d say re-elections are more likely than they are giving them credit for in general, but seems fine to pass.

9. Rebels will not capture Addis Ababa (55%): N/A → Yep, also escalated quickly.

10. China will not reopen its borders in the first half of 2022 (80%): Scott buys to 90%, I held at 80%.

Happened. Market was listed at 98% and yeah, that was insane confidence. I can see the argument for 90%, but given they did reopen in the second half, my guess is 80% was closer to the right answer than 90%.

11. Chinese GDP will continue to grow for the first 3/4 of the year (95%): Scott sells to 90%, as do I.

Happened.

It was clear at the time, and seems even more clear now, that the calibration on the Vox predictions was quite bad, confidence was way too high. You need a good reason not to sell in this spot.

12. 20% of US kids between 0.5 and 5 years old will get at least one COVID vaccine by year’s end (65%): Scott holds, I sell to 60%.

My logic was wrong here. I thought there was approval uncertainty, and instead we couldn’t find 20% of parents willing to do this despite approval, so that’s bad predicting no matter the resulting numbers. I was definitely surprised that the early child vaccine was rejected so resoundingly – sure I expected that in some areas, but 20% is not a lot.

13. WHO will designate another Variant Of Concern by year’s end (75%) → Scott holds, I sell to 70%.

Did not happen.

Everyone was in the 70s, and on reflection this should have been more like 60% for the same reason as other logic above. Too much anchoring once more.

14. 12 billion COVID shots will be given out globally by 11/2022 (80%) → Scott holds, I hold as a pass rather than a strong opinion.

This happened. Still haven’t looked into it.

15. At least one country will have less than 10% of people vaccinated with two shots by 11/2022 (70%): Scott buys to 95%, I buy to 85%.

This happened. I interpreted Scott’s buy as ‘there is at least one obvious country that will count’ and I’m guessing that it didn’t come to anything like that and it was a gimme and I chickened out.

16. A psychedelic drug will be decriminalized/legalized in at least one more US state (75%): Scott holds, I hold having not considered it.

Happened, as part of the continuing trend. Doesn’t feel like it was inevitable, so still ‘seems high’ without rising to the threshold of action.

17. AI will discover a new drug promising enough for clinical trials (85%): Scott holds, I say ‘depends on definitions so hold.’

This was graded as having happened, I stand by this being too ambiguous.

18. US govt will not renew the ban on funding gain-of-function research (60%): Scott holds, I buy to 75%.

Happened, alas. What I said here: “Buy to 75%, we’re really stupid. I mean really, really stupid and terrible at this.” Yep. If anything should have been higher.

19. The Biden administration will set the social cost of carbon at $100/ton or more (70%): Scott Holds, I buy to 80%.

There is a resolution dispute, FP/Vox claims probably true, I agree with Scott this was false, ‘will happen real soon now’ does not in any way count, doesn’t matter what they do in 2023. My mistake here was assuming that it almost certainly wouldn’t take them into 2023 to set a social cost of carbon. Somehow I overestimated government efficiency. I’d like to say it won’t happen again.

20. 2022 will be warmer than 2021 (80%): Scott holds, I held.

Happened. As I noted this is a math question, not clear I did enough of the relevant math, but coming off a down year this was presumably a sizable favorite.

21. Kenneth Branagh’s Belfast will win Best Picture (55%): Scott Sells to 30%, I sell to 35%.

As I said, uninformed sale, I was right that Dune had no chance. I had too much respect for Vox’s predictions and calibration, could have easily sold this down to 25% despite being uninformed – if Belfast had been that likely to win other sources would have mentioned it, although it’s not obvious I’d have seen them.

22. Norway will win the most medals at the 2022 Winter Olympics (60%): Scott holds, I held as a pass.

Not on spreadsheet but this happened and they had 37 versus next highest being 27. So the right answer was much higher, this was never going to go another way. Doesn’t mean one could find that out in five minutes. There’s ‘this should be knowable in advance’ pulling in favor of going higher given that Vox said 60%, there’s ‘Vox is overconfident in things’ pulling it down. Not sure which should win that fight.

Yglesias Predictions

1. Democrats lose both houses of Congress (90%): Scott holds – as above, hold here.

2. Democrats lose at least two Senate seats (80%): Scott holds
3. Democrats lose fewer than six Senate seats (80%): Scott holds

I also went with ‘he’s thought about this a lot, might as well go with it.’ Distribution continues to seem vaguely reasonable.

4. Nancy Pelosi announces retirement plans (70%): Scott holds

I sold to 60% as it seemed uncertain. Scott resolved this as ambiguous since it’s not clear if it means leadership or her house seat. I’m inclined to say Matt meant the leadership even if it’s technically ambiguous and I’m pretty sure that’s what I was predicting. I mostly like being modestly more uncertain.

Of course, noticing and avoiding ambiguous language is an important skill too.

5. Stephen Breyer does not retire (60%) N/A → Escalated quickly.
6. Some version of Build Back Better passes (60%): Scott holds, I sell to 40%.

The BBB question wasn’t graded, presumably because of the news in the interim being quite bad for its chances. I am pretty sure the IRA does not count and this did not happen, and if that is considered clear even 40% was too high.

7. Joe Biden is still president (90%): Scott holds, I buy to 95%.

Market says 91% but this is one of those classic Free Money situations where the people on the other side are being dumb. This was a good buy.

8. At least one Biden cabinet-rank official resigns (70%): Scott holds, I buy to 75%.

Happened. As I noted there are a lot of them. I don’t even know who it was. I like being high here, 75% still seems a bit low.

9. No military conflict between the PRC and Taiwan (a worryingly low 90%): Scott holds, I buy to 95%.

As I noted in my post, this is a lower threshold than Scott’s which is why probability is lower here. I do think 90% was a bit low here even with everything we learned, but 95% was likely a bit high. The chance of a minor flareup does not seem all that high, China either goes for it or it doesn’t and Taiwan isn’t starting anything.

10. New U.S. sanctions on Russia (70%): Scott holds, I sell to 60%.

This seems like a better sale than the sale on Russia going to war, by a substantial amount. So if I was being consistent I should have sold more here. Given that I was wrong about the chances of the war, the sale would have been bad, but I didn’t know that at the time. Therefore this still counts as a mistake not to sell more.

11. Saudi Arabia and Israel establish diplomatic relations (60%): Scott sells to 50%, as do I on principle.

This did not happen. I notice I would have been actively surprised if it did happen, which means being as high as 50% was wrong. Needed to be down to 40% or lower.

12. Fewer U.S. Covid deaths in 2022 than in 2021 (80%): Scott buys to 90%, I buy to 95%.

Happened. Should we have been this confident? Barring a new variant this was essentially a Can’t Happen, so it would have required a new variant that spread everywhere while also being either deadlier or able to ignore previous immunity with respect to death not only with respect to infection. Was that more of a 5% or a 10% shot at the time? I love the buy to 90%, less certain about the further buy to 95%. The market price here says 96%, which is surprising to me, I would have expected them to express more uncertainty. Given that, the 95% is obviously fine.

13. Emmanuel Macron re-elected 60% (repeat question, skipping)
14. Traffic light coalition exploits loopholes to get around the constitutional debt brake (70%): Scott holds, I sell to 60% on ‘this sounds strangely specific and weird.’

This isn’t listed in the spreadsheet. I looked up what this was and then looked at Wikipedia and it looks like they did get around the debt brake but did it by getting a two-thirds majority to vote for it. That’s not a loophole, so I’m grading this false, and my sale directionally correct for the right reasons.

15. No recession in 2021 (90%): Scott sells to 80%, I buy to 95%.

For a second I thought I was being super aggressive for no reason then I realized this says 2021 not 2022. Free money. Past is easier to predict than the future. Didn’t make it onto the spreadsheet. If 2021 was a typo, of course, 90% would have been absurdly overconfident.

16. Liz Cheney loses primary (80%): Scott holds, I hold

She lost in a landslide, and never had a chance. I wrote I wanted to sell, but thought better of it. We know this was in fact almost impossible, the question is how confident we could have been in this. The 80% number still seems reasonable, perhaps a bit low because Cheney had already made it clear she would rather lose with honor and dignity than give herself a better chance. I approve, but didn’t help her chances.

17. Some version of USICA passes Congress (70%): Scott holds, I laugh and sell to 55%.

It did happen on a bipartisan basis, and market was even higher at 80%. My sale was based on the prior that congress does not pass things, especially bipartisan things. Biden did quite a good job of passing bipartisan things, whether or not you think they were good things. Market, it seems, was right to anticipate this type of thing as likely.

18. Lula elected president of Brazil (60%): Scott sells to 50%, I sold to 40%, question is mostly duplicative it was a two-horse race.

Same thoughts apply here, except this is an even more explicit reminder that while we had to choose between Hillary Clinton and Donald Trump, we can take comfort in the fact that it really can be strictly worse on both counts. What a vote to have to cast.

19. China officially abandons Covid Zero (70%): Scott holds, I sell to 60%.

This did happen right at the end. My comment hints at the final result: Abandon Covid zero, have infinite Covid, simply decide not to count any of it. It’s a bold choice. Given how hard China fought to stall this decision, it seems right to have been lower than 70%.

20. Fewer U.S. Covid-19 deaths in 2022 than in 2020 (80%): Scott buys to 90%, I buy to 95% as per above.

21. Additional booster shots of mRNA vaccines authorized for seniors (80%): Scott holds, I buy to 85%.

Happened, as I expected. Seems hard to imagine a scenario where this didn’t happen, if anything 85% feels a little low.

22. November 2022 year-on-year CPI growth is below 6% (70%): Scott buys to 80%. I bought to 75%.

This came in at 7.1%, so it didn’t happen. Same mistake as the other inflation markets, so not much to say.

23. November 2022 year-on-year CPI growth is above 4% (70%): Scott sells to 50%, I sold to 40%.

My justification mentions the possibility of hedging in TIPS, so perhaps I wasn’t being as crazy as this now looks? Either way, more bad inflation take department.

24. The Fed ends up doing more than its currently forecast three interest rate hikes (60%): Scott holds, I buy to 70% saying it’s a dumb forecast.

Turned out to be a very dumb forecast. So this was a good prediction on its own, but the real question is, am I contradicting my inflation predictions here? I think no, this was a really dumb forecast and even with 4% inflation they would have hiked four times anyway, just less, and it was only my bad inflation takes keeping me as low as 70%. The market’s 84% seems much better and 70% was unacceptably low (despite the superforecaster average being 57%).

25. Russia does not invade Ukraine (60%): Scott holds, I buy to 70%.

Once again, my mistake, and a rather important one.

26. Viktor Orbán loses power in Hungary (60%): Scott holds, I sell to 50%.

Market is listed as 26%, and, well, yeah. No credit for selling only to 50%, that’s terrible, my logic of ‘people predict these things every year and mostly they don’t happen’ was exactly right and I needed to be much lower, at most 40% and ideally less.

27. Sinn Fein becomes the largest party in the Northern Ireland assembly (60%): Scott holds, I hold as not knowing anything.

Market was 80% so turns out this was more likely. In the absence of time or willingness to do more research it’s not obvious which direction I should have moved.

28. The U.S. and Canada reach an agreement on softwood lumber (70%): Scott holds, I buy to 75%.

I notice that given what I know now about the world, I would have been more inclined to sell even without doing research, saying that without knowing whether agreement was reached – Biden seems highly willing to fail to reach mutually beneficial agreements on trade and such things are generally a mess, and I should have known this at the time. Then when I looked at Wikipedia I realized this is a long standing dispute. So if I’d given this a few minutes and thought about it cleanly I would have sold down to at most 50% or so. And no, as far as I can tell we didn’t solve this.

30. Democrats go down at least one governor on net (60%): Scott holds, I bought to 70% on Covid restriction backlash.

This did not happen, Democrats had a good night overall and no one cared much about Covid so my logic was wrong. We all moved on. Was pretty silly to be more skeptical of Democratic chances than Matt was being, given his other probabilities.

Wait Till Next Year

I found broad support for the idea that I should do a 2023 predictions post, but I have less to work off this year in terms of source posts and also the events to predict mostly seem less interesting than usual. Commenters are invited to link to sources of others’ predictions to work off and of relevant prediction markets, or use other arguments or incentives, to try and convince me to do this. Currently I am leaning weakly against doing so.

The Substack version of this post has two polls at the end if you are interested.

3 comments

Comments sorted by top scores.