2024 ACX Predictions: Blind/Buy/Sell/Hold
post by Zvi · 2024-01-09T19:30:06.388Z · LW · GW · 2 commentsContents
International Politics American Electoral Politics US Politics and Government (excluding elections and AI ) Economics Science (Mostly Rocketry for Some Reason) AI Hard Fork: Bonus Buy/Sell/Hold None 2 comments
My 2023 ACX predictions showed a clear lack of confidence in taking on the market. I won 30 markets for an average of +185 each, and lost 12 for an average loss of -185 each1. When one goes 30-12, hitting a 71% mark versus the about 58% average price initially paid, that is worth noticing. It is possible that I generally benefited from 2023 being a year where not much happened outside of AI, but I think it’s time to know what we really think.
That means this year I’m going to add a phase, where I predict blind. Blind means I’m not allowed to look at any prediction markets. I can still look up facts, and financial markets are fair game, but nothing beyond that. Only after that will I look at Manifold. Metaculus makes this viable, as they have the ACX questions without listing probabilities.
Note that it would still be a mistake to make major moves on Manifold right away. While I did well in the end, I had plenty of opportunity through the year to double down.
The questions are listed on Manifold and Metaculus in a random order without being numbered, so I’m going to attempt to organize them reasonably.
If you are scoring me in the future for calibration or Brier score, this means you have three sets of numbers:
- Blind. No market influence at all.
- Bought/Sold to. Mostly market prices that I adjust slightly.
- Metaculus predictions. Me after adjusting for the market.
If you are comparing me to other predictors, please use the Metaculus predictions, although of course all the numbers have some interest.
International Politics
I hadn’t heard about this, so I read a few articles. The hypocrisy of comparing our reaction of ignoring the much larger tragedies and ‘war crimes’ that have played out around Ethiopia, while the same people cry (well, basically everything) and spew hate around other conflicts, is palpable throughout.
I got burned in Bayes points thinking (up until only days prior) that Russia was unlikely to actually invade Ukraine, one of my worst recent prediction mistakes. There have already been multiple live conflicts here, and there is a realpolitik reason for Ethiopia to fight for a Red Sea port – I know little about the area, but I was very surprised back in the day that they allowed themselves to be cut off when Eretria became independent. I still think ‘given specific thing or conflict actually happens’ tends to be unlikely, so I would only be on the YES side of this at very low numbers, and my blind guess for fair is something like 20%.
Manifold was trading at 31%, and linked me to this article. That sounds like a very good reason for Ethiopia not to get into another conflict, an excellent solution to their real problem of needing a port. So I’m going to sell to 25%, and say 22% on Metaculus.
This has to be a complete cease-fire and stand for 30 days and begin in 2024. On the one hand, wars are Lindy. Once they last, they are likely to last longer. This conflict has barely moved all year in 2023, defense is clearly favored over offense. There seems to be no ZOPA (zone of possible agreement), and no reason for the parties to agree to a lasting cease-fire without the outlines of a deal. Ukraine is not going to accept major territorial losses, and Putin is not going to accept going back to 2014 borders let alone going past them. I have not been following, but this war seems unlikely to end in 2024. If it does, I imagine it often being a frozen conflict without a formal cease-fire. So I’m going to say maybe 15% blind.
That’s where Manifold is currently trading, on the dot, so no trade, I will get better opportunities elsewhere. I’m predicting 14% on Metaculus because I expect Manifold to come in too high in such situations.
I am not following this anymore, but my general sense is this is a harder lift than it looks. It is a disputed region, but Russia paid a heavy price to gain Bakhmut over other more strategically important objectives in 2023. Losing it would be a major loss of face. Russia reliably cares more about face than Ukraine, whereas Ukraine cares about winning, so I think that Russia would sacrifice other areas to save Bakhmut, and Ukraine would happily accept that trade. So I’m going to say 20% blind without doing further research, but adjust to the NO side of Manifold almost no matter what.
They say 15%, so I’m going to believe them and then add my considerations on top, and sell to 13%, and put down 11% for Metaculus.
In previous years we have asked about a full scale invasion. If that was the question I would have put down 3% and been done with it. The party congress was last year, and I don’t think AI is going to drive them to make this move any time soon. However, this is a different threshold. Ten is not very many deaths, and accidents are explicitly included. According to GPT-4, this did not happen between 1970 and 2022, so it is still rather unlikely it happens at random, and I see no reason for anyone to engage in escalation dances, but I still have to tack on a few percent for the changed criteria, so let’s say 5%?
That is indeed where Manifold is trading, at 5%. Given hat Manifold should always be high in such spots due to cost of capital and longshot bias, I am going to go with 4% at Metaculus.
I sure hope not. Note that this is the only question here related to Israel on the list. Netanyahu is deeply unpopular, but seeking to hold on in a ‘don’t change horses in midstream’ style argument given the ongoing conflict, a conflict for which he personally has a large share of the blame. In general betting on a leader being ousted is a sucker’s bet, but in this case he seems clearly to be on borrowed time. This conflict does not seem likely to last a full additional year, and how does he hang on once that happens? So my inclination is to say something like 25% blind, but without much inclination to trade if the market is anywhere close.
Indeed it is not close, with Manifold coming in at a stable 53%, with 130 traders. I sold to 50%, hesitant to be on the structurally wrong side of the trade, expecting to get better prices by moving slowly and reflecting my low confidence. For Metaculus I will say 40%.
We ask this every year. He is 84 years old. All other considerations point to him being less likely than before to stop being supreme leader. Last year Metaculus was at 18% by mid-February and this was 7% by mid-year, reflecting a real annual rate of something like 10%. So I am going to say 10%, with a willingness to bet on NO but not YES.
Manifold clocks in at 22%. On reflection, yeah, I was too low, both because he might be stupid enough to get into a fight with Israel (market says 15%, I’d sell that but it’s still not zero) and because 84 really is more than 83, he is unlikely to last more than 10 additional years. So I’m going to sell to 17%. and input 14% at Metaculus.
There is no reason to expect this in 2024 in particular that I know about, but small skirmishes are more common than they are with China and Taiwan. I’d put the probability of a real war very low, but a skirmish? Bard gives us 2003, 2008, 2014, 2016 and 2019 since the last major incident in 1999. So that’s 6 incidents in the last 25 years, although it is not fully clear all of them would count, so call it 5 for a base rate of 20%. My guess is this has declined over time, so maybe 15%?
Manifold is trading at 19%. As with many such things, I expect it to trade high, and to keep trading high over time, so I’m going to sell to 17% and enter my 15% into Metaculus.
At this point I am highly confident that this will not happen in Ukraine, although not as confident as if there wasn’t a war going on of course. I am not worried that Israel or its enemies will do this. I don’t see North Korea doing it either. A test would presumably ‘kill 10 people’ in expectation via radiation but I assume that does not count. This should still trade above the baseline of zero times since 1945, but I’d say we are back down to a few percent. Maybe 5%?
This is trading at 1.5%. No, unless you are counting heavily on the ‘no one will have the endurance to collect on his insurance’ feature. I’ll throw the M10 minimum at it and get to to 2.2%. I’m going to enter 3% at Metaculus, in recognition of the incentives, and that it is generally terrible to bet on true longshots.
I accidentally got partly spoiled on this because I didn’t realize it was there, and saw that Manifold is at 6% before thinking about it. But also I think I would have gotten there anyway? Last year Metaculus was 20%, and the chances are now much lower, I’d basically say this only happens if Russia does it on purpose, plus some very low baseline rate of accidents. Russia was willing to play some forms of chicken here to get an edge but that is miles away from doing this on purpose. And if they were keen on that, why wait? Also you would want to wait for the election and Trump, who might let you get away with it. So this should clearly be in the single digits. Maybe I would have said 5%? But it is hard to know. I will sell to 5% on general principles and enter 4% at Metaculus.
I have no idea. Nor does it seem worth the effort to be able to have an opinion.
So I’m going to accept the market. It says 83% chance Claduia Sheinbaum wins, 15% Xochitil Galvez, 1.1% Movimento, 0.8% other. I’m copying that over to Metaculus.
American Electoral Politics
This one makes it impossible to be fully unspoiled, of course, but I will do my best anyway, and ask what I would believe with no predictions.
First, the easy part. I will take green down to the minimum of 0.1%. This is not a thing that is real. I will take Libertarian down to 0.3%. There is some chance that someone unexpected steps up, and uses the ballot line. Maybe they will back RFK Jr. and things go completely nuts, or Marc Cuban, or Marc Andreessen or Mitt Romney surprises everyone, who knows, indeed do many things come to pass. I can even see Trump ending up on this line in theory, in some narrow weird cases. But they’ve made themselves rather toxic and effectively been taken over by something similar to the alt-right, which is a damn shame. I really hope this is fixable.
What about other, a true independent, perhaps including RFK Jr? I’ll say 0.6%. These are historically deeply unpopular candidates in Trump and Biden, and also Trump could run independent if the Republicans somehow snub him although I don’t see how he wins a 3-way race if he does but maybe the Republican drops out late or something.
So that’s 1% between the three scenarios, leaving 99% for the major parties.
I expect a much higher probability than the markets say that this is Trump vs. Biden. If we assume that is the case, who is the favorite?
I think it is Biden. Yes, the polls do not look good right now, but that does not mean so much when you look at the historical record. I also think those polls reflect negative polarization, with Biden being in people’s thoughts and Trump mostly not being in them, so people vote NO on Biden rather than YES on Trump.
Once people focus on both candidates equally, it will look different. Remember that time Trump sort of, kind of committed what those courts keep calling ‘insurrection’? You might want to forget, except I expect both candidates to remind you.
I expect economic conditions to improve. Inflation will have been not super high for an additional year and probably decline further. Interest rates will probably be down, with the Fed being able to cut. That will impact felt housing costs, and I expect that to be a big deal. AI will be contributing to growth more every year, and people will start to see improvements in products there. To the extent AI becomes an issue, Biden is by all indications going to end up on the popular side, Trump on the unpopular one.
Right now, Biden is being blamed for everything and credited for nothing. There is nowhere but up to go from there. The Israel-Hamas conflict is especially causing big trouble right now but is unlikely to last another year.
I expect both sides to run terrible campaigns. Neither will listen to their data people. Democrats will continue to say lots of actively counter-productive things, demand extreme position taking, say that anyone who disagrees with them is beyond the pale and so on. Trump will keep trying to relitigate 2020 and talk about Hunter Biden and how he’s going to punish his enemies, when he would be wise to shut up about all that and talk economics and immigration all day.
There’s also the real chance that a criminal conviction happens. I do think that would be bad for Trump.
Biden’s disadvantage is that he will run a not so vigorous campaign, but this could be a year where that is not such a bad idea, since it is all about who you hate less.
So in the Trump-Biden scenario, I think I give Biden something like a 65% chance of winning.
If Biden is somehow not the nominee, then is it Harris? If it’s Harris, then that’s actively worse. She is not a good politician, and everyone dislikes her. If we end up with ‘generic Democrat’ instead, even if it’s a joke like Newsome, then that already seems great for Democrats. If it’s someone actively talented and without baggage, like Mayor Pete or Gretchen Whitmore, they’re large favorites. The longer we wait, the more likely it would have to be Harris, but I still think this is a net positive. I also think it’s a lot less likely than the markets do.
If Trump is not the nominee, and he endorses Haley or DeSantis, that is very good for Republican chances. Neither is an amazing political talent, but not being Trump is worth several points. The flip side is, what happens if Trump runs third party or otherwise flies into a rage? If that happens, then it’s over, and I give him a real chance of doing that, unless he is barred from running by the courts or his health, but also I think that’s most of what stops him.
So I think on net maybe 60% chance of Democrats winning if I actually ignore all the markets? The actual PredictIt market is 52% for Democrats, 48% for Republicans, 1% Libertarian. I am confident splitting the difference and saying 55% Democratic, 44% Republican, 1% Other.
I’ve been listening to the Serious Trouble podcast for fun, so I have some information about the state of play. It seems likely that at least one case will resolve before the election, likely two or more. Trump will try to stall as much as he can, but the courts are doing their best not to let him. Purely in terms of the underlying facts, as a matter of law, he is guilty, the evidence very strong. But it only takes one juror to not convict. So what will happen?
Unfortunately, a ‘related question’ at Metaculus spoiled me at this point, which said 60% chance of a conviction before end of the year. There is some chance he gets convicted in November or December, especially if he first loses the election, so this is representing a 50% chance or so. That was actually where I was leaning, in a mostly unprincipled way, and on reflection I’d be a bit higher. There are multiple venues where I think the jury is unlikely to hang, and juries tend to uphold the law, but anything could happen. I’ll say 60% semi-blind for the question here.
The Manifold market was trading at 35%, which is clearly out of whack with many other markets, so I bought to 42%. I also found another market with a resting 30% NO order, and took it out. I put 50% into Metaculus.
This is a parlay of three things. Biden and Trump must both be running. Biden won’t run without the nomination. Trump might, so we need to factor that in, and he would definitely poll above 15% if he did. Then there’s the question of whether Trump agrees to debate, which he didn’t in the primary. I think he happily debates Biden, he thinks Biden looks weak and he can win that debate, whereas he correctly didn’t want to take the risk in the primary of giving others stature or opportunity.
So I’d say something like 90% chance Trump agrees to debate, 90% chance Biden is the Democratic nominee (market is at 78% and this is crazy but I am trying to show some respect and health is a concern), 88% chance Trump is nominee or runs third party (market says 76% chance of nominee, which seems low). So multiply them together and we get 71%. I’ll take off a percent for random factors, call it 70%.
And indeed, the market at Manifold says 72%. So I won’t trade there, that is close enough.
I hate that this is worded as a plural, but as written the minimum seems to be one. As one comment notes, I would assume it is not ‘faithless’ to change votes if the candidate is dead, ruled ineligible by SCOTUS or otherwise clearly unable to serve, but then if they are faithless to the successor I’d think that would still count? This is about if someone goes rogue. I also assume it doesn’t count legal moves to exclude or replace electors in various ways a la 2020’s attempts to do so.
This is a rather common recent occurrence. It happened in 1976, 1988, 2000, 2004 and 2016. There were a full ten in 2016. It is interesting that they managed to avoid this in 2020. Further in the past, we have 1960, 1912, 1872, 1836 and 1820.
Presumably both parties will make a real effort to stop this from happening, since there are rather plausible scenarios where the election is close. It would also be unsurprising if there are people who would be tempted to do this, perhaps scheming to get in position to do this, and no doubt many will try to make this happen. Trump will no doubt call for it to some extent, whether or not he is still running. There are also going to be more laws trying to further bind electors, for what they are worth.
This looks a lot like a coin flip. I am going to say 50% blind?
Manifold says 27%, presumably thinking legal restrictions tightening combined with better vetting. Unless there’s a lot of new laws out there, that seems clearly too low. I wonder if this is a case of people not realizing the base rates. I definitely don’t have super high confidence here. I’m buying to 33% and entering 38% for Metaculus.
US Politics and Government (excluding elections and AI )
Every time, it seems both impossible that there will not be a shutdown, and crazy to think that they will not once again work things out. Occasionally there is a shutdown. We have 1981, 1982, 1986, 1990, 1995, 1995-96, 2013, 2018, 2018-19.
We came damn close in 2023, and only avoided it when the Speaker of the House effectively fell on his sword. Mike Johnson has a plan to avoid a shutdown. But fundamentally, the house consists of Democrats who won’t help a Republican be speaker, Republicans who are reasonable on this but constitute a minority, and a few Republicans who demand that unreasonable demands be made or else. That is not a stable situation. Will an election year keep them in check, or make it worse? Will they ever learn their lesson?
Or will everyone else learn the lesson of not letting a few nuts hold everyone hostage?
I know, that sounds crazy, but it could happen.
My guess is that Mike Johnson will do other symbolic things to keep the peace and then, if necessary, agree to fund the government with Democratic votes and dare the Freedom Caucus to remove him. Solve for the equilibrium, median voter theorem, plus expectation that a shutdown would help Biden get reelected. But I am not so confident.
So I’ll say something like 30%?
And indeed, the market says a steady 35%. I’ll buy it down to 33% for tracking, and I’ll put 31% into Metaculus.
That’s a great question. Not in the sense that it is important, but in the sense that it is fun to think about, and I’ve never seen a prediction on it.
I am confident that the Democrats will not lift a finger to save Mike Johnson. The question is, when the time comes to let the government continue to function, will the Freedom Caucus decide that this is unacceptable, the way they did with McCarthy? Or, if Johnson shuts it all down, will moderates pull the plug? Or if Johnson shuts it down and then is forced to fold, as he would presumably have to (although I do think there is a small chance of real ongoing stalemate that goes off the rails, it seems impossible until it isn’t), what then?
This is going to be a tough spot. Johnson is as friendly a face as they can hope to get. If they bring him down, who can replace him? The moderates caved last time because they always caved, but if the demand is someone to the right of Johnson willing to actually shut down the government indefinitely, I suspect that’s when they actually do grow a spine, or credibly threaten to do so.
Meanwhile, everyone involved really wants it to be over, and everyone will want to be off campaigning, and everyone will know that taking out Johnson will hurt at the ballot box.
So my guess is that Johnson remains in place, but crazy is as crazy does. 80%?
Market agrees, says 81%. Feels really good to nail the market blind. So no trade, I did buy very tiny YES so I can have it listed in my portfolio to trade later, and I’ll be entering 80% in Metaculus.
Could Biden get a new pick through the Senate? My presumption is that he could. If everyone involved was wise and cared about the composition of the court, Democrats would replace Sotomayor and Kagan, as they might not get another chance for a long time. But are they going to voluntarily step aside where RGB didn’t? Not on your life.
I certainly don’t think they ever dare do it on purpose in a lame duck Congress. I’d be in awe if they tried.
I do think this will make them more willing to step aside if they have severe health issues, as would Jackson, but I presume that is what it would take. The other six would need to flat out drop dead. Taking all the bribes clearly is not an issue, so what else would be?
So we have ages of 69, 76, 70, 55, 57, 59, 53, 54 and 73.
I asked GPT-4, which was surprisingly reluctant to use actual data and got 9.1% and seemed to ignore probability to incapacity.
Another sanity check is that there have (quick check) been 29 justices appointed since 1950, which would imply about a 30% base rate per year, which we should lower since people are living longer and justices are more often refusing to retire, and we have a younger than average court right now and also no pending health issues we know about matters a lot.
So it sounds to me like the 9% is the lower bound, so the answer is something like 15%.
That’s what some people in the Metaculus contest got for the mortality number, but I think the unknown factors favor the mortality number being lower than a basic calculation, as we have the evidence that we haven’t seen evidence of an issue, and they’ll have access to the best care including heroic measures, and a strong will to live.
So I’m going to sell down to 18%, and enter 16% at Metaculus.
This has not happened since the early 1990s. They peaked at 85k under Obama in 2016 before declining under Trump, then Covid crashed the numbers, now they are picking back up with 60k in 2023. What is described as a ‘migrant crisis’ seems to me more along the lines of standard historical precedent? This is what happens when you don’t have a Trump or post-9/11 crackdown?
The question is for some reason fiscal year 2024, so we already have three months in the books. It looks from this chart like we’d need to have another increase as large as last year’s, last year’s was the largest on record and reflected a Covid rebound. Biden will have strong incentive not to let this happen. If I had to guess I’d say 15% chance of a spike that big, given the question is being asked, but I’d hate to be on the YES side of that?
Indeed, Manifold said 16%. I sold to 15% to keep an eye and will put 14% at Metaculus.
Why is this question here? I have seen zero concerns raised that there could be an issue. I do get that weight loss drugs have a habit of causing issues, but this thing has been widely on the market for a while for diabetes. This seems highly unlikely, unless I am missing something. If this question wasn’t here I definitely would have said something like 2% and plausibly lower. Manifold says 7%. I am confident that is too high. I sold to 5%, stopping there due to capital costs. I’m going to put 3% on Metaculus.
The Biden administration wants this to happen. It looks like the decision will happen this year. I am sure it will help their reelection efforts, legalization is at 70% in polls. The FDA has signed off. HHS will rule next, DEA has never disagreed with an HHS recommendation. Why would the administration not be able to do this? I’m going to say blind this is 80% to happen at this point, as I don’t see a reason why it wouldn’t.
And we have our first large disagreement. With the market at 34%, although there are other similar markets trading modestly higher. I bought to 40% including in a similar market, there is no rush if I want to get more.
The question is what to put into Metaculus. I am going to say 55%. That seems simultaneously like the height of arrogance and also quite cowardly, so that should mean it is about right.
Economics
No. Elon Musk talks a good game, but he is not in the habit of letting things fail. Nor is he, as we call it, broke fi broke. There are some very powerful interests in not letting this happen, and the amount of money Twitter bleeds will be relatively minimal, so this is highly unlikely to happen.
What’s the chance I am wrong and it happens anyway? Twitter would have to be bleeding a lot of money, or otherwise be in deep trouble. My presumption is that legal trouble is the only way this happens. If the FTC or EU is at their throats things could become unsustainable. Is the Biden administration willing to single out and kneecap Twitter that blatantly? We will find out. I’ll say 10% chance, noting that I briefly saw a number by accident that was something starting with a 1 while navigating other pages, which I did my best to ignore but one is never perfect. Turns out it was 11%, so my 10% is in line.
Which, in these situations, means it is too high. I sold to 10%, and I’ll be putting 8.5% into Metaculus.
Only one of these this year so we can’t do arbitrage beyond buying Bitcoin. The price is almost exactly unchanged so far, so this is a clean question as of writing this on the afternoon of January 6. As always, Bitcoin is asymmetrical. If it goes up, it could go up a lot more than 100%, if it goes down there is a hard limit. A few percent inflation won’t make up for that. So economically this must be an underdog, and if anything I think Bitcoin has more relative downside than usual at the moment (nothing I say is ever investment advice and all that). So this should be an underdog, but how much? I’m going to say 40%.
The manifold market says 65%, so I will buy it down to 60%. In all seriousness, if you think Bitcoin is 60% or more to rise in value in 2024? It is a screaming buy, go buy Bitcoin, and don’t trust the relatively tiny prediction markets on this. I’ll put 42% into Metaculus.
All the professionals say yes. Statista projects 2.3%. FOMC predicts 2.4%. Economists say 2.5%. Right now we are already, officially, at 3.1%.
So this is a strong favorite by all accounts. A lot of thought and money has gone into this. I don’t know what this translates to as a probability instinctively, but I know a number above 3% would be very surprising. But also I would not be so shocked, we still did print an awful lot of money and there’s a huge deficit and the economy is remarkably strong. Dare we say 20%?
Oh, we are. The Manifold market says 32%.
I forgot to check TIPS. The 5 year and 10 year TIPS are at 1.83%, versus a 5-year treasury at 4.02%. So that’s implying 2.2% over five years, which would be hard to get to if we were stuck above 3% for a year. I also forgot that core inflation is still 4%. And this is asking about core.
Manifold is kind of calling BS on financial markets here. I kind of thought I was doing a little of that, they’re doing a lot more. I’ll sell a little down to 29%, but I think my blind number was a little optimistic, so I’m willing to predict 27% at Metaculus for now, I think I was too hasty with the blind number.
The S&P is also flat so far. In the last 50 years the S&P has gone up 68% of the time. Number tends to go up. With AI on the way and my expectations that general conditions are good, and all the motivations involved, I’d say we are a bigger favorite than usual, and I’ll say 72%?
Manifold says 78%, which I sold to 76%. That’s confidence. I’ll make it 73% for Metaculus, but no reason to get overconfident.
Unemployment is holding stubbornly steady at 3.7%. Last year people were about 70% confident it would go from 3.7% to 4%, with inflation needing to come down. Now inflation is mostly under control without doing that. Our chances should be much better. I’m going to say 35%. Market is only willing to go down to 46% on this. I sold to 42%, will likely sell more over time unless there’s a reason not to, and enter Metaculus at 38%.
This has been very different from the S&P. It began in 1990 and only has gone up 44% of the time in a given year, with much higher volatility. China has a lot of headwinds right now, and it seems like its prices might be a bit artificially high in an unsustainable way. I expect Chinese policy to be disappointing for business most of the time relative to the median, given Xi. And it’s down 1%, which does matter. I still have a hard time being too skeptical, but I’m going to say the 44% seems fine here?
Whereas Manifold says no, it’s like America, this is 67%. I bought a little no down to 62% and I’m going to enter 50% into Metaculus. That’s what we call being bold.
The Fed funds rate is 5.25%-5.5%. We need six 25bp rate cuts to get there on time. There is no timetable on cuts. Traders are pulling forward rate cut expectations. I think they’re wrong to do so. Nothing that has happened seems to point in that direction? It does not seem that we will need it so urgently? Unemployment is 3.7%, inflation is well above 2%, why cut? Powell is not fooled into thinking his inflation fighting work is done. Bloomberg says FOMC forecast 3 cuts in 2024. Futures traders are expecting six starting in March.
I am loathe to bet against the market when not getting market prices, and I don’t know what the market will price this at, but I am going to say 50% based on traders expecting this to happen so I don’t get too out of line. With the understanding that I can only sell, I want to be below whatever Manifold will say.
The Manifold traders are smart. They have it at 45%. Which implies either I was looking at the wrong expectations, or they’re willing to flat out call BS on the traders and instead believe the Fed itself, which I love. So I’m going to sell to 41% and put in 40% at Metaculus.
Quick research says this is already up to 9.84% for 2023. So 11% is almost a done deal. That is what an exponential means. All the projections are higher than that, and the projections are always too low on exponentials. My worry is that I might have misunderstood the question or miscalculated? Otherwise why ask the question at 11% rather than 13% or 14%? So I’m going to say 85%.
Manifold says only 59%. I bought to 66%, which makes me the biggest YES holder, but as always I’m not going to ram through the market. I’m still predicting 80% at Metaculus.
Science (Mostly Rocketry for Some Reason)
I have no idea, any more than I did last year. I do expect people to be overconfident. Things tend to go wrong. The government might try to shut this down. Last year this traded at 84%. This year I would think should trade lower, even if we do get to launch we are in a better position but we have bad news about the old position, including how the government will react if we fail the first time? So assuming I’ve been told the truth that we are cleared to try, I’m going to say 75% but as one of those ‘you must always be below market’ situations.
The market says 81%, which is what I expected it to say. So I sold a tiny amount to 79% and called it a day, and will be entering 75% at Metaculus.
I mean, sorry, what? Phind says yes, they are going to do that. I wish them luck. I always want space missions to work out. It is not clear to me what counts as an attempt to catch the booster, or what stage things have to get to before it counts as an attempt? My read of the Phind answer is that an attempt is highly likely, but a bunch of stuff can go wrong. I honestly have no idea and intend to fold right away on so many levels, but I’m going to throw up my hands and say 50%?
It turns out the market says 56%, so good punt. I sold the minimum to 55%. I’ll give 53% to Metaculus.
Today I learned there is a planned manned moon mission. It sounds like Apollo 13 style failure counts here. So the question is, if NASA plans a mission for November 2024, how often does it delay, and how often does it succeed?
Historically, Apollo was often delayed a few days, but never more than a month. Artemis I was delayed two days, and importantly succeeded at the criteria here.
Apollo 1 exploded, but the remaining 10 would all have counted including Apollo 13.
If you look at Gemini, Skylab and Mercury, we have 19 successes and 2 partial failures, both involving docking.
The Space Shuttle has 135 launches with 133 successes, plus Columbia reentry failure and Challenger’s failed launch.
The quality of NASA has likely been declining over time, but also this is known tech. There is some chance things are in various forms of chaos by November that scrap the launch through no fault of NASA, but I’ve seen no indication that budget negotiations threaten the mission.
So I like our chances here at first glance to approach the moon. Given Artemis I succeeded this far, I’d say 85% after all the various things that could go wrong?
That was fully blind, I mean fully blind. And wow, Manifold does not think this is happening. They say 17%. And yeah, on reflection I should have worked in a lot more unknown unknown level uncertainty here, I literally heard about the mission a few minutes before the prediction. I’m sure I forgot some factors. I bought to 21% since I kind of have to.
But Metaculus shows this is because everyone expects delays. My presumption is that when I was thinking ‘delay’ in the record it was from much farther along in the process, so I had the wrong reference class. So whoops, I guess.
But still, this 17% seems really low? Especially given it looks like they are giving Artemis II a very high success rate once it does launch? I’m going to say 22% at Metaculus.
My willingness to fold here is indicative of my level of confidence and knowledge, which is, as they say, very low. I consider my real mistake overconfidence on model error, and I ‘should have’ said 75% instead, then made the adjustment to 22%.
That is not a high altitude in context. The flight is planned for August 2024. This seems like a relatively safe test? So I’d be inclined to say it probably works on instinct? But it definitely does not seem worth doing more than one minute of research on this, I very much don’t care, so given what I see I’m going to say 65% and move on.
Looks like the market says 45%. I mean, sure, fine. 45% it is, I say to Metaculus.
We did this last year. I concluded that should have been 30%. So this probably should also be 30%, says the blind man. Indeed, the market affirms at 28%, which of course made me remember I’m an idiot, I said I should have sold to 30%, not that I should have core believed 30%. It’s still happened 7 times in the last 30 years. So really I should have said 25%. I sold tiny to 27% and will be entering 25% at Metaculus.
AI
These definitely aren’t the questions I’d have thought to ask, except perhaps the coding contest. Regardless, let’s go.
There is a huge difference between doing well and winning in a contest like this. The best programmers are way, way, way better than the average ones. Right now, yes, we can do 85th percentile, but that is more than a year from 99.99th percentile. The AI does get human filtering of answers, so that helps, without that it seems hopeless. My blind guess is that this looks way harder than one year of progress. Indeed do many things come to pass, but this seems like a 10% type of thing?
The market says 16%. I bought a bit down to 14% and will stick with 10% for Metaculus.
OpenAI does not publish that many papers. I think this is one of its big advantages, that when it finds something it doesn’t feel obligated to tell everyone. If they did develop Q* and it was worth talking about, it would be worth not talking about. We also don’t know that it is anything worth talking about. Of course, the bar for description here does seem pretty low, so it could easily happen. It’s such a weird question. I don’t know. Let’s say 35%?
Market says 26%. This is one of those places where I don’t want to be betting high, and I probably didn’t discount enough for it perhaps not being real. So I’m going to agree to the market. I bought the minimum to 27% because I want to be able to track it later, and expect to be on NO by midyear the majority of the time if I pay attention.
- Will Ilya Sutskever still lead OpenAI’s Superalignment team at the end of 2024?
This is very similar to asking whether Ilya will stay at OpenAI. I have a hard time imagining Ilya agreeing to stay at OpenAI if he was forcibly removed, and a harder time seeing him choose to do something else and also be allowed to do that something else. So this is mostly, with a discount, a market on how often he stays put in the wake of recent events.
Ilya is presumably at this point awaiting the investigation into what happened before deciding what to do. He could also be preparing his next move. My gut tells me that Ilya’s timelines are short, he sees OpenAI as the place that matters, and he has no intention of leaving and wants to hunker down and solve the problem, but of course I do not know. My wild guess is that 60% of the time this holds, discounting for various ways it could technically fail to happen?
Bingo, says the market, 60%. I placed a nominal tracking bet on the YES side.
It says there are 37 questions and I count 36, but I can’t find the missing one. If someone points it out, I’ll edit it in, don’t tell me anything expect the question wording.
Bonus Question: Whose Predictions Will Do Best?
I appreciate everyone’s sight unseen confidence in me!
All the generic averages of Metaculus will be similar to each other, so one can say they will win or lose ‘as a group’ and then if they win you get into the nitty gritty of which one won. The meta option is fun, and mostly depends on how many options get created.
I thought about doing the Matthew Yglesias predictions. I’m going to not do that, because (A) they are largely about political outcomes in ways that seem uninteresting to me and (B) his prediction post is behind in paywall.
Hard Fork: Bonus Buy/Sell/Hold
Hard Fork, however, seems have some high-participation markets. I unfortunately did not see an easy list outside of Manifold, so I won’t be able to do these blind.
Numbers in parenthesis here are Manifold. Vagueness is an issue throughout.
H1. Will Elon Musk get his own Hunter Biden laptop scandal in 2024 (25%)?
I sold this to 23% and declare a ‘fair’ of 15%. I do not think Elon Musk will do this. He has essentially no motivation to do this in either direction, rather he is a toon.
I also think that the Hunter Biden laptop story was unique, in the sense that if there was no Hunter Biden laptop, then Twitter would not have suppressed some other story instead.
H2. Will the Apple Vision Pro be successful enough to revive interest in mixed reality and the metaverse (45%)?
This is especially vague. What does it mean? If interest is revived but not due to the Vision Pro, then what? If it’s unclear? It’s so crazy this has 410 traders and no idea how it resolves.
My confidence that interest in mixed reality and the metaverse will see substantial renewed interest is high, let’s say 75%, as the technology catches up. So the question is how often Casey then says this is due to the Apple Vision Pro. I don’t think commercial success by Apple will drive this even if it happens, the tech and proof of concept matter more. I’m cautiously going to say I want the YES side of this and buy a bit to 46%, and I’d say maybe 55% fair, but there’s nothing like this kind of market to make you appreciate Metaculus grading guidelines.
H3. Will white-collar workers unionize to prevent job loss to AI/automation in 2024 (20%)?
This market will resolve to yes if there is significant new union activity in at least 3 of these 5 fields by December 31, 2024: law, finance, medicine, media and tech. To count, unions don’t need to be NLRB-certified by the end of 2024, but preventing AI and automation-related job loss should be among the organizers’ top stated priorities.
If it happens in law or finance I will be in eat my hat territory. That’s not how this would play out. And yes, we might see unionization in tech, but in order to prevent automation? Really? So I see three of these five as really unlikely. Media I can believe. Medicine seems too early, in the sense that I expect doctors to fight via the AMA and lobbying rather than via unions, but this is at least possible. I sold this to 18% and anticipate taking a large NO position, my fair is more like 10% at most.
This has a double digit chance to have highly uncertain resolution as worded. My guess is that Google does cut into market share. Bard is already much more usable than it was, not obviously worse than free ChatGPT. If you let me get Ultra at a reasonable price it would be unsurprising to me if that mostly matched GPT-4.
I am treating this market mostly as a ‘will OpenAI substantially improve in 2024’ market, with a redraw for Google. Google is still substantially behind in time, so I assume OpenAI will find ways to keep pace. But I wonder. ChatGPT has been disappointing, to the point of in some ways getting actively worse. GPTs are a bust so far, Dalle-3 cut back to be less useful, many outputs are now full of long caveats and lists and repetitions, and I’m increasingly tempted to use Bard even as-is just because it is more down to Earth and more likely to know facts. I initially sold tiny, but I’m talking myself into thinking this is more than 36% likely, and indeed am reversing course and buying small to 37%. I think I’m making this closer to 45%.
H5. Will a lawless LLM chatbot get 10 million daily active users by the end of 2024 (51%)?
I was already on the NO side of this before I knew it was a prediction from Hard Fork, and I’m still on that side. To qualify it has to be willing to answer essentially all sexual and violent requests. The toxicity level of that shouldn’t be underestimated. My guess is that it won’t actually be easy to get a version that answers such queries ‘well’ either, because of how the models are being trained right now, unless you trained with such lawlessness in mind. Who is going to do that? We shall see. My fair here given the market is still around 40%, someone could totally do this well enough to get the users and I respect the market, but my guess is no.
H6. Will threads overtake Twitter in daily active users by the end of 2024? (9%)
No. I’ve already piled into a different version of this for size, and got someone to lift my 10k sell order at 15% a few days ago. Looks like that gravy train is over for now. At only 9% I’m not that interested in adding more, since this isn’t actually impossible, but I think it’s something like 5% and declining steadily. Threads is a miss in the style of Google+, so this only happens if Twitter implodes.
Casey’s other predictions are here, was a good source of easy arbitrage on the others as well.
H7. GPT-5 begins training (90%).
It would be rather surprising if it didn’t. 90% seems fine.
H8. TikTok skates unscathed through another presidential administration (84%).
What happens during the election is an interesting question, but also moving against it during an election seems super unlikely. I would watch out in 2025, no matter who wins, but yeah this seems if anything a bit low, more like a 90% at this point if we are not being true sticklers on ‘unscathed.’ Time is almost gone. I bought to 85% but this is not an exciting opportunity.
H9. The quality of search results degrades as Google proves unable to reliably detect AI-generated content (83%).
Magnitude is a question but this seems almost certain. I bought to 85%, would again have something like a 90% fair. If it doesn’t happen, it is because Google uses Gemini Ultra to improve the search results and pull us back from the brink. I don’t think Google can pull it off. Prove me wrong, kids.
H10. Fragmentation in social media will make it more difficult to understand which narratives are resonating most with voters (77%).
Yeah, what does that even mean? I don’t think it is actually true in a super meaningful way, but I’m going to basically N/A this.
H11. Meta kills off its incomprehensible celebrity-based AI characters (70%).
Low liquidity, I bought to 73%. Meta seems like it will know to throw in the towel on this, especially if ‘replace with new versions that are completely different’ counts. If it doesn’t, and the question is whether there are celebrity versions around at all, then this is likely trading too high. Again, vaguness.
H12. The US presidential election is full of synthetic media, and it mostly doesn’t matter (68%).
I am big on the second half. It mostly won’t matter this time around. The question is whether the election will be full of such media, and what threshold that represents. If it’s 5% synthetic, is that ‘full of’ such media? 10%? My guess is that I will think this resolves no, but that Casey will still often resolve it yes, and I don’t need that kind of trouble, so N/A, and the 68% for how Casey resolves it seems fair.
H13. Scaled-up AI-generated sludge outcompetes many digital media companies for advertising and affiliate-link dollars, sparking further waves of job losses and consolidation (67%).
In digital media specifically, that is? I sold this a bit to 63% on low liquidity, my fair is likely something like 50% but again what is a ‘wave’ of either of these things?
Already a few days later, as I prepare to post this, some of these numbers have moved substantially. I am not, however, going to update for that, since that process would otherwise never end.
Will my predictions, or Manifold’s maximize Brier score? You can trade on that at the link. Right now the odds favor Manifold. They definitely have an edge on the know-little predictions, especially Artemis II, where I presume I simply messed the thing up. In other places, I like my chances.
I am considering whether to do a second prediction post with select additional markets, the more good topics I have the more likely I am to do it. I am less interested in elections, especially foreign elections, where the research seems not useful, more interested in developments in AI or that otherwise feel impactful, including longer term questions, and also in personal stuff,, which I’ve been experimenting with. Don’t forget to check out my questions for ideas if that appeals to you.
Finally, a poll, and yes I am aware of the ‘98% of poll respondents like responding to polls’ issue:
Loading…
This includes every market where I won or lost at least M1, and includes all subsequent trades as well. The subsequent trades give me an unfair advantage, since Manifold creates opportunity, but this is still pretty crazy. There was one market where I didn’t trade but would have clearly been on the losing side if I had dared to, but I certainly wouldn’t have counted that as a win were it reversed, so I didn’t count it.
2 comments
Comments sorted by top scores.
comment by isabel (IsabelJ) · 2024-01-15T07:53:02.403Z · LW(p) · GW(p)
It says there are 37 questions and I count 36, but I can’t find the missing one. If someone points it out, I’ll edit it in, don’t tell me anything expect the question wording.
The question that you're missing is "Will a member of the United States Congress introduce legislation limiting the use of LLMs in 2024?"
comment by NoUsernameSelected · 2024-01-10T05:58:29.376Z · LW(p) · GW(p)
Artemis II has already been delayed to 2025, as of yesterday.