The EMH states that “asset prices reflect all available information”. The direct implication is that if you don’t have any non-available information, you shouldn’t expect to be able to beat the market, except by chance.
But some people were able to preempt the corona crash, without any special knowledge! Jacob mentioned selling some of his stocks before the market reacted. Wei Dai bought out-of-the-money 'put' options, and took a very handsome profit [LW(p) · GW(p)]. Others shorted the market.
These people were reading the same news and reports as everyone else. They profited on the basis of public information that should have been priced in.
And so, the EMH is dead, or dying, or at the very least, has a very nasty-sounding cough.
I think that rumours of the death of efficient markets have been greatly exaggerated. It seems to me the EMH is very much alive-and-kicking, and the recent discussion often involves common misunderstandings that it might be helpful to iron out.
This necessarily involves pissing on people's parade, which is not much fun. So it's important to say upfront that although I don't know Wei Dai, he is no doubt a brilliant guy, that Jacob is my favourite blogger in the diaspora, that I would give my left testicle to have Scott's writing talent and ridiculous work ethic, that Eliezer is a legend whose work I have personally benefited from greatly, etc.
But in the spirit of the whole rationality thing, I want to gently challenge what looks more like a case of 'back-slaps for the boys' than a death knell for efficient markets.
First: how the heck did the market get the coronavirus so wrong?
The Great Coronavirus Trade
Lots of people initially underreacted to COVID-19. We are only human. But the stockmarket is not only human—it’s meant to be better than this.
The stock market is a giant coordinated attempt to predict the economy, and it reached an all-time high on February 12, suggesting that analysts expected the economy to do great over the following few months. On February 20th it fell in a way that suggested a mild inconvenience to the economy, but it didn’t really start plummeting until mid-March – the same time the media finally got a clue. These aren’t empty suits on cable TV with no skin in the game. These are the best predictive institutions we have, and they got it wrong.
But… this isn't how it went down. As AllAmericanBreakfast and others pointed out in the comments, the market started reacting in the last week of February, with news headlines directly linking the decline to the ‘coronavirus’. By the time we get to mid-March, we’re not far off the bottom.
(You can confirm this for yourself in a few seconds by looking at a chart of the relevant time period.)
EDIT: Scott has explained his rationale here [LW(p) · GW(p)]. Although I still think his version of events is incorrect as phrased, I want to make it clear I am not accusing him of deliberately massaging the data or any other such shenanigans, and the next paragraph about revisionist history etc was only meant to be a general observation about how people responded. My apologies to Scott for the unclear wording, as well any perceived slight against his very good reputation.
For whatever reason, COVID-19 seems to be a magnet for revisionist history and/or wishful thinking. In other comments under the same post, the notion that people from our ‘tribe’ did especially well also comes under serious question—in fact, it looks like many of the names mentioned seem to have jumped on the bandwagon after it was obvious, and certainly long after the market was moving.
The facts are that the market reacted faster than almost all of us. But not before a few prescient people placed their bets!
So now the question becomes: why didn’t the market react earlier than February 20, like those smart people did?
The null hypothesis is that the market reacted exactly appropriately on the basis of the information available. After all, there were other potential pandemics in the recent past that were successfully contained or eradicated.
On February 20, there were only 4 known cases in Italy. We were a long ways from the bloodbath that was coming. Maybe it was correct to move cautiously until further information came in?
EDIT: previously misattributed to AllAmericanBreakfast (who agrees with it).
[At that time] it may have been plausible for many people to think this would continue to play out like SARS – East Asia would solve their problem, everyone else would watch airport arrivals and quarantine them effectively, and within a few weeks everything would stabilize and gradually go away. By Feb. 27 it was clear that this wasn’t happening, since community spread was very clear from the public data in Italy and Iran, and probably also clear from genetic data in the United States and elsewhere.
So we reached a tipping point in those next few days, at which point, the market started responding more vigorously.
If the null hypothesis is true, then those early trades were not quite as prescient as they look. We might be making the mistake of ‘resulting’, and confusing the reality we ended up in with all the others which were possible at the time, in which those traders lost their shirts. It's really hard to have a useful object-level discussion about this, because these events are one-offs (this is the same argument we get into every year on Scott's predictions threads!) It's not like we can run the experiment again, and thank goodness for that.
Nevertheless, Wei Dai suggested that this was the final nail in the coffin of EMH—at least for him.
I want to pause here to give mad props to Wei Dai for being totally open about everything, and especially for doing the following:
warning that options are dangerous, and you can easily lose the lot
generic disclaimer about seeking financial advice
updating the thread after he ended up losing 80% of his paper profits
mentioning selection bias, and saying we'd be right to discount his evidence
Which only leaves the initial claim [LW(p) · GW(p)] that "at least for me this puts a final nail in the coffin of EMH."
This is a polite way of hinting that you might be a brilliant investing wizard with the power to beat the market. Honestly, after making such a beautiful trade—and my gosh it really was beautiful—whom amongst us could resist that temptation? Certainly not me. And anyway, it might even be true!
In making sense of claims of this nature, the first thing we have to establish: what does it even mean to be able to beat the market?
Can Uncle George Beat the Market?
Uncle George really likes his new iPhone. Man, these things are nifty! The dancing poop emoji is hilarious. On the strength of this insight, George dials his broker and loads up on AAPL stock.
the chosen one! the scourge of efficient markets! the stuff of eugene fama’s nightmares!
Over the next year, AAPL stock goes up 15 per cent, while the broader S&P 500 only goes up 10 per cent. George becomes insufferable at family dinners as he holds forth on his stock-picking powers. Guess the market isn’t so ‘efficient’ after all, huh. Suck it, Eugene Fama!
So: did Uncle George beat the market?
In the narrowest possible sense… yes.
In the sense in which we aim to string words together so that they mean things: no, of course not. By this definition, every single trade leads to one of the two parties ‘beating the market’. Millions of people beat the market while I wrote this sentence. I can flip a coin between Pepsi and Coke right now, and have a 50 per cent chance of becoming a market-beating genius.
The Uncle George example makes it glaringly obvious that a successful trade does not somehow ‘break’ efficient markets. And yet, this is the same naive criticism constantly leveled against the EMH: if the market moves in literally any direction, that must mean it was wrong before! My cousin who sold/bought before it went up/down beat the market!
Same goes for the Great Coronavirus Trade. The fact that some people got out of the market early is not even the tiniest bit surprising. Investors constantly think the market is going to crash, for any number of plausible reasons. This is the default state of affairs: we have successfully predicted 73 of the last five market crashes, etc.
But even a stopped clock is right twice a day. And of course, we’re much more likely to hear about the occasional brilliant successes than the near-constant dull failures.
So the most naive criticism of the EMH boils down to ‘it's possible to make a good trade’. This is just a property of trading. It tells us exactly nothing about the market’s efficiency, or lack thereof.
But some people really do beat the market—and not in the trivial sense. They're not merely stopped clocks, or highly visible 'survivors'. I’ll suggest a definition later on which strips out the effect of randomness.
Before we get there—doesn’t the concession that people can non-trivially beat the market already drive a stake through the EMH’s heart?
This is the second great misunderstanding: there is no conflict between the EMH and beating the market. That’s how the market gets efficient! You find an information asymmetry that isn’t priced in yet, and in exploiting it, you move the market a little further towards efficiency.
Let’s call this information asymmetry an ‘edge’.
If the EMH is true—or even just true-ish—that doesn’t mean the market can’t be beat. It means:
You shouldn’t expect to beat the market without a unique edge, except by chance
Now, this usually gets simplified down to ‘you can’t beat the market’. And most of the time, this simplification is good enough: you might get lucky and win in the Uncle George sense, but over an investing lifetime, you’ll almost certainly revert to the mean (which isn’t matching the market return—it’s underperforming it).
But if you can find some kind of edge, you really can win! So, what might a genuine edge look like?
The Uncle Georges of the world don’t have an edge. All of their thoughts have already been thunk by someone else (probably by millions of someone elses). Instead, their fortunes are entirely at the mercy of the myriad other forces that drive stock prices: consumer demand, workplace harassment scandals, money printers going brrr, the exact virulence of a novel coronavirus, the price of cheese in Spain last Friday afternoon, etc.
All of this stuff—billions of inputs processed by the greatest collective intelligence ever built—is a black box unto us mere mortals. It’s impossible to assign perfect causal explanations to stock prices, which means we can pick whichever story suits us best. As a market reporter, this was pretty much my whole job: calling brokers and economists to wrap a plausible narrative around totally inexplicable events, and generate sage nodding of heads.
All Uncle George can see is that he placed his bet, and AAPL went up. It was the poop emoji for sure!
And so, Uncle George spends his days dishing out hot stock tips on online forums, oblivious to the fact that his success was meaningless.
[uncle georging intensifies]
What does a real edge look like?
A century ago, investors started noticing they could consistently pick up bargains by running very simple formulas over stock prices. The most famous is the ‘value investing’ approach developed by Ben Graham, and used by Warren Buffett and Charlie Munger. There was a genuine, big old inefficiency in the markets, and these guys had a great time exploiting it.
I think this might be the image most people have in their head when they think of ‘beating the market’—diligently studying The Intelligent Investor and learning about PE ratios or whatever.
But this is like trying to use a stone-age axe against a fighter jet. The Ben Graham information asymmetry has long since disappeared because…markets are efficient(ish)! Once the formula was widely known, it stopped working. Investors developed more sophisticated versions, more formulas, more pricing models. Once those got out, they stopped working too. Now there’s a great debate as to whether even the most complicated descendants of value might be totally dead. In which case, the anomaly has officially gone for good.
Either way, this is not how Buffett gets his edge, and it hasn’t been for decades. Here’s Munger:
The trouble with what I call the classic Ben Graham concept is that gradually the world wised up and those real obvious bargains disappeared. You could run your Geiger counter over the rubble and it wouldn’t click.
Buffett’s most brilliant achievement is weaving this folksy legend that he is a cute old grandpa who beats the market by backing the best companies. Let's take a look at how market-beating investors really make their money.
Modern Edges are Completely Ridiculous
greatest showman on earth
1. The Warren Buffett Halo Effect
In recent decades, Buffett has made a killing through juicy private deals which are completely out of reach of the average investor. Like, six billion dollar deals with three billion in preference rights and a guaranteed dividend. Like, lobbying the government to bail out the banks, then carving off a huge piece of the action. Like, being able to play around with Berkshire Hathaway’s $115 billion insurance float. Much of his fortune is built on taxpayer largesse.
Warren Buffett’s brand is so powerful that at this point, his success is a self-fulfilling prophecy: when Berkshire invests in a stock, everyone else piles in after him and drive the price up. Buffett even lends out his ‘halo’ to companies that need it—most famously during the GFC—so long as they give him a generous discount to the market price, of course. (Matt Levine has written some fascinating columns about this).
And yet, and yet… Berkshire Hathaway has underperformed for the last decade. Buffett would have been better off to take his own advice and put it all in index funds.
2. Hedge funds with armies of drones
There you are, sitting in your home office going through Walmart’s quarterly report and calculating PE ratios or whatever. Meanwhile, the professionals are using an army of drones to monitor the movement of shopping carts in Walmart parking lots in real time.
See also: sending foot soldiers out to every branch of a bakery chain at the close of business each day, because the numbered dockets start out at zero, and thus contain live sales data unavailable to the market.
And so, when renowned hedge fund manager Michael Steinhardt was asked the most important thing average investors could learn from him, here’s what he suggested:
“I’m their competition.”
And yet, and yet…almost all hedge funds underperform. Not all of them are trying to beat the market, but the tools at their disposal gives us a sense of the difficulty here.
3. High-frequency traders move mountains
If multiple people have access to the same information, the speed in which you can bring it to market also matters. So, we have high-frequency traders.
One firm spent $300 million laying a direct cable between Chicago to New Jersey. They cut straight through mountains and crossed rivers. The cable stretched 1331 kilometres. And they did this to shave four milliseconds off their transmission time.
And yet, and yet…microwaves came along and rendered the whole project obsolete. Trying to get an edge is expensive.
4. Being willing and able to commit felonies
Insider trading is a thing. See also: criminals who hack or otherwise steal sensitive private information.
And yet, and yet…even when criminals have advance access to earnings reports, they still don’t do all that well, which is evidence for the very strongest form of the EMH (the one that no-one, including me, believes can possibly be true).
So…what sort of edge do us lesser mortals have?
If we mumble something about having ‘good intuition’, or ‘subscribing to the Wall Street Journal' then we should consider the strong possibility that we are Uncle George.
If the answer involves ‘fundamental analysis’ or ‘Fibonacci retracements’, we're still in Uncle George territory. The only difference is that doing something complicated makes it easier to internally justify the belief that we know a secret no-one else does. But it is still (probably!) a mistaken belief.
The EMH Gets Stronger With Every Attack
So we know for sure that market-beating edges exist—I’ve even written them down for everyone to see!
I can only dream of possessing a halo effect so strong that everyone piles into a stock right after I announce I have graced it with my favour. I don’t have an army of drones at my command, or the ability to bore through mountains to shave milliseconds off my trading times, or a weekly round of golf with the CEO of a Fortune 500 company.
The market is never perfectly efficient. But relative to me, it might as well be.
Critics have pointed out plenty of cases in which the EMH doesn’t jive with reality—and they are absolutely right. So this is where it gets really weird.
The EMH is the only theory that grows stronger with every attack against it.
Every edge is constantly at risk of being gobbled up by an efficient-ish market. The ones I’ve mentioned can only be publicly knowledge because they're somewhat defensible: they’re based on personal relationships, capital investment, proprietary technology, etc. But they disappear too.
Most edges can’t even be spoken out loud without disappearing. If stocks systematically rise on the third Thursday of each month but only under a waxing moon, and then someone writes about it in public, you can kiss that anomaly goodbye. The EMH sucks it into its gigantic heaving maw, and it’s gone forever.
In other words: every time someone picks a hole in the theory and points out an inefficiency, they make the predictions generated by the EMH more robust! It’s like some freaky shoggoth thing that Just. Won’t. Die.
you may not like it, but this is what peak efficiency looks like
Which gets us to the totally justified criticism of the theory: the only reason the EMH can pull this stunt is because it’s bullshit science.
It’s unfalsifiable! It responds to criticism by saying, ‘OK, good point, but now that I’ve factored that in, you should believe in my theory even more.’
And…we really should?
The only way I can think about the EMH without going insane is to remind myself that it generates a useful heuristic. It’s not a stable law, like we might find in hard sciences. It’s not perfectly accurate. At any given point in time, there are always competing models that do a better job of describing reality. But all those other models can stop working at any moment, with no warning! By the time you find out their predictive power is gone, it’s too late, and you probably lost a bunch of money! By contrast, the EMH is a reliable model—reliably vague and hand-wavy, yes, but also reliably useful.
We know there are inefficiencies in the market. In the fullness of time, they will be absorbed into the gelatinous alien-god’s hivemind. But before that happens, maybe we can make some money off of them.
So now we come to the final test. How do you tell if you’ve really found a market-beating edge—that is, a model of reality that has more predictive power than the EMH—or you’re fooling yourself like Uncle George?
If You’re so Smart, Why Aren’t You Rich?
Everyone knows a secret about themselves, or the people they know well, or can arbitrage some opportunity in a niche that few people are paying attention to. These illiquid private ‘markets’ are much more fertile hunting grounds for asymmetries, and something I encourage everyone to think about.
But the public security markets are a gigantic agglomeration of everyone’s predictions, which constantly hoovers up every new fragment of information, and recalibrates itself in real time. Your challenge is to try and predict why this giant meta-prediction is wrong, and in which direction.
If you think you can reliably beat, say, an index fund that passively tracks the S&P 500, this is a much stronger claim than it first appears.
For one thing, you’re claiming to be better than Warren Buffett, who has failed to pull this off in the last 10 years, despite his huge advantages, and has started saying the game is so hard that everyone should just buy index funds. But that’s nothing. You are also claiming that you have the power to beat the greatest collective intelligence humanity has ever created.
This is an extraordinary claim, and the thing about extraordinary claims is that they require extraordinary evidence.
Uncle George’s AAPL trade ain’t going to cut it. Here is the extraordinary evidence that I would personally want to see before agreeing that an investor can beat the market:
1. Big heaps of money
This is the one area of life where there really is no dodging that most venerable of sick burns: if you’re so smart, why aren’t you rich?
So the first piece of evidence I would accept is the fact that someone is very, very rich. Sitting atop a big old pile of cash. And they’d probably also open a hedge fund so they can take other people’s money and turn it into millions more.
2. Track record of outperformance
Maybe a genuine market-beater doesn’t have enough starting capital to make big piles of money on a relatively slim edge, and for some reason is unable to come up with any scheme to beg or borrow more? Or the anomaly is real, but disappears before they can get filthy rich?
In these scenarios, I would also accept a complete record of out-of-sample investment returns over time—no backtests! no selecting the best trades!—as compared against the appropriate risk-adjusted benchmark.
These evidential standards work both in the case that the EMH is ‘dead’, i.e. you can reliably beat the market using public domain info which ought to already be priced in, and in the case that it’s not, i.e. you really do need novel information to get an edge.
As for evidence that the EMH really is dead…hmm. It’s not a proper theory to begin with. But I guess it would be ‘dead’ when the predictions it generates stop being accurate or useful? Which would look something like: ‘finding out that plenty of people meet the standards above, despite having never been in possession of a scrap of information that wasn’t already available to the market’.
In doing so, we’d have to be very careful to make sure we aren’t just looking at the Uncle Georges who unwittingly drew the winning lottery ticket. After all, we should expect plenty of investors to beat the market for a long time—even for years on end—entirely by chance.
The Great Coin-Tossing Experiment
Say we held a national coin-flipping contest. After 15 rounds, one in every ~32,800 people would have managed to call every single toss correctly, perfectly predicting a sequence like this:
H T H H T T H H H H T H T T T
Pretty impressive, huh!
Well, only in a world where we don’t know about probability. In that world, we might mistake blind randomness for skill. The lucky few winners would be hailed as the heroes of their hometowns, do interviews with breathless breakfast TV hosts, and explain that it’s all about the precise flick of the wrist. Aspiring flippers would queue up to buy the inevitable best-selling book, Flip Me Off, and pay exorbitant sums for one-on-one coaching sessions with the master tossers.
Depressingly, this is exactly what happens in the world of investing. What does it mean to achieve the kind of success which only happens by chance with 0.0003 probability? In the United States alone, it means you end up with 10,000 lucky dopes who are indistinguishable from brilliant investors.
And fund managers don’t need to do anywhere near that well to attract a market-beating aura. They’re incentivised to swing for the fences, increasing the odds they beat the market in some highly visible fashion over some shorter period—say, a lucky season or two. They inevitably regress to the mean, sometimes crashing and burning in spectacular fashion, but it doesn’t matter so long as they manage to hose naive investors in the meantime.
We can never entirely rule out the effect of randomness—there will always be some tiny chance that Warren Buffett is really just the world’s greatest coin-flipper—but we have to draw the line somewhere, or the standard is impossibly high.
Once the odds of a fluke get pretty slim—someone is super duper rich, and they’ve made a ton of consistently good trades over time—I’d happily congratulate them on their market-beating prowess, give them all my money to invest, listen eagerly to their advice, etc.
Being good Bayesians, this is obviously a spectrum: if they are not at that point, but trending in the right direction, I would be less skeptical, etc. But the bar has to be pretty high, or there's no way to separate skill from randomness.
Scott and Eliezer have both alluded to their comments being informed by private information. Here's a reddit comment in which Scott responds to criticism of his 'EMH is the real victim' riff:
I think we're in an asymmetric position, in that I know these people pretty well, I know they've thought about efficient market before, and they're the sort of people I would expect to beat the market if anyone could. I agree that if you just hear some blogger say he saw some people beat the market once, that's not much evidence.
Eliezer definitely understands the EMH, because the descriptions of it in Inadequate Equilibria are among the most brilliant and insightful I've ever come across. And Scott is obviously super smart.
I would love to know what their evidential standards are, but I'm explicitly not calling them out, or any of the people mentioned earlier in the post. No-one is under the slightest obligation to share private evidence, and I would be thrilled if those folks were indeed the market-beating chosen ones.
But I am saying that people, in general, make these kind of claims all the time—in good faith and with no malicious intent—and in general, taking their advice is an extraordinarily bad idea.
A heuristic: if you (or someone you know) is confident they can beat the market, and yet you notice you are not sitting atop an enormous pile of wealth, it's at least worth considering the possibility that you might be fooling yourselves.
The Four Types of Investors
There are very obvious and well-known reasons why everyone loves to think they can beat the market: overconfidence, confirmation bias, ‘resulting’, selective memory, survivorship bias, etc.
These forces are so powerful that many people—myself included—blithely ignore the vast piles of evidence that suggest beating the market is incredibly difficult, and go ahead and try anyway. All of us think we are special, and (almost) all of us are wrong.
Even if we don't personally harbour this particular fantasy, there's also a natural tendency to want our tribe or our friends to be the brilliant visionaries who were ahead of the action, possess sweet market-beating skills, etc.
So we can roughly place investors into one of the following four quadrants:
('Losers' and 'winners' here is tongue-in-cheek, and not a value judgment: literally, losing/winning this game by either successfully beating the market, or failing to do so)
Deluded losers (‘Suckers’)
“Apple stock really is undervalued, but the market hasn’t recognised it yet. I just got unlucky—it was because of [elaborate rationalization]. Also, even if I got it wrong this time, I was really close. Next time!
“What’s that? Do I track my portfolio returns over time, and compare against the relevant risk-adjusted benchmark to see whether I’m actually outperforming? Well, there’s no need. I usually do pretty well for myself, and I’m expecting to improve—in fact, I just picked up this classic book called The Intelligent Investor…”
Deluded winners (‘Dumb Luck’)
“I knew Apple stock was undervalued! And I remember that other time I made a really good trade, too. Guess I’m pretty good at this game!
“…What’s that? I might have just got lucky? Hah, no. I even did the Fibonacci retracements and everything."
Realistic losers (‘Clear-Eyed Fools’)
“I keep a meticulous record of my portfolio returns, which forces me to acknowledge the fact that even though I occasionally do well, I am underperforming my benchmarks on a risk-adjusted basis. I am under no delusions about my prospects of finding an edge, and I know I really ought to take Warren Buffett’s advice and put all my money in index funds.
“But I enjoy playing the markets! It's like how a night in Vegas can have negative EV but still be positive utility, because of all the non-financial factors. So I’m gonna keep gambling with a small part of my portfolio, just for shits and giggles. In the event that I ‘win’, I will try really hard to resist the incredible internal pressure to start thinking of myself as a brilliant investing guru."
Realistic winners (‘Chosen One’)
“I keep a meticulous record of my portfolio returns, which have outperformed the appropriate risk-adjusted benchmarks to such a degree that I am confident I have found a genuine informational asymmetry. I will of course never tell anyone about it, or it will become useless.
“And I can never be entirely sure: it’s also possible that I just got lucky. But at the very least, I am sitting atop great piles of money, which is pretty nice.”
The vast majority of people who actively trade their account are ‘Suckers’. Some smaller number fall into the ‘Dumb Luck’ quadrant (Uncle George would stay there if he never places another trade, but he almost certainly won’t be able to help himself.)
The right-hand quadrants are much more sparsely populated. I guess there are a few ‘Clear-Eyed Losers’ floating around, and a tiny handful of ‘Chosen Ones’.
This rough distribution is probably not too controversial. The question is, which one are you?
(I made a poll on my website at this point in the post, just for a bit of fun: the results so far are brilliant)
Trying to Beat the Market is Like Crack for Smart People
There is a tendency for smart people to wander into areas they know very little about, and think they can do better than the actual experts who have years or decades of domain-specific knowledge, on the basis of being very smart, or having read some blog posts online about being more rational.
This would be OK if it was just a bit cringe. I love armchair pontificating as much as the next guy! The consequences are usually limited to mildly annoying the people who actually know what they’re talking about, and much eye-rolling when you triumphantly reinvent the wheel.
There is some upside too: reinventing the wheel is fun, because you get to, like, invent wheels. And very occasionally, it might even be true! No doubt smart outsiders are occasionally able to breeze into a new field and exploit some obvious inefficiencies.
But…oh boy. It’s really not true of this particular domain. And it’s not harmless either.
The central prediction generated by the EMH is that you should not expect to be able to beat the market (in the non-trivial sense) unless you have unique information or some similar edge.
This prediction is tested every day. We have great piles of evidence which suggest that it is correct: the vast majority of active investors do really badly.
Crucially, it’s not only regular schmucks who underperform. So do paid professionals, and active managers, and hedge funds, and all sorts of brilliant people who have made this their life’s work.
It's possible that I am not making many friends with this post. I certainly feel pretty nervous about publishing it. Everyone who thinks they can beat the market will have their hackles up! If it helps at all, I am not claiming the high ground. I have made every dumb investing mistake you could think of, and then a few more besides. I am painfully aware of how hard this is.
These days I would put myself in the ‘Clear-Eyed Fool’ quadrant, but only by a fingernail. It’s a constant battle even to stay there. I still do clever things that contradict my own boring advice, and annoyingly, am rewarded for my hubris just often enough to start entertaining the thought that I’m a brilliant investing genius after all. Then I force myself to calculate the IRR on my publicly-traded investments, and compare it against appropriate benchmarks, and manage to get a fingernail-hold back on boring old reality.
To the extent that I have succeeded as an investor, and I am doing quite nicely thank you, it has only come through forcing myself to acknowledge the main prediction that emerges from the very-much-alive-and-kicking EMH. The huge and underappreciated benefit of doing so is that I occasionally divert some of my attention elsewhere, to domains where I actually do have an edge—and then I win.
The EMH is a weird, counterintuitive, freaky-ass shoggoth of a thing and it still confuses the heck out of me, even after almost a decade of writing about finance. I almost certainly made some mistakes in the post—in the process of writing it, I noticed several ways in which my initial beliefs were subtly wrong, which has already been super useful for me, and helped me understand some of the (valid) objections people have raised.
So I would also like to use this post to open up the floor to any and all EMH discussion, and try to benefit from the smaller-but-still-powerful collective intelligence that is Less Wrong.
I'm going to add comments with some of my own questions and uncertainty. It would be nice to become less confused together, and try to get a better sense of where we should apply our efforts at the margin.
I think this post is missing the important part of actually doing this well / being a chosen one, from my perspective. That is, it seems to think of the EMH as something like an on/off switch, where either you think the market is better than you always and so you just blindly trust it, or you think you're better than the market always, and so you should be actively trading.
But my experience has been about every five years, an opportunity comes by and I think "oh, this is a major opportunity," and each time I've been right. [FWIW I didn't have this for COVID, because of a general plan to focus on work instead of the markets leading up to the pandemic, and then when I started thinking "oh this will actually be as bad as 1918" I was too busy trying to solve practical problems related to it that I didn't think seriously about the question of "should I be trading on this?"; I think the me that had thought about trading based off it would have mostly made the right calls.]
This is, of course, a small sample size, and I've made many active trades that weren't associated with that feeling, whose records have been much worse. Each of the 'edge' investments has also had another side to it, and the other side didn't have the same feeling to guide me. For example, I correctly timed BP's bottom during the Deepwater Horizon crisis, but then when it recovered my decision of when to sell it was essentially random. I think most of the people who predicted COVID a week early were then not able to outperform the market on the other side, and various things that I've seen people say about why they expect a continued edge seemed wrong to me. (For example, someone mentioned that they could evaluate potential treatments better than the market--which I think is true, because I think this person is actually a world-class expert at that specific problem--but I think that ability won't actually give them an edge when it comes to asset prices. I don't think anyone thinking just about biology would have correctly predicted the recent bottom or where we'd recover to, for example.)
Nevertheless, I'm pretty convinced that I sometimes have an edge, and more importantly can tell when I have an edge and when I'm just guessing. I think something like 1-10% of rationalists are in this category, or could be if they believed it, much like I think a comparable number of rationalists could be superforecasters if they tried. And historically, knowing to take the "oh, this is a major opportunity" signal seriously, instead of treating it as "just another good idea", would have made a huge difference, and I think I've under-updated each time on how much to move things to be more ready the next time one comes along. Which is the main reason I think this is worth bringing up.
[Like, inspired by his weakened faith in the EMH, Eliezer attempted to time the bottom of the market, and succeeded. It seems better if more people attempt this sort of thing, at an appropriately humble frequency.]
I think I agree with this point and I'm glad you raised it—you might non-trivially beat the market every once in a while, but not all the time. To be clear, I don't think of the EMH as an on/off switch, and if the post gives that impression, that's my bad. In fact, the most successful investors I know of wait, wait, wait, and then very occasionally load up when they see a major asymmetry.
But I do still have a couple of reservations. The first is that this would still show up in the evidential standards I've suggested? If you match the market most of the time, but every five years you beat it, you're eventually gonna have a pretty amazing track record, especially as you compound early successes. Note this is more or less the same frequency offered by some anomalies—for example, the momentum strategy I experimented with only outperforms during downturns, but ends up with a super impressive CAGR.
The second question is, how confident are you ('you' in the general sense, but also 'you' Vaniver if you feel inclined to answer) that knowing the difference between the edge trades and the others actually helps? If it's a very strong feeling, why trade on all the other occasions when you know you're guessing?
I think it's interesting in an academic sense that these edge trades might really exist, but if they still ultimately result in an unexceptional record because of all the usual human foibles, I'm not sure how it practically justifies the suggestion that anyone should do more of this sort of thing?
Each of the 'edge' investments has also had another side to it, and the other side didn't have the same feeling to guide me. For example, I correctly timed BP's bottom during the Deepwater Horizon crisis, but then when it recovered my decision of when to sell it was essentially random. I think most of the people who predicted COVID a week early were then not able to outperform the market on the other side, and various things that I've seen people say about why they expect a continued edge seemed wrong to me.
This is a super important point too. Any investor who is long the stockmarket on a long horizon ultimately has to get back in, which means they have to time the market twice. Personally, I was gobsmacked by the speed of the recovery (or the height of the dead cat bounce, if that's what it is). April was the stock market's best month in 30 years, which is not really what you expect during a global pandemic.
Right, April's rally wasn't due to "actually, everything is great now", it was due to "whew, it looks like the most apocalyptic scenarios we were seeing in March aren't likely, and there's a limit to how bad it's going to get".
I agree with all of this. (And have had similar experiences.)
The opportunity feeling on the way in, and then something much closer to randomness on the way out in particular resonates, and I'm not sure I'd seen that actually written out as a general rule before.
EDIT: See also my comment here [LW(p) · GW(p)] for some discussion of how I think this works. And see my reply to Eliezer here for some heuristics regarding when an opportunity might be real.
One thing I would add is that the EMH also suggests one can make deviations that don't have very high EMH-predicted costs. Small investors do underperform indexes a lot by paying extra fees, churning with losses to spreads and capital gains taxes, spending time out of the market, and taking too much or too little over risk (and especially too much uncompensated risk from under diversification). But given the EMH they also can't actively pick equities with large expected underperformance. Otherwise, a hedge fund could make huge profits by just doing the opposite (they compete the mispricing down to a level where they earn normal profits). Reversed stupidity is not intelligence [LW · GW]. [Edited paragraph to be clear that typical retail investors do severely underperform, just mainly for reasons other than uncanny ability to find overpriced securities and buy them).]
That consideration makes it more attractive, if one is uncertain about an edge, to consider investments that the EMH would predict should be have very modest underperformance, but some unusual information would suggest would outperform a lot. I was persuaded to deviate from indexing after seeing high returns across several 'would-have-invested in' (or did invest a little in, registered predictions on, etc) cases of the sort Wei Dai discusses. So far doing so has been kind to my IRR vs benchmarks, but because I've only seen results across a handful of deviations (one was coronavirus-inspired market puts, inspired in part by Wei Dai and held until late March based on a prior plan of letting clear community transmission in the US become visible), and my understanding from colleagues in the pandemic space), the likelihood ratio is weak between the bottom two quadrants of your figure. I might fill in 'deluded lucky fool' in your poll. Yet I don't demand a very high credence in the good quadrant to outweigh the underdiversification costs of using these deviations as a stock-picking random number generator. That said, the bar for even that much credence in a purported edge is still very demanding.
I'd also flag that going all-in on EMH and modern financial theory still leads to fairly unusual investing behavior for a retail investor, moreso than I had thought before delving into it. E.g. taking human capital into account in portfolio design, or really understanding the utility functions and beliefs required to justify standard asset allocation advice (vs something like maximizing expected growth rate/log utility of income/Kelly criterion, without a 0 leverage constraint), or just figuring out all the tax optimization (and investment choice interactions with tax law), like the Mega Backdoor Roth, donating appreciated stock, tax loss harvesting, or personal defined benefit pension plans. So there's a lot more to doing EMH investing right than just buying a Vanguard target date fund, and I would want to encourage people to do that work regardless.
I’d also flag that going all-in on EMH and modern financial theory still leads to fairly unusual investing behavior for a retail investor, moreso than I had thought before delving into it.
Seconding this. It turns out that investing under the current academic version of EMH (with time-varying risk premia and multifactor models) is a lot more complicated than putting one's money into an index fund. I'm still learning, but one thing even Carl didn't mention is that modern EMH is compatible with (even demands) certain forms of market timing, if your financial situation (mainly risk exposure through your non-investment income) differs from the average investor. This paper gives advice based on academic research but was apparently written in 1999 so may already be outdated.
One thing I would add is that the EMH also suggests it's often not super high cost to deviate: small investors can lose out by paying extra fees and taxes, spending time out of the market, and taking too much or too little over risk (and especially too much uncompensated risk from under diversification), but given the EMH they also can't actively pick equities with expected underperformance.
It doesn't suggest that. Factually, we know that a majority of investors underperform indexes.
When there's an event that will cause retail investors to predictively make bad investments some hedge fund will do high frequency trades as soon the event becomes known to be able to trade the opposite site of the trade.
All events that cause more retail investors to buy a stock then it cause retail investors to sell the stock needs some hedge fund or bank to take the opposite side of the trade and likely that hedge fund or bank is in the trade because it has models that suggest it's a good trade for them.
A hedge fund that provides liquity to trades is going to make as much money under EMH as it cost to do the market making when it competes with other hedge funds.
It worth noting that a targeted date index fund does make predictable trades where someone needs to do the market making and will likely make a small profit for doing the market making.
It doesn't suggest that. Factually, we know that a majority of investors underperform indexes.
Absolutely, I mean that when you break out the causes of the underperformance, you can see how much is from spending time out of the market, from paying high fees, from excessive trading to pay spreads and capital gains taxes repeatedly, from retail investors not starting with all their future earnings invested (e.g. often a huge factor in the Dalbar studies commonly cited to sell high fee mutual funds to retail investors), and how much from unwittingly identifying overpriced securities and buying them. And the last chunk is small relative to the rest.
When there's an event that will cause retail investors to predictively make bad investments some hedge fund will do high frequency trades as soon the event becomes known to be able to trade the opposite site of the trade.
I agree, active investors correcting retail investors can earn normal profits on the EMH, and certainly market makers get spreads. But competition is strong, and spreads have been shrinking, so that's much less damaging than identifying seriously overpriced stocks and buying them.
Active investors need to spend money to hire analysts, build computer models and high-frequency trading computers.
Let's say it costs $10 dollar/per trade to do the analysis to be able to do a trade with a retail investor that nets the hedge fund $10.10. Even when there's no strong competition with other hedge funds over that $0.10 of profit, the retail investor is still screwed by a significant $10.10.
Sure, it's part of how they earn money, but competition between them limits what's left, since they're bidding against each other to take the other side from the retail investor, who buys from or sells to the hedge fund offering the best deal at the time (made somewhat worse by deadweight losses from investing in speed).
I found out a couple of days ago that there's a debate within academia over EMH. Here's how John Cochrane described it in an interview:
When house prices are high relative to rents, when stock prices are high relative to earnings—that seems to signal a period of low returns. When prices are high relative to earnings, it’s not going to be a great time to invest over the next seven to ten years. That’s a fact. It took us ten years to figure it out, but that’s what (Robert) Shiller’s volatility stuff was about; it is what Gene (Fama)’s regressions in the nineteen-eighties were about. That was a stunning new fact. Before, we would have guessed that prices high relative to earnings means we are going to see great growth in earnings. It turned out to be the opposite. We all agree on the fact. If prices are high relative to earnings that means this is going to be a bad ten years for stocks. It doesn’t reliably predict a crash, just a period of low returns, which sometimes includes a crash, but sometimes not.
Ok, this is the one and only fact in this debate. So what do we say about that? Well, one side says that people were irrationally optimistic. The other side says, wait a minute, the times when prices are high are good economic times, and the times when prices are low are times when the average investor is worried about his job and his business. Look at last December (2008). Lots of people saw this was the biggest buying opportunity of all time, but said, “Sorry, I’m about to lose my job, I’m about to lose my business, I can’t afford to take more risk right now.” So we would say, “Aha, the risk premium is higher!”
So that’s now where this debate is. We’re chewing out: Is it a risk premium that varies over time, or is it psychological variation? So your question is right, but it is not as obvious as: “Stocks crashed. We must all be irrational.”
There is another important argument widely used for efficient markets, the argument that a model like (4) with an intermediate φ cannot represent a stable equilibrium because the smart money would get richer and richer and eventually take over
the market, and φ would go to zero. In fact this will not generally happen, for there is
a natural recycling of investor abilities, the smart money people usually do not start
out with a lot of money, and it takes them many years to acquire enough wealth to
influence the market. Meanwhile they get old and retire, or they rationally lose interest in doing the work to pursue their advantage after they have acquired sufficient
wealth to live on. The market will be efficient enough that advantages to beating
the market are sufficiently small and uncertain and slow to repay one’s efforts that
most smart people will devote their time to more personally meaningful things, like
managing a company, getting a PhD in finance, or some other more enjoyable activity, leaving the market substantially to ordinary investors. Genuinely smart money
investors cannot in their normal life cycle amass enough success experience to prove
to ordinary investors that they can manage their money effectively: it takes too many
years and there is too much fundamental uncertainty for them to be able to do that
assuredly, and by the time they prove themselves they may have lost the will or ability to continue (Shiller 1984; Shleifer and Vishny 1997).
As you pointed out in the post, the version of the EMH you are describing, is unfalsifiable. To me it looks like an obvious misstep by our own rules, so why does it hold so much sway?
It's exactly the kind of thing we were warned about in the Mysterious Answers sequence: A Semantic Stopsign [? · GW] that stops you from asking the obvious next question. A Curiosity Stopper [? · GW]: because there are Smart People who know better, we are not socially obligated to even look.
Its vagueness makes this EMH feel like a motte-and-bailey argument. Can you tell which part is the bailey and which is the motte? Think about this for at least ten seconds before reading on.
Here's a guess: "You can't hope to beat the market, so I don't have to try, and neither should you!" (bailey). But look at all these anomalies! "The market will price those in now that they are known. They'll evaporate. (Eventually.)" (motte).
But the vagueness means I can only guess. That's the nature of such arguments. Maybe the bailey is something else or there's more to it. What is the EMH defending? We may have to taboo "EMH" to nail that down. I suspect I'm insulting a Sacred Cow. Which one? Maybe "prediction markets are a cool! It's a general solution to a lot of problems!" (If the stock market isn't efficient, prediction markets aren't either.) Or rationalists say "Bets are a tax on bullshit," so why ain'tcha rich? (Sour grapes. You only make money if you're lucky. Rationalists don't buy lottery tickets. Is this an Ugh Field? [LW · GW]) Or "Capitalism!" (Applause light! vs. I don't know, something else?). Why are you attached to an unfalsifiable idea?
Is there a falsifiable version of the EMH? OK, forget Popper, we can do one better: To do a Bayesian update we need both sides of the likelihood ratio. What does the world look like if the EMH is true? If it's false? Which world does ours look like? That's your prior. If these cases are indistinguishable, then it's not a Hypothesis, and the "H" in "the EMH" is a lie. A hypothesis that forbids nothing permits everything. Now what evidence would change your mind?
So let's try some obvious next questions. If the EMH were true, the market's moves would be a random walk, with a small bias over time based on growth and interest rates, etc. that explain the buy and hold strategy. Subtract out the bias and the residual should be completely random. Can we distinguish random from nonrandom data? Can you measure its entropy? Is the market random? You know how to use a computer. Look and see!
Can you make a profit from this edge? Why aren't you?
I think that when when I look at economics (another great example) you have a series of papers by Larry Summers and Brad DeLong in the early 90s that as far as I can tell drive a stake through the heart of the idea of efficient markets. They show that if you make the extremely minimal assumption of people not being perfectly capable of assessing how much risk is involved in an investment, there should be a systematic tendency for markets to become less efficient with time.
And as far as I can tell this behavior--this paper despite being done by pretty much the top people in economics--just got ignored and had no impact on the--or this series of papers--had no impact on the progression of the field. It was logically ironclad. That's the sort of thing I basically expect from most sciences in the modern world--almost everything but applied physics--and it's (you know) this is a particularly clear case though because you have essentially the strongest possible argument done by the most prestigious possible people with just no recollection of it ever having happened in the profession.
Does anybody know what papers he's talking about? (I'm not sure if I transcribed the names properly.) They seem very relevant to this discussion.
" The unpredictability of noise traders' beliefs creates a risk in the price of the asset that deters rational arbitrageurs from aggressively betting against them. As a result, prices can diverge significantly from fundamental values even in the absence of fundamental risk. Moreover, bearing a disproportionate amount of risk that they themselves create enables noise traders to earn a higher expected return than rational investors do."
(This paper has been quoted 6831 times according to Google Scholar).
I upvoted for relevance (although it's not quite what I asked for). Interesting read.
The efficient-market hypothesis is coherent, falsifiable and wrong [LW · GW], even in its weakest form: The paper describes a way to introduce an inefficiency, that were it arbitraged optimally, would solve an (NP-complete) satisfiability problem. Even if P = NP (which seems unlikely), it remains unproven, so nobody yet knows how to solve NP-complete problems in PTIME, which would be required for markets to be truly efficient.
The eldritch abomination may grow stronger with every attack, but it is decaying even faster than we can fight it. The number of possibilities to search for anomalies grows faster than our ability to compute them when you account for the possibility of anomalies in the pricing of sets of securities, like for pairs trading. There might be something like 100,000 publicly-traded stocks in the world, but that would mean (1000002) combinations or nearly five billion pairs. And that's not even considering larger subsets, like triples, which would be on the order of hundreds of trillions.
So the EMH can't be literally true. but can it be approximately true? It must be to some degree for the question to be this contentious. The real question I'm interested in, is "Are the markets exploitable?" Is there such a thing as alpha, or its it just luck? If finding exploits is as expensive as exploiting them is profitable, then the EMH might as well be true for that purpose. But if there are plenty of anomalies to go around, then I could have a realistic chance of finding and profiting from one nobody has noticed before.
Confusion: Is there some (incredibly weak) sense in which every person who makes a trade is bringing unique information to market?
Uncle George's opinion of the new iPhone is reflected in sales figures and aggregate reviews. But technically, he is in possession of a unique piece of private information: 'what does this one guy called Uncle George think about AAPL stock?'
This information is vanishingly close to being worthless, and a million miles way from a tradable edge, but it's not quite worthless—it still collectively helps to generate a signal. What does this mean, if anything?
Is there some (incredibly weak) sense in which every person who makes a trade is bringing unique information to market?
Seems like the answer to this question is straightforwardly yes. People's desires to hold various assets will be super correlated with each other. But there's non-zero information in each person's preferences.
I'm pretty sure there is an entire literature (but cannot know think of the term) regarding the problem if everyone were to be index investors. Basically, if no one is investing in the underlying assets, in the extreme, we don't get pricing for those assets and ultimately cannot even price the indexes well. In short, the market implodes on itself without all those individual market interactions.
Yeah, active investors are providing a valuable service to everyone else, both by exploiting genuine asymmetries, and by collectively generating a signal in the Uncle George sense. People sometimes worry about the passive revolution for this reason, but the vast majority of funds under management are still active, and human nature being what it is, there will presumably always be plenty of people willing to have a crack.
the vast majority of funds under management are still active
Is this true? A quick search suggests not:
With actively managed U.S. stock funds posting outflows in 11 out of the last 12 years, and an unbroken streak of inflows for passively managed U.S. stock funds, assets in index-tracking strategies have caught up with those in active funds.
Our latest U.S. fund flows report shows that at the end of April , both passive and active U.S. equity funds had a total of about $4.3 trillion in assets, essentially reaching asset parity.
Huh, I got that from a recent Bloomberg article which says 15:1...not sure who's right or why the numbers are so different.
Active management in the equity market, both in the U.S. and abroad, is dominant. And not by a little: Active management in the U.S. trounces passive by a ratio of 8-to-1 in dollar investments. Expand that to include the entire world, and the ratio is closer to 15-to-1. If we include fixed income in our calculations, the ratio balloons to 60-to-1.
I think the reason not to worry too much about the passive revolution is that as long as there are money-making opportunities from active trading -- people will be incentivized to do it. The end state "everyone is passive -- there is no price signal" is not one we can reach from where we are, and it is not a stable equilibrium. If we ever did cross over into "too few hedge funds", there would such a strong incentive for more, that I don't think it would last long. But perhaps I misunderstand this critique.
You criticize my description in "A Failure, But Not Of Prediction", which was:
The stock market is a giant coordinated attempt to predict the economy, and it reached an all-time high on February 12, suggesting that analysts expected the economy to do great over the following few months. On February 20th it fell in a way that suggested a mild inconvenience to the economy, but it didn’t really start plummeting until mid-March – the same time the media finally got a clue.
As my post said, the market started declining a little in February. Using the S&P link you provide, on March 2/28, it reached 2954, just 12% lower than its all-time high, then quickly recovered to only 8% lower a few days later. For comparison, the market fell 7% in May 2019 because Donald Trump made a bad tweet, and then everyone laughed it off and forgot about it within a few weeks. I think my claim that "it fell in a way that suggested a mild inconvenience to the economy" is a fair description of this.
It had its next major fall on March 9, reaching a new low (34% off its all-time high) March 23. I think it is fair to say it started plummeting in mid-March, though I would not blame you if you consider March 9 more "early" than "mid". For comparison, Jacob wrote his post warning that the coronavirus would be a big deal in late February, and I wrote one saying the same on March 2.
Some of this depends on the "correct" amount of market crash. I was writing my post in early April, when the market was near its floor. If that was the "correct" amount of market crash, then the early February crash underpredicted it, and the market didn't "get it right" until mid-March. As you write this post now, the market has recovered, and if it's at the "correct" price now, then the early February crash was basically correctly calibrated and the mid-March crash was an overreaction.
To be clear, I think time has proven you correct about the EMH (and this is easy for you to say, now that the market has stabilized). I'm not debating any of the points in your post, just your accusation that I am a "revisionist historian".
If we take the 6th of March—the last trading day before the March 9 fall—then the market is down 12.2%, which is already in 'correction' territory, and an extremely rapid descent by historical standards.
If we take the 16th of March—the closest trading day we have to 'mid-March'—the market is already down 29.5% per cent, which is not too far off the bottom, and well and truly into 'bear' territory.
To be clear, I think time has proven you correct about the EMH (and this is easy for you to say, now that the market has stabilized).
I thought the talk about EMH being dead was weird at the time, and left a comment saying as much. I also wrote a post on March 24, which later turned out to be the bottom, saying that timing the market was a really bad idea, and the buy-and-hold forever strategy was about the best anyone could hope for. I am as surprised about the speed of the rebound as anyone! I possess no predictive powers, but I have consistently been defending boring orthodoxy.
I admire and respect you very much as a person and a thinker—seriously man, you have no idea—so I feel extremely bad if I have made you feel bad. I didn't mean to accuse you specifically of being a revisionist historian—it was more of a general vibe that seemed to be happening a lot—and although I think the passage as stated is misleading, I don't think it's deliberately so, and I've edited the post so it comes off as less accusatory.
I enjoyed both the perspective and the tone of the post. Up voted.
Two, probably not to deep thoughts.
First, years ago I had a discussion with some day traders and one was talking about the SPY EFT He mentioned that the fund owners actually did not invest in all 500 equities but only in the top performing ones (no idea what the criteria was on that). The argument was largely they could not make money running the fund if the invested in both the winning and losing stocks.
Second, and it relates to the first, one of the other things (different time) that was pointed out was "market" is a problematic terms. Each asset that is traded has a market and that is not the same as "the market". I think this tends to be something of a problem area for people when the issue of EMH is in question. Many (most) do okay in thinking about a market in APPL (supply and demand, buyers/sellers, bids and offers) we all get that. But "the market" is a much more complicated (even chaotic) animal. At a high level most also get general portfolio theory but probably could not put a customized plan in place for their personal portfolio. What I suspect happens here, with regard to EMH, is a bit like the fallacy of composition error you get in logic. It's not clear to me exactly how easy it really is to sum up the many demand and supply functions for each individual asset and then suggest any aggregate price or return to "the market".
EMH (or at least most of the discussion here and everywhere else I've seen) doesn't seem to make any distinction between the asset's market and the general market (much less the global market).
Second, and it relates to the first, one of the other things (different time) that was pointed out was "market" is a problematic terms. Each asset that is traded has a market and that is not the same as "the market". I think this tends to be something of a problem area for people when the issue of EMH is in question.
Yes! This is another really great point. I think Noah Smith described it by saying something like, 'there is no EMH—there are an infinite number of EMHs all happening at the same time'. Which is another reason the theory is vague and unfalsifiable.
I find it helpful to think about each market in the narrowest possible terms. For example, the market for AAPL stock is likely to be less efficient than the S&P 500 market as a whole, although presumably not by much. The market for a thinly-traded stock languishing on the secondary board of the Tanzanian stock exchange is likely to be much less efficient than that. The market for a private company with no disclosure obligations is much less efficient again. By the time you get down to 'the market for the time and labour of this one guy called Richard', there are truckloads of inefficiencies and EMH doesn't really apply.
Three anecdotes about having an edge but only in illiquid markets where the derivable value is limited:
1. Back in 1991 a friend of mine had the following market-beating insight. He was watching Freddie Mercury on TV a few months before he died; at this point it was not publicly known he was ill with AIDS (nor even, bizarrely, that he was gay - I think all was only revealed just before he died).
My friend said "He's not looking well..." and then made a brilliant inference: "...so Bohemian Rhapsody will be the Christmas No. 1 record this year, then". (In the UK anyway, a big deal is/was made about which single would chart at no. 1 on Christmas Day, and you could even place bets on it.) And so it came to pass - Freddie Mercury unexpectedly died a few weeks before Christmas, Bohemian Rhapsody (dating from 1975) unexpectedly went to no. 1, and my friend kicked himself forever more about not placing a bet on it - bookmakers would have offered odds of 100-1 or even 1000-1.
The catch though, from a couple of times I've tried placing big bets on unlikely events, is that (most) bookmakers don't seem to accept them. They might accept a $100 bet but not a $1000 one on such odds. They suspect you have inside information. (The same happens I've heard if you repeatedly win at roulette in some casinos. Goons appear and instruct you firmly that you may only bet on the low-stakes tables from now on.)
Freddie Mercury's unhealthy appearance was public information. If the market in bets on Christmas no. 1 records were liquid enough, thousands of others would be considering the third-order consequences of such information & making similar inferences, the odds would be rapidly changing, and my friend would have lost his edge by the time he placed his bet.
So while he could have beaten the illiquid market, he wouldn't have made much in expectation - maybe not enough to justify the time & risk - and conversely he couldn't have beaten a liquid market which would accept big enough bets.
2. Similarly, another friend of mine worked for a while with a tech startup that was arbitraging between betting exchanges (like Betfair) on which ordinary people can lay odds as well as place bets. As a result of which, different exchanges often show somewhat different odds on the same event. After much development and testing, the startup realised that though they could reliably identify & arbitrage these odds differences, they could only make dollars per hour doing so, not thousands. There wasn't enough liquidity on the betting exchanges. The wronger the odds someone was laying, the smaller the bet they'd accept. So the startup closed down.
3. I've been trading equities since 2006 on various index derivatives, in very large amounts. I have expertise on one niche area, involving a lot of data & programming, and beat the market for some years with it, until the effect in question went away (as it was already known about, albeit quite obscure).
Recently I've found a derivative of a particular index that seems to have excess risk-adjusted returns in certain bets on it, so have been trading it for a while. Again I've had to source and estimate and crunch large amounts of hard-to-obtain historical data, writing my own software to manipulate it. But the inefficiency is probably only because this derivative is quite illiquid. Some days I'm doing a significant fraction (even a quarter or a third!) of the whole day's trading on it! (Though I am betting very large amounts.)
So I reckon I probably am beating the market, but only because I've found a sweet spot where there's extra money to be made, but not enough for more than a few others to do so too. I'm grabbing a large part of the edge to be had, by using kinda public but hard-to-get information. If the derivative were more liquid, others would be finding similar things to me. If it were less liquid (like Christmas no. 1 bets) I couldn't place big enough trades to be worthwhile.
(There is also extra risk from the limited liquidity (e.g. of slippage when stopped out) which might reduce returns a lot in an extreme situation - though I suspect not enough to remove my overall edge. The risk needs very careful handling.)
The catch though, from a couple of times I've tried placing big bets on unlikely events, is that (most) bookmakers don't seem to accept them. They might accept a $100 bet but not a $1000 one on such odds. They suspect you have inside information. (The same happens I've heard if you repeatedly win at roulette in some casinos. Goons appear and instruct you firmly that you may only bet on the low-stakes tables from now on.)
Right, the EMH doesn't fully apply when sharks can't swoop in with bets large enough to overwhelm the confederacy of Georges. The odds bookies offer are a hybrid between a market and a democracy.
Thanks for sharing your experience—the third section in particular is really interesting. Relative to the central discussion: how much time and effort did you have to put in to find those edges, and do you think it was worth it? Would you encourage others to try and do something similar? Or is it more like a hobby?
It's probably taken about a year's full-time work (since 2006). Though I could have spent much less time by using a simpler albeit less effective version of what I do (as indeed I did initially) - just that I like improving it and investigating new things that might work. It is kind of a hobby, but involving a sufficiently serious amount of time and money and risk to be rather more than that.
I think it probably has been worth it, as I reckon the value of my edge is rather bigger than my time cost in salary equivalent, though it is hard to be sure because of the large variability in returns (not least because of two big crashes since I started).
But actually I wouldn't encourage others to try. There can't be many opportunities to beat the market, not many people can risk big enough bets to justify the time taken finding them, and there's a lot to be said for adopting a simple mechanical system that takes little time; maybe just buy & hold (though maybe a bit fancier than that). Not sure it's worth trying to pick stocks (I almost never do that) - just buy an index; though there is some fun to be had doing so, like picking horses, and kidding yourself you have special insight!
mentioning selection bias, and saying we'd be right to discount his evidence
I would discount it more if I hadn't if a random person had made those investments then Wei Dai. Wei Dai made previous very good investments in crypto and domain names with similar returns.
When thinking myself about investing in the middle of February I was thinking about advice on Wei Dai that's years old on LessWrong before actually talking with him about investing in this situation.
I think any of the high return investments of Wei Dai of crypto, domain names and Corona puts are investments ideas that are in a different class then Uncle Georges and it's reasonable that there will be further similar opportunities for rationalists in the future where it's worth trading on them.
Eliezer wrote after the crypto aftermath something that suggested that he endorsed his past strategy that resulted in him not profiting of crypto based on EMH evidence. It's that strategy endorsement that seems to be changed.
I don't think in any of the three cases in which Wei Dai made very high returns he would have had a problem with telling other people about his investment and it would make sense to behave as you suggest a Realistic winners would (I will of course never tell anyone about it, or it will become useless).
Crypto and domain names might be in a different class, for sure—I don't know how those markets work, although I'd be interested to hear more. But options contracts that involve e.g. the fortunes of the S&P 500 are not.
(I want to encourage rationalists to look for future opportunities to trade on, too. I just think those opportunities are vanishingly unlikely to come from highly liquid securities markets, and a lot of the recent talk is going to lead to misplaced effort.)
I don't think in any of the three cases in which Wai Dai made very high returns he would have had a problem with telling other people about his investment and it would make sense to behave as you suggest a Realistic winners would (I will of course never tell anyone about it, or it will become useless).
Yeah, I'm still not exactly sure about whether anomalies can persist in public (see one of my confusions below). But if the trade is based on a one-off event, like this one, it's obviously fine to tell people about it. If it's a formula for an ongoing edge that persists in [stock-picking/domain name-picking/crypto] then not so much.
I think that when it comes to events that have such high societal impact as COVID-19 and that don't really fall in anyone's expertise that happen in the future it's likely that rationalists can lucratively trade on them and pre-run the market. These kinds of event might be once per decade, but when they come along it's worth trading on them.
As far as domain name picking goes, Wei Dai brought WeiDai.com at a time where US domains did cost money but there Chinese domain names weren't worth much and sold it years later. The edge isn't in trading either of that in the current market.
I'm still not exactly sure about whether anomalies can persist in public
In hindsight mispricing of Chinese domain and Crypto exist for years while being public. I think there was maybe 1-2 weeks of mispriced pandemic information in the case of COVID-19 when I take my personal information at the time into account.
In neither of those cases it would have changed much to be public about one's investment about how long the anomalies persisted.
I lost the thread when I got to the section "If You're so Smart, Why Aren't You Rich?". An assumption seems to become implicit there that if the EMH is false, then you can beat the market. But why is that so? I tend to think of the market as a random walk during day trading, largely driven by viral memetic fashion trends on the scale of weeks to months, and then often only "weighing" effectively on the scale of months to years.
Take the day trading example: the S&P had almost no cumulative returns during the day from 1993-2017, while after-hours & pre-market were ~600%. If market moves during the day are a random walk, then clearly they are not efficient *and* one cannot beat the market consistently, over that short time window.
Over the last couple of years of trading options, my least favorite source of randomness has been Trump tweets. I think it would be very hard to argue that the market prices them efficiently or that it is possible to have an edge in predicting them but they sure do move prices around and destroy your trading positions.
I tend to think of the market as a random walk during day trading, largely driven by viral memetic fashion trends on the scale of weeks to months, and then often only "weighing" effectively on the scale of months to years.
Is this meant to be in contrast to what a believer in the EMH would think? It sounds pretty similar to me.
Is your point just that the information that the market is pricing in on short timescales more about the demand to hold various assets than new fundamental information about a company? I suppose if you think of the EMH as saying that the market only moves on fundamentals then that would be a contrast. But I guess I tend to think of the EMH as saying that all the fundamentals are priced in. Not that nothing else is priced in.
Confusion: Can an anomaly be an open secret, and not defensible, and still somehow persist?
For example, I'm especially interested in the momentum anomaly. The momentum folks make a very compelling case that the behavioural econ reasons that brought it into existence are so dang powerful that it somehow persists, even though there are many papers/articles/books about it that anyone can read.
I personally found this just compelling enough to run a momentum experiment, based on a strategy that has done extremely well in reducing volatility and drawdowns in the past. Momentum strategies naturally lag bull markets, but more than earn their keep during the downturns, so this was the long-awaited test.
...and it performed almost comically badly. Either the anomaly evaporated, or the specific version I was testing evaporated, or it did badly this one time but still outperforms over multiple cycles, or it was only ever an artifact of overenthusiastic back-testing.
If open secrets can persist in public... for how long? What are the factors that lead to them ultimately not working? Is it possible that an anomaly could exist in public indefinitely?
Could you expand on which momentum anomaly you tested? One type is cross-sectional momentum (buy the top 1/10th of stocks that went up and short the bottom 1/10th), which is subject to major periods of drastic underperformance. This is all well documented in the literature. The other type is using momentum on the market as a whole, perhaps switching between asset classes based on momentum. I would not think that you could assess a strategy based on 6 months of performance.
My view as an investor since the 1980s is that the EMH is true to a 0th approximation. However massive agency issues in the fund management industry leave room to outperform on a risk adjusted basis if your incentives are different from the average fund manager. Some anomalies are just not exploitable by fund managers/agents because they would lose all their funds under management after periods > 12 months of underperformance.
Sure—I was testing a dual momentum strategy over the market as a whole, with a 12-month lookback period. The 'dual' refers to both absolute momentum, and relative momentum between asset classes (bonds, US stocks, non-US stocks).
I haven't evaluated it properly yet, but the signals it generated told me to stay in stocks until the end of March, at which point I ought move into bonds, just in time to miss the recovery. Over the period I've tested it so far (16 months) it has returned -4%, while my benchmark is at +18%. I am still mildly interested to see how it pans out, but I ignored the signals and am now only tracking it on paper.
Sports bookies deal with "investors" that have net biases - lots of people will, for example, bet on the home team because they're fans. I also have it on pretty good authority that people drastically underestimate the variance in soccer matches between closely matched teams; when the World Cup comes around, a simple strategy of "bet on the underdog in every match" makes money.
I'm not exactly sure what you mean by "defensible". It's true that some anomalies evaporate almost immediately once they are noticed, but others persist long after that--even for decades.
One notable example of a long-lived inefficiency was the capping of the Euro/Franc rate at 1.20 from 2011 to 2015. This was announced publicly by the Swiss National Bank so it was available information. A price cap is a glaring market inefficiency, exploitable with a simple grid strategy. If the EMH were true, the rate should have settled at 1.20 almost instantly, but in fact, it took about three years for this to happen. And Forex is about as liquid as markets get.
Here's a thought experiment: Suppose that a market is perfectly efficient, except that every 50 years or so there's a crash, which sufficiently smart people can predict a month in advance. Would you say that this market is efficient? Technically it isn't, because smart people have a systematic advantage over the market. But practically, no trader systematically beats the market, because no trader lives long enough!
I suppose you can create a long-living institution, a "black swan fund", that very rarely makes bets on predictable crashes, and over a few centuries can prove it has higher returns. But I guess not enough people care about returns over these timescales.
FWIW, I very much wanted to adjust my investments based on the covid news, but was stopped from doing so mainly via friction:
1. I made recent profits from my stocks, so selling them (even for a short window) would have had bad capital gains repercussions for me... in the US it's a PITA to deleverage stocks that are in the green.
2. I'm not enough of a trading wonk to feel comfortable buying puts, and the little in the past that I've played around with those tools seemed to indicate that it's hard for a non-professional to avoid excessive fees.
3. I just randomly happened to not be aware of the "zoom" phenomenon, so I didn't invest in that. I DID invest in 3M (because of their mask business) but they actually have lost a ton of money during the crisis for some reason (I assume the "price gouging" laws that prevent them from making any profits are a big part of that)
So, to me this is a story mainly of market inefficiencies preventing EMH from coming into play.
HOWEVER, there was also one other reason why I didn't profit off the crisis, which is actually compatible with EMH: I was expecting a large cash infusion from the fed, so I was expecting stock prices to remain stable, at the cost of subsector price inflation. Turns out, I partially got this wrong (because I didn't consider that inflation takes time to develop, so stock prices plummeted despite fed efforts) but was partially right (stocks have mostly recovered, precisely because of the Fed intervention)
TLDR: I was mostly a rational actor that understood the covid threat, but misunderstood the effect of Fed intervention... but you needed to have BOTH in this case, because of the extra overhead of non-EMH market inefficiencies.
Curated, this is a great and really grounding contribution to the ongoing conversation around this, it explains lots of key arguments really well. I hope to see more discussion about whether the EMH is dead and what can be learned from the covid situation, and I liked much of the comments section as well.
Something that occured to me while reading this post:
In a given situation, some people who are not at the bleeding edge of finance in general will be well placed to beat the market in a way we would tend to think of as due to skill rather than luck.
In the coinflip example, the people who correctly predict the sequence do so through luck. We call the sequence random because we are unable to model the process which generates it, and we call it luck when they are able to predict it because they don't know how to model it either. In the case of detecting a nascent pandemic however, we do know some things about what makes you better able to see it coming.
If you are attuned to global risks, plugged into discussion networks that provide early warning of interesting developments in the area, have got the probablistic thinking thing down, are the sort of person to hoover up enough details to distinguish between a Spanish Flu-like scenario and a SARS-like scenario, and are enough of a disagreeable contrarian to not worry what anybody else thinks until you have vetted their sanity thoroughly, then you are a member of a subpopulation that is especially well placed to spot the shape of things to come in the particular case of covid-19. Add in previously aquired knowledge about how to make the relevant trades, plus a lack of laziness, plus getting struck by a little inspiration that doesen't come to everybody, and you are Wei Dai or Jacob instead of me.
How do you end up in that subpopluation? You do so randomly, if we think of it as drawing that particular ticket in the birth lottery. So in one sense you randomly got lucky, but in another sense you succeded through skill.
I don't know how much of the apparent discrepancy between the EMH and the fact that some people timed the downturn is explained by this, but I think it explains at least part of it.
In general if someone used a weird voodoo ritual it would be natural to liken that to a coinflip. If the voodoo ritual starts to be consistently profitable then it can be likened to a deliberate technique. It might not be that markets are especiallly skillfull or wise but they get to define what counts as success so automatically win in that dimension.
One reason why you might not regard this information as available, is because it's costly. In order to profit from it, you need to spend time and effort in order to receive and understand the information. I don't think that is super plausible, given large banks and corporations with substantial research budgets.
Maybe these researchers were just lucky, there were certainly other researchers at this time who were far less concerned. But given that the information existed with at least some authority and was quite available, I would have expected the markets to at least somewhat react before mid-january.
Markets did react just about as soon as COVID started making the American news. If you look at VIX, which is an index of S&P500 futures volatility (aka the "fear index"), it jumped to a new, higher plateau right as COVID-19 started making the news in mid-January.
Hm, all I can find are these small bumps in the end of January. [But I can't figure out how to attach screenshots here.] I also can't really see a plateau effect afterwards. An actual reaction, from a cursory view, only seems to happen on the 20. February. I'm not capable of saying whether these bumps show a market reaction or if it's largely noise. Looking at the time before, it doesn't seem like an unusual behaviour. [But I'm really not good at properly reading such charts, so I'd be interested in how you came to your conclusions.]
Hmm, maybe you are right. I had a discussion about this in regards to another one of my posts about COVID-19 and the stock market, but now that I'm looking again I don't really see anything like what I was describing.
The steelman that VIX responded to COVID-19 in January is that it went up 40% from Jan 23-27. For whatever reason, this chart makes it easier to see the bump I'm talking about. It's not dramatic compared with the explosion in mid-Feb, but that accords with the idea that the market was unsure about whether COVID-19 was going to be a replay of SARS or something much worse, and didn't decide until the hard evidence came in of international community spread on Feb. 21.
Confusion: Can multiple traders profit by bringing the exact same information to market?
I want to say yes? At least if they don't individually have enough heft to fully price it in? That would explain how multiple traders are able to exploit the same anomaly, whereas hedge funds have to cap their fund size because getting too big hurts them.
There's a certain amount of money that can be made of an inefficiency in the market depending on the liquidity of the underlying market. There are a limited number of people who want to take the opposite site of the trade you are making.
Usually, you get dimishing returns as you bid people lower till the price is at it's efficient level.
Thanks, I loved this post. Its obviously hard to distinguish luck from skill when it comes to investment returns, I don't think focusing on money and track record would be enough to convince me of skill. I think I would need to also rationally evaluate the investment strategy to determine if it can be reasonably expected to beat the market. Common strategies like "we do what everyone else does but better", or "we invest in value/growth" would probably not convince me (risk premiums, liquidity premiums, animal spirits probably affect market prices but probably not in a way that can only be reliably predicted by a "chosen one"). Strategies like "we have a data connection that's 1 millisecond faster" or "we hide microphones in every lawyer's office" seem like they have a chance to beat the market, but they're hard/expensive to pull off. With respect to covid, it wouldn't be enough to say "I knew stock prices would go down" without explaining how you knew, what your confidence was, how you determined your confidence and your prediction accuracy/confidence calibration (the lack of confidence estimates on predictions is a huge red flag for me). My sense is that when someone says "my intuition tells me stock prices will go down due to covid", what they mean is something like "the market price currently implies the risk of a global pandemic is 20%, but my intuition tells me it's closer to 50%, therefore stock prices should go down half the time and my returns should be positive after 2 or 3 global pandemics". However, even with hindsight, it seems hard to determine if the actual risk of a global covid pandemic in Feb 2020 was 20% or 50% or some other amount, so focusing on outcomes doesn't look reliable. It seems like evaluating the prediction process rather than the outcome would be an easier approach ("we hide microphones in all WHO offices" seems like it would work).
Interesting post, would love to hear your opinion about my suggested alternative to EMH, I don't see anything in your post that contradicts it. But i'm having a very hard time accepting the idea that you can't beat the market using public data and infinite amount of computing power.
Your formulation is nifty, and intuitively makes sense to me. I am feeling too wiped right now to think about it carefully, but my off-the-cuff response is that it has something to do with the fact that 'informational edge' is a much broader category than information about the actual underlying assets.
For example, a complicated day-trading algorithm is on some level a reflection of the fact that the market is missing some information. But that information looks more like 'there is a complex relationship between assets under XYZ specific conditions' than 'the EBTDA figure in Walmart's quarterly report'. I guess this is what most anomalies represent: they are more like meta-information than information.
In which case, if you had a superintelligence with near-infinite compute, I think your intuition that it could indeed beat the market is right. I don't know much about quants, but I imagine this is pretty much what they are trying to do, with the difference being that they have bounded resources.
Indeed, Jalex Stark is a quant and says: "I spend most of my days working on specific (proprietary) instances of the general problem "design and enact decision procedures that identify market inefficiencies as well as possible, measured in terms of maximizing the ratio (expected value in dollars of trading against the inefficiency) / (amount of human time required to find the inefficiency and execute the trades)."
If the market regularly reacts to public statements of inefficiency ("stocks systematically rise on the third Thursday of each month but only under a waxing moon") by eliminating it, then this is *refutation* of the efficient market hypothesis.
Inefficiencies are priced in when they are: obvious, easy, and safe,. A non-obvious inefficiency is a true hidden edge. A non-easy inefficiency has a high barrier (capital, technical, physical, etc) to entry. A non-safe inefficiency has a high social barrier to entry.
Because names have power, I dub this the Passive Market Hypothesis.
I’ve never been a strict EMH believer. My definition is that you can’t find any free money anywhere. The market does obviously stupid things sometimes—see gamestop or amc recently. You can say what you want, but in my mind at least one of the meme stocks was for all intents and purposes objectively overvalued. But my definition is revealed by the state of the world currently—I am not rich because I saw something that was, in my mind, clearly wrong. See, even when GME was at it’s highs, the cost to borrow was over 100%, with the possibility of a squeeze going even higher and thus a margin call. Buying puts at 600 IV, yeah sure. The thing was, everyone recognized that it was too high, but the cost to realize that made it such that you were essentially indifferent. And that’s what markets being efficient means to me. It’s not that it’s always right, it’s often wrong. But it’s really hard to be lesswrong than the market.
I'm probably missing something - but when you say the vast majority of active investors do really badly, shouldn't that be impossible too? If markets are truly efficient, isn't it just as hard to underperform them as outperform?
Really excellent discussion of the EMH. Love your description of it as some kind of demon-god creature.
I'll jump in on a minor point.
"And yet, and yet…even when criminals have advance access to earnings reports, they still don’t do all that well, which is evidence for the very strongest form of the EMH (the one that no-one, including me, believes can possibly be true)."
This might instead be evidence that there is a lot more insider trading than we think -- making it so that those we know of were competing for gains against other insider traders.
I think you're understating the amount of private information available to anyone with a reasonable level of intelligence. If you have a decent level of curiosity, chances are that you know some things that the rest of the world hasn't "caught onto" yet. For example, most fans of Tesla probably realized that EVs are going to kill ICEs and that Telsa is at least 4 years ahead of anyone else in terms of building EVs long before the sudden rise in Tesla stock in Jan 2020. Similarly, people who nerd out about epidemics predicted the scale of COVID-19 before the general public.
The extreme example of this is Venture Capital. People who are a bit "weird" and follow their hunches routinely start companies worth millions or billions of dollars. Every single one of them "beat the market" by tapping private information.
None of this invalidates the EMH (which as you pointed out is unfalsifiable). The key is figuring out how to take your personal unique insights and translate them into meaningful investments (with reasonable amounts of leverage and appropriate stop-losses). Of course, the easier it is to trade something, the more likely someone has "already had that idea", so predicting the S&P500 is harder than predicting an individual stock. But starting your own company is a power move so difficult that it's virtually unbeatable.
Plenty of Venture Capitalist underform the market. Saying that everyone of them beats the market is not based on real data.
I didn't say every Venture Capitalist beats the market. Venture Capital in particular seems like a hobby for people who are already rich. I said every founder of a $1B startup beat the market.
I propose the following bet: take any founder of a $1B startup that you please, strip them of all of their wealth, give them $1M cash. What percent of them do you think would see their net-worth grow by more than the S&P 500 over the next 10 years? If the EMH is true, the answer should be 50%. Would you really be willing to bet 50% of them will under preform the market?
No, you would only assume that if you bill the capacity of that founder to work at zero. Successful founders have skill at managing companies that distinct from having access to private information.
Care to elucidate the difference between "skilled at managing companies" and "skilled at investing". Do you really claim that if I restricted the same set of people to buying/selling publicly tradable assets they would underperform the S&P 500?
Care to elucidate the difference between "skilled at managing companies" and "skilled at investing".
Tennis is not about tapping private information. Management is not about tapping private information.
Do you really claim that if I restricted the same set of people to buying/selling publicly tradable assets they would underperform the S&P 500?
The set of people buying/selling publicly tradable assets is the set of money managers at mutual funds. Those do underperform the S&P 500.
Additionally, the EDH says that all inefficiencies that can be profitably exploited disappear. If I can hire an analysts for 100k that allows me to find an efficiency that makes me 100k I'm not outperforming the S&P 500. The EDH assumes that the price you have to pay to uncover marginal inefficencies is equal to the profits that you can make from the inefficency.
The most skilled analysts likely do get employed by funds, the funds still overall underperform.
I'm still curious if you would be willing to bet against a fund run exclusively by founders of the S&P500Those do underperform the S&P 500.
Oh yeah, I definitely agree that mutual funds are terrible. Pretty sure they're optimizing for management fees, though, not to actually outperform the market.
I'm still curious if you would be willing to bet against a fund run exclusively by founders vs the S&P 500. Saying the management fee for such a fund would be ridiculously high seems like a reasonable objection though.
For that matter, would you be willing to bet against SpaceX vs the S&P 500?
Private information should be very hard to come by, it is not something that can be learned in a few minutes from an internet search.
That Tesla is far ahead in EV technology than other car producers it is not private information, so it won't give you any advantage in trading its stock. There are many other aspects to building a successful car company than just the technological aspects, and Tesla could still very well fail there. It seems also that you implying that January 2020 rise in Tesla price is due to the market suddenly realizing the Tesla lead, but there could many alternative explanations. In fact, it is a highly shorted stock, so for sure your position is debated by many smart people.
Venture capitalists do have access to private information, from financials to getting to known the founders, to internal metrics/datasets on what makes a company successful.
I agree that to be a successful entrepreneur you need to tap private information/build edges. To me it doesn't look trivial/nor easy at all: there are orders of magnitude more intelligence people than rich intelligence people.
Private information should be very hard to come by, it is not something that can be learned in a few minutes from an internet search.
I think we have different definitions of private information.
I have private information if I disagree with the substantial majority of people, even if everything I know is in principle freely available. The market is trading on the consensus expectation of the future. If that consensus is wrong and I know so, I have private information.
Specifically, when Tesla was trading at $600 or so, it was publicly available that they were building cars in a way that no other company could, but the public consensus was not that they were therefore the most valuable car company in the world.
Similarly, SpaceX is currently valued at $44B according to the public consensus. But I would be willing to be a substantial sum of money that they are worth 5-10x that and people just haven't fully grasped the implications of Starlink and Starship.
When you think about private information this way, in order to have private information all you have to do is:
1) Disagree with the general consensus
2) Be right
Incidentally, those are precisely the skills that rationality is training you for. Most people aren't optimizing for the truth, they're optimizing for fitting in with their peers.
To me it doesn't look trivial/nor easy at all: there are orders of magnitude more intelligence people than rich intelligence people.
Very few intelligent people are optimizing for "make as much money as possible". A trivial example of this, almost anyone working in academia could get a massive pay raise by switching to private industry. In addition, people can be very intelligent without being rational, so even if they claim to be optimizing for wealth they might not be doing a very good job of it. There are hordes of very intelligent people who are goldbugs or young earth creationists or global warming deniers. Why should we expect these people to behave rationally when it comes to financial self-interest when they so blatantly fail to do so in other domains?
I'm not even sure I buy the idea that there are more intelligent people than rich people. The 90% percentile for wealth in the USA is north of $1M. Going by the "MENSA" definition of highly intelligent, only 2% of people qualify. That means there are 5x as many millionaires as geniuses.
The efficient market says that the market price reflects "all available information". That's tautological, and it's hard to argue with tautologies. The problem is that trades are made by people or algorithms designed by people. Different people have different tolerances for risk, different ideas about market performance and different strategies for making money. What does it even mean for a price to reflect all available information? A lot of that information is embedded in people's minds and situations and in our institution structures.
So, as stated, the EMH is correct. It has to be. What does this mean for trading? It helps to remember is that one can only sell a stock for what someone else is willing to buy it for and one can only buy a stock for what someone else will accept as payment. The EMH says that's the price which is great, but not useful. I think the problem is that people tend to think that there is some absolute true price that is somehow discovered by trading. There's no reason to believe this, but it is comforting. People try to estimate this price by looking at earnings, by looking at liquidation value, by studying prospects, by surveying potential customers and so on. That might be useful for deciding your own buy or sell price, but that's your price.
The idea of the EMH is that the price is somehow or another actually the absolute true value of the item at a particular instant. I tend not to indulge in such mysticism. I've been through too many market crashes and transitions. I think societal and institutional structure are more important. Most people don't have money. That limits investment opportunities, and for most of my lifetime the rule has been, if there is nothing to invest in, put your money in the stock market.
P.S. Back in January, I expected the market to crash for the COVID-19 epidemic and was tempted to sell. I did sell a few losers I was planning to sell anyway, but I had enough cash on hand to decide not to sell morre. Unless COVID-19 leads to a major restructuring of our economy that spreads the wealth more broadly, odds are it is going to recover nicely.