Noting an error in Inadequate Equilibria

Scott Sumner offers some comments here FWIW, copying and pasting:

I certainly believe the BOJ policy had the effect of boosting Japan’s real GDP, but the figure cited by Yudkowsky (“trillions of dollars”) seems excessive.

A few points:

1. In the long run, money is neutral. Hence monetary stimulus won’t impact the long run level of Japan’s RGDP or employment.

2. There’s a lot of evidence that Kuroda’s policies boosted Japan’s NGDP.

So here’s the issue. How much evidence is there that faster NGDP growth boosted Japan’s real economy (and employment) for a period of time? (Alternatively, how flexible are Japanese wages and prices?)

I’d say there is substantial evidence. Japanese stocks responded as if the policy was boosting growth. Unemployment fell to levels well below the 2006 boomlet. Also, keep in mind that growth in Japan’s working age population slowed sharply in the past decade, so trend RGDP growth is slowing substantially. Growth held up better after the 2014 tax increase than after the previous (1997?) version. Thus if Yudkowsky’s evidence was too cursory, so is this critique.

To summarize, the article makes some good points, but only shows that Yudkowsky might be wrong, not that he is wrong. I still think there’s lots of evidence that he was right and the pessimists at the BOJ were wrong, even if he exaggerates the benefits.

As an aside, he mentions my name. But people with very different views on monetary policy effectiveness—such as Paul Krugman (2018)—also see the evidence as clearly suggesting that Kuroda’s policy worked to some extent.

(There’s lots more I could say, but I’m on vacation.)

Just to say it explicitly, I also don't see at all how the graphs you've shown could be interpreted to support Eliezer's interpretations. I also don't understand how Eliezer arrived at the decision to tell this story in this way based on this data.

My only background on this issue is reading Inadequate Equilibria and this post, so that's all I'm going on here.

I think this post deserves a response.

However, I think this error undermines a significant part of Yudkowsky's thesis. This example was one of two major anecdotes that Yudkowsky presented to show that he can often know better than experts, and he cited it repeatedly throughout the book. Yet, I think he got it wrong.

It does seem plausible that the Bank of Japan thing was an error. However, I don't think that would undermine his thesis.

It's been some time since I read the book so I could be misremembering, but I remember the big idea here to basically be "consider the incentives". If there are strong incentives to be correct about something (like the price of a stock) you should be weary of disagreeing. On the other hand, if there aren't strong incentives to be correct about something, you shouldn't be too hesitant to disagree. More generally, your hesitance should be in proportion to the strength of the incentives.

I think the Bank of Japan example was meant to be illustrative of this idea, not strong evidence for it. As I understand it, the bulk of the evidence is more gears level/theoretical/deductive. Maybe something like this:

• We know that people respond to incentives.
• Therefore, the more heavily you incentivize people to be correct about something the more likely they are to actually be correct.
• Therefore, a thing where people are heavily incentivized to be correct is likely to actually be correct.
• Therefore, you should be hesitant to disagree in places where people are heavily incentivized to be correct.

Attempted paraphrase of this post:

At time1, Eliezer thought that Sumner's macroeconomic analysis was correct, and that it showed that the Bank of Japan's monetary policy was too tight, at a cost of trillions of dollars.

At time2, Eliezer wrote Inadequate Equilibria where he used this view of time1 Eliezer as one of his central examples, and claimed that events since then had provided strong evidence that it was true: Japan had since loosened its monetary policy, and their economy had improved.

Now, at time3, you are looking back at Japan's economy and saying that it didn't actually do especially well at that time, and also that [EDITED] its monetary policy never actually became loose enough to generate inflation. So Eliezer at time2, who thought that the views of time1 Eliezer had been tested and shown correct, was wrong about that.

Eliezer's views at time1 might have been correct, but we never got a clear empirical test of that, and Eliezer at time2 was wrong to claim that we had.

These graphs seem concerning to me, but I'm worried about an information cascade before Eliezer's responded or someone with substantial expertise in macroeconomic policy has weighed in, so I'm planning to refrain from voting on this post until a week from now.

(Posting as a comment in case others feel inclined to adopt a similar policy.)

Edit: I've now upvoted, since no contrary info has come in that I've seen and at least one person with experience in economics has commented supportively.

Further illustrating Eliezer's misplaced confidence, Sumner's view is about NGDP targeting, so the success of the BOJ's policy should be based on delivering NGDP growth, not real economic variables like RGDP growth or employment rate as Eliezer implies. They were in fact successful at this (RGDP growth + Inflation = NGDP growth; with RGDP growth continuing on trend and Inflation bucking the downtrend, that's a new NGDP trajectory, baby!). Here, with 100=March 2013 as Kuroda ascended, you can see the shift in CPI trend even before the VAT impact in April 2014. Sumner was bullish on the new BOJ policy by September 2013.

So, Eliezer, you think you have identified which econbloggers, like Scott Sumner, know better than the Bank of Japan, do you? Eliezer did identify Sumner successfully, but he got lucky. His belief in Sumner was based on a misread of Sumner's position, one that led him to wrongly believe real economic variables would supply evidence for the veracity of the theory. Further compounding the issue, while employment rate might have been readable as supportive, as Matthew Barnett points out, RGDP was not. He is overconfident and should be more humble about his approach.

Ironically, Eliezer's mistake actually more strongly makes his key point. The demand for humility Eliezer was writing about stemmed from the belief that even a very good reasoner oughtn't be able to outperform "the experts." And yet, here we have a mistaken reasoner outperforming "the experts" (at least, outperforming the hawkish experts, before they were replaced by the dovish experts who implemented the new monetary policy at the BOJ). Perhaps the case for humility is not so strong after all: "it is perfectly plausible for an econblogger to write up a good analysis of what the Bank of Japan is doing wrong, and for a sophisticated reader to reasonably agree that the analysis seems decisive, without a deep agonizing episode of Dunning-Kruger-inspired self-doubt playing any important role in the analysis." I suppose one might need to decide how interchangeable "humility" and "agonizing self-doubt" are...

Eliezer is driving an intellectual racecar when many are driving intellectual horse-and-buggies. Still needs to be vacuumed out from time to time though.

Sometimes when I re-read Yudkowsky's older writings I am still comfortable with the model and conclusion, but the evidence seems less solid than on first reading. In this post, Matthew Barnett poses problems for the evidence from Japan in Yudkowsky's Inadequacy and Modesty [? · GW]. Broadly he claims that Haruhiko Kuroda's policy was not as starkly beneficial as Yudkowsky claims, although he doesn't claim the policy was a mistake.

LessWrong doesn't have a great system for handling (alleged) flaws in older posts. Higher rated posts have become more visible with the "enriched" feed, which is good, but there isn't an active mechanism for revising them in the face of critiques. In this case the author is trying to make our extinction more dignified and revisiting Japan's economy in 2013 isn't an optimal use of time. In general, authors shouldn't feel that posting to LessWrong obliges them to defend their writings in detail years or decades later.

I don't know that Barnett's critique is important enough to warrant a correction or a footnote. But it makes me wish for an editor or librarian to make that judgment, or for someone to make a second edition, or some other way that I could recommend "Read the Sequences" without disclaimers.

At some point in my life I was convinced by a theory of macroeconomics called modern monetary theory. I was convinced everybody was wrong about macroeconomics and I was right.

The reader might find it amusing to learn that, in stark contrast to many other issues of this type, this particular belief counterfactually cost me tens of millions of dollars.

Humbled by age, experience and debate with actual economists, no longer possessed by a Yudkowskian sense that I am right and everybody is wrong I am no longer certain I know the answer on macroeconomics. But I will point out that it is a characteristic prediction of MMT that monetary operations will have NO effect on inflation - only fiscal deficits determine inflation.

I'm aware this is kind of hard to justify, but I'm basically an apologist on this one. I think he was mostly right and just exaggerated the measurable magnitude. It's just so damn hard to come up with examples that are not only true and illustrative and compelling and not confusing, but also very measurably true. I do wish he had provided a more justifiable core example and overstated the result less, but I do basically the same when I'm trying to make a point. On my list of metrics I think he satisficed basically fine—I can't think of any better examples off the top of my head from pre-COVID. 

[ETA: someone asked why I thought the effect size was more than 0, which is a good question that I was trying to dodge... Here's my attempt at some justification.

First: the “reality drives straight lines on graphs” thing. The line of your economy growing stays straight because you keep doing things to make the economy keep growing. Every time someone does something to boost the economy, that line gets a little straighter. If they didn’t fix the money supply they probably would have started growing less, but they did fix it because it was the next bottleneck and that’s how lines stay straight. I’ve seen a lot of times where an intervention that has to have a clear effect by any model of the world just doesn’t show a clear effect on graphs. So at this point I’m not that convinced by being unable to pick signal out of the noise.

Second, as someone else pointed out, they once again didn’t print enough money. So while Eliezer did exaggerate to say that they had actually fixed the problem and seen his preferred result, I think he was still directionally right: they did a small intervention, it helped some, and doesn’t really show up because they didn’t do enough. It wasn’t a Volcker situation where he really drilled it into people.

Third, least convincingly, I’m just schizo enough to be able to eyeball those graphs and say sure, does look a bit like prime LFPR was up faster. And after a crunch you’re supposed to see catch-up growth, and in this case it does seem like the catch-up growth of Japan was slower than I’d expect and the post-catch-up was equally fast but relatively faster (compare Germany’s RGDP, the first non-US country I looked at). Also, I hear there was some sort of economic problem around 2014-2015 maybe? Anyways this is of course in the context of point 1, where normally it’s hard to see anything on a graph.

Fourth, again in the context of point 1, in cases like this I'd lean more on models and historical context for what we know must be going on, rather than actually expecting to see the results clearly in any given case. And I feel pretty confident that increasing the liquidity available in a faltering economy HAS to increase GDP—like, decreasing it surely decreases GDP, right? So by the reversal test…]

his example was one of two major anecdotes that Yudkowsky presented to show that he can often know better than experts

What was the other anecdote? Was it similarly overstated?

Same as DirectedEvolution, my only background is reading inadequate equilibria and this article. Does someone, after reading this post, still think Eliezer was right about the Bank of Japan?

I think your remarks are good, but I would like to add a few more critical points as an economist. First, the RGDP is a measure that captures more of the "supply side" of an economy, so it addresses issues related to the total factor productivity of an economy. In the case of Japan, as previously noted by Prescott and Hayashi in their article on the crisis of the Japanese economy in the 1990s, there is a major problem of productivity that affects the capacity for real growth. To the surprise of people who grew up with the myth of Japanese efficiency, Japan has been the economy with the lowest labor productivity among the G7 economies for over 50 years. There is a big problem with respect to lifetime employment, difficulties with union reforms, diseconomies of scale and an extremely low number of startups. A recent research by BOJ researchers indicates that the Japanese economy has serious problems of capital accumulation and low productivity of its main companies (which have practically become cyberpunk megacorps paralyzed by the lack of incentives for innovation).

In this scenario, it is very unlikely that there will be a real positive change in the picture of real economic growth for Japan. Eliezer could advocate whatever monetary policy he wanted, but he would still probably be wrong, because the solution would be more through microeconomic reform. Perhaps an analysis from the point of view of new-classical macroeconomics is more fruitful than an analysis from the market monetarism paradigm.

Prime age labor force participation rate is the standard measure the econobloggers I've followed (most notably Krugman and Brad DeLong, who are part of the community also pushing for this interpretation of monetary policy) tend to use to measure economic health, and there are reasons to see it as pointing most closely to what we actually care about (that and hourly productivity, which isn't in these charts). 

We need more epistemic spot checks like these for important claims made in other posts

bad monetary policy that cost Japan trillions of dollars in real economic growth.

Eliezer seems to be talking about trillions of dollars at stake worldwide. That's more plausible than trillions in Japan.

The LessWrong Review [? · GW] runs every year to select the posts that have most stood the test of time. This post is not yet eligible for review, but will be at the end of 2024. The top fifty or so posts are featured prominently on the site throughout the year.

Hopefully, the review is better than karma at judging enduring value. If we have accurate prediction markets on the review results, maybe we can have better incentives on LessWrong today. Will this post make the top fifty?

This monetary expansion seems to confirm MMT that government can expand money supply without inflation when production is able to respond in kind to increased demand without inflation. THIS IS HUGE, theory previously was that taxation was only route to government spending but government can just print more Trillions, national debt essentially becomes a footnote. 

Isn't it more striking that inflation didn't rise despite doubling the money supply? what would have happened if money supply didn't double???

Not an economist but it seems like deflation would have landed Japan in economic contraction rather than expansion hard to compare status quo vs resulting equilibrium from intervention likewise; actual equilibrium from intervention from resulting status quo affects. 

Japan's problem is Keynes "optical illusion" problem.  Banks don't lend deposits.  Deposits are the result of lending/investing.   It is an extraordinary delusion.  Ergo, all bank-held savings are frozen.  That destroys the velocity of circulation.  

Japan’s “lost decade” is due to the impoundment and ensconcing of monetary savings in their banks. The BOJ has unlimited transaction deposit insurance, the Japanese save more, and keep more of their savings in their banks.

“Japanese households have 52% of their money in currency & deposits, vs 35% for people in the Eurozone and 14% for the US.”

Economists simply can’t differentiate between an individual bank and the system. The equation, the capacity of a single bank to create credit as a consequence of a given primary deposit (and newly created deposits flow to other banks), is also applicable to a nonbank, financial intermediaries.  A bank: L = P (1-d) & A nonbank: L = S (1-s)

But this comparison is superficial since any expansion of credit by a commercial bank enlarges the money supply, enlarging the system, whereas any extension of credit by an intermediary simply transfers ownership of existing money within the system (a velocity relationship).

An interesting thing is that you can't print real money (gold, bitcoin, dollar if you are in Japan). Any money you can print will stop to be real ones soon, as people will exchange them into the real ones.  As a result, you will have inflation in fake money but deflation in real money. Most governments who tried to print too much money has experienced it (e.g. Russia in 1990s). 

Inflation is a Goodharted measure of economic growth. As soon as we start target it, it will decouple from economy.

All this has special relation to Japan which has very crazy monetary policy because of two things: tightly controlled exchange rate of yen to dollar (there are rules which prevent Japanese goods from being too cheap) and carry-trade (getting credits in yen to invest in US economy). In short, new printed yens are getting pumped in the US, not Japan economy (stock market), and too much inflation is prevented by regulation of exchange rate.