A dashboard for progress

post by jasoncrawford · 2021-03-21T03:51:20.855Z · LW · GW · 9 comments

This is a link post for https://rootsofprogress.org/metrics-for-progress

Is progress slowing down? In a previous post I explained why I got convinced that it is. Some people making this argument point to quantitative evidence, such as GDP or total factor productivity. I gave more qualitative evidence, or perhaps very crude quantitative evidence, by counting technological breakthroughs/revolutions: five in the Second Industrial Revolution, versus only one so far in the Third.

But this approach has its own difficulties:

So maybe we need something more objective, and more focused on outputs. Here’s a half-baked idea.

It’s easy to measure progress in specific domains. For instance, we have a very good handle on the progression of Moore’s Law. The problem is that no one narrow metric captures all of economic progress.

So instead of trying for a single metric, what if we look at a dashboard comprising a handful of metrics. Make them as broad as possible while keeping them objective and well-defined, and deliberately choose a variety from across the breadth of the economy. These won’t capture everything, but together they might capture enough to give us a picture of progress.

Here’s a candidate list:

Most of these are just consumption metrics, based on a simple theory that consumption is good and is closely correlated with material well-being. For agriculture, I chose a productivity metric, because of the nature of the market: we only need so much food, after which point progress has largely been made by providing it with fewer people.

Some goals this set of metrics satisfies:

Some ways in which this approach is not perfect:

Still, no metric or dashboard is ever going to be perfect. I think this dashboard would, if nothing else, provide a useful comparison alongside GDP.

One reason I think it would be useful is that I can imagine it changing my mind, or at least altering my narrative, around progress/stagnation. If there are no fundamentally new manufacturing techniques, but we’re continuing to consume exponentially more materials on a per-capita basis, isn’t that a form of progress in the manufacturing realm? Ditto if there are no new types of vehicles, but we consume more passenger-miles or freight ton-miles.

I happen to know two of these metrics, from previous research, and both have stagnated. Here is energy, from Where Is My Flying Car?:

J. Storrs Hall, Where Is My Flying Car?

And here is mortality, from my research on infectious disease. Infectious disease mortality actually regressed slightly after about 1980 (due in part to AIDS):

US crude mortality rates. Armstrong, Conn & Pinner 1999, Fig. 2

I’m curious how the others turn out.

9 comments

Comments sorted by top scores.

comment by River (frank-bellamy) · 2021-03-21T12:17:22.699Z · LW(p) · GW(p)

I like the idea of a dashboard, but I'm not at all sold on using consumption as a proxy for progress. To take an obvious example, the computer I am on now uses fewer watts than the computer I was on a decade ago, and will therefor show up as a decrease in energy use. Yet it is superior by every metric that we judge computers by. A Tesla will get more miles for the same amount of energy compared to a traditional gas car. In general, I think we may be at a point in history where progress takes the form of producing better quality products with fewer inputs, and a dashboard that focuses on consumption will incorrectly show that as regression. 

This may be more difficult to actually get numbers for, but I think a better sort of metric might be output over input - number of watt-hours divided by number of hours of human labor used to produce those watt-hours.

Replies from: jasoncrawford
↑ comment by jasoncrawford · 2021-03-21T19:29:48.416Z · LW(p) · GW(p)

Not obvious: in many cases, greater efficiency actually increases total resource usage! See Jevons Paradox.

It's possible we're at a turning point on one or more resources where we've kind of saturated the market… but I'm far from convinced.

That said, a productivity metric like you mention is also a good idea. I chose that for agriculture because I do think we have saturated that market.

comment by antanaclasis · 2021-03-21T06:41:41.254Z · LW(p) · GW(p)

As you briefly mentioned, the focus on input measures (like quantity of materials consumed) can be different from the progress we’re really looking for. In making a progress dashboard, I’d be pretty wary of including such measures in roughly the same way I’d be wary of judging how good a university is by how many employees/student it has — at best the measure is correlated with good things, but even then it’s a cost being paid to get those things, not a benefit in its own right.

Similarly, much of the gain of technology is in making better use of resources, and especially given that many human wants can’t really be satisfied just by scaling up the quantity, I’m not sure how good of a proxy the input measures are. For example, certainly cars have gotten a lot better over the last half century, but this doesn’t much show up as any increase in the amounts of rubber and metal used in their construction.

Lest my response give the wrong impression, I do like the idea of a progress dashboard; I just am not sure that input measures are all that good a candidate for inclusion in it.

Replies from: jasoncrawford
↑ comment by jasoncrawford · 2021-03-22T00:06:18.563Z · LW(p) · GW(p)

But cars getting better and cheaper shows up as more total cars getting sold, which does show up as increased material usage, at least up to a point.

Replies from: antanaclasis
↑ comment by antanaclasis · 2021-03-22T04:25:58.902Z · LW(p) · GW(p)

Yes, I’d just say that there’s a lot resting on that “up to a point”. Lots of goods, cars included,, fairly rapidly saturate in the benefit that they bring, and hence in how much of them get consumed. At least in the US, we’re at the point where there’s almost as many cars as people, and there’s fairly little use to more than one car per person. This puts a pretty hard upper limit on how much increased car production quality/efficiency will show up (and to a lesser extent, has shown up) in material use.

My informal perception is that in the “developed world” at least, a significant proportion of goods are already far enough along in this process that there’s a substantial decay in the quality of their inputs as proxy measures.

Replies from: jasoncrawford
↑ comment by jasoncrawford · 2021-03-22T04:41:24.039Z · LW(p) · GW(p)

Maybe, but cars aren't the only things we make out of metal, either. I deliberately made the categories as broad as possible while staying objective. Still, I agree there's an issue here.

comment by Fergus Fettes (fergus-fettes) · 2022-10-16T10:06:47.513Z · LW(p) · GW(p)

You might like this[0]. If I were to make a dashboard per your description my first attempt would be to replicate realtime and per capita that paper.

[0]: Chaisson, E. J. (2010). Energy rate density as a complexity metric and evolutionary driver. Complexity, 16(3), 27–40. doi:10.1002/cplx.20323 

comment by sophia_xu · 2021-03-21T07:43:18.305Z · LW(p) · GW(p)

I love the dashboard idea!

I think the energy metric can give us good intuition visualization-wise, since it kind of is less arbitrary and many parts of consumption/progress can be seen as components of overall energy use. 

If I'd like to add something to the graph I'd say include some additional breakdowns of the energy metric: 1) we need to figure out how to measure efficiency and represent it somehow: this is entirely in the domain of epistemic uncertainty, cause massive shifts in terms of centralization/geopolitics in the last couple decades has brought significant changes that reflected by energy usage; 2) we can add a personal/industrial division to diagnose which parts are responsible; 3) in terms of sustainability we can add something like a renewable/non-renewable breakdown to see where we're going.

I think it will also be useful from a "progress measuring" perspective to see which percents of energy are used in transportation; manufacturing; service; computing etc, in addition to other parts of the dashboard you outlined.

Replies from: korin43
↑ comment by Brendan Long (korin43) · 2021-03-22T17:48:52.623Z · LW(p) · GW(p)

I was actually coming to the comment section to say that I think energy usage isn't necessarily a good metric to track progress, since we really care about "amount of stuff we can get done", not energy usage (or even efficiency).

For example, energy usage by computers would be much higher if we had similarly powerful and useful computers that used tons of power. But what we care about expanding the number of things we can do with computers. Energy usage is sort of correlated, but if one aspect of progress is portability, including battery-life of portable computers, then energy usage may be inversely correlated with progress.

There's similar problems with transportation, where progress is something like "ability to get things where we want them with minimal effort (including the effort involved in working to get money to pay for cars and gas)". Since energy usage directly causes higher prices, energy usage is plausibly inversely correlated with progress. To a certain extent we expect there to be a feedback loop (if the cost goes down, people will drive more), but there's a limit to how much people want to drive, and there's an anti-feedback cycle, where the things that become worth it to do when the price gets cheap enough are things that are less tempting to do (people don't want to do them as much or they would have already been doing it at the higher price).

It's possible this all averages out and we should still see a straight line of progress, but I feel like it's a really big assumption and depends on exactly what people's goals are. To take an extreme case, the graph where energy usage levels off is directly and unambiguously a sign of progress for a subset of environmental activists because reducing energy usage is part of their definition of progress.