Posts

The Solow-Swan model of economic growth 2021-08-29T18:55:34.848Z
Black ravens and red herrings 2021-07-27T17:46:03.640Z
Could Advanced AI Drive Explosive Economic Growth? 2021-06-30T22:17:23.875Z
How much interest would there be in a fringe theories wiki? 2021-06-28T21:03:16.298Z
Three reasons to expect long AI timelines 2021-04-22T18:44:17.041Z
A new acausal trading platform: RobinShould 2021-04-01T16:56:07.488Z
Conspicuous saving 2021-03-20T20:59:50.749Z
Defending the non-central fallacy 2021-03-09T21:42:17.068Z
My guide to lifelogging 2020-08-28T21:34:40.397Z
Preface to the sequence on economic growth 2020-08-27T20:29:24.517Z
What specific dangers arise when asking GPT-N to write an Alignment Forum post? 2020-07-28T02:56:12.711Z
Are veterans more self-disciplined than non-veterans? 2020-03-23T05:16:18.029Z
What are the long-term outcomes of a catastrophic pandemic? 2020-03-01T19:39:17.457Z
Gary Marcus: Four Steps Towards Robust Artificial Intelligence 2020-02-22T03:28:28.376Z
Distinguishing definitions of takeoff 2020-02-14T00:16:34.329Z
The case for lifelogging as life extension 2020-02-01T21:56:38.535Z
Inner alignment requires making assumptions about human values 2020-01-20T18:38:27.128Z
Malign generalization without internal search 2020-01-12T18:03:43.042Z
Might humans not be the most intelligent animals? 2019-12-23T21:50:05.422Z
Is the term mesa optimizer too narrow? 2019-12-14T23:20:43.203Z
Explaining why false ideas spread is more fun than why true ones do 2019-11-24T20:21:50.906Z
Will transparency help catch deception? Perhaps not 2019-11-04T20:52:52.681Z
Two explanations for variation in human abilities 2019-10-25T22:06:26.329Z
Misconceptions about continuous takeoff 2019-10-08T21:31:37.876Z
A simple environment for showing mesa misalignment 2019-09-26T04:44:59.220Z
One Way to Think About ML Transparency 2019-09-02T23:27:44.088Z
Has Moore's Law actually slowed down? 2019-08-20T19:18:41.488Z
How can you use music to boost learning? 2019-08-17T06:59:32.582Z
A Primer on Matrix Calculus, Part 3: The Chain Rule 2019-08-17T01:50:29.439Z
A Primer on Matrix Calculus, Part 2: Jacobians and other fun 2019-08-15T01:13:16.070Z
A Primer on Matrix Calculus, Part 1: Basic review 2019-08-12T23:44:37.068Z
Matthew Barnett's Shortform 2019-08-09T05:17:47.768Z
Why Gradients Vanish and Explode 2019-08-09T02:54:44.199Z
Four Ways An Impact Measure Could Help Alignment 2019-08-08T00:10:14.304Z
Understanding Recent Impact Measures 2019-08-07T04:57:04.352Z
What are the best resources for examining the evidence for anthropogenic climate change? 2019-08-06T02:53:06.133Z
A Survey of Early Impact Measures 2019-08-06T01:22:27.421Z
Rethinking Batch Normalization 2019-08-02T20:21:16.124Z
Understanding Batch Normalization 2019-08-01T17:56:12.660Z
Walkthrough: The Transformer Architecture [Part 2/2] 2019-07-31T13:54:44.805Z
Walkthrough: The Transformer Architecture [Part 1/2] 2019-07-30T13:54:14.406Z

Comments

Comment by Matthew Barnett (matthew-barnett) on The Solow-Swan model of economic growth · 2021-08-29T19:37:41.162Z · LW · GW

My next post will be about endogenous growth theory, which tries to understand the causes of technological progress. It may take a while to come out though, so for now you can take a look at the excellent blog The Roots of Progress.

Comment by Matthew Barnett (matthew-barnett) on The Death of Behavioral Economics · 2021-08-27T18:46:20.288Z · LW · GW

I note that I am confused. I am confused mostly because the claim "loss aversion exists only for large losses" seems to be completely disharmonious with my anecdotal experience, and I tend to view anecdotal experience as an often semi-reliable guide to the accuracy of social science. If the strong version of this claim is true, how would you explain the following facts?

  • This video, in which a bunch of people on the street did not want to bet at favorable odds that a coin flip would land tails.
  • A large fraction of the population holds their money in low-interest savings accounts, rather than putting their money in a brokerage account where they can buy stocks.
  • People systematically refuse to throw away stuff that they own, even if they haven't used the items in years. But when asked whether they would pay a dollar to put one more useless item in their home, they will refuse.
  • People usually view petty theft as horrible when it's committed against them, even though they usually don't become extremely happy when you unexpectedly give them $50.
  • It's well known that people will panic-sell during a market correction, even though if the market rises by 10% people don't change their behavior by much, and even if they don't have much money in the stock market.
  • Anecdotally, I used to gamble small amounts of money and would feel really bad if I lost it, even if it was just $20. Lots of people don't like taking friendly epistemic bets for similar reasons.

My guess is that you could come up with ad-hoc explanations for all of these things without reference to loss aversion, but that doesn't seem very elegant to me. A proclivity for loss aversion present in 50+% of humans appears like the most natural, simple explanation.

[ETA: However, after reading the link from Kaj Sotala I'm starting to feel my mind being changed.]

Comment by Matthew Barnett (matthew-barnett) on Obesity Epidemic Explained in 0.9 Subway Cookies · 2021-08-23T02:11:16.326Z · LW · GW

I don't quite understand what you mean. National wealth works just like individual wealth. You linked to the Wikipedia page on "Austerity" (presumably because you think such policies do not work) but I don't understand your point. Do you think either spending cuts or tax increases usually never increase national wealth? I also don't think those things are equivalent to national income or expenses.

Comment by Matthew Barnett (matthew-barnett) on Matthew Barnett's Shortform · 2021-08-20T20:26:21.375Z · LW · GW

Maybe it is more food and less hard work, in general. Or maybe it is something in the food that screws up many (but not all) people's metabolism.

I'm with you that it probably has to do with what's in our food. Unlike some, however, I'm skeptical that we can nail it down to "one thing", like a simple additive, or ingredient. It seems most likely to me that companies have simply done a very good job optimizing processed food to be addicting, in the last 50 years. That's their job, anyway.

Scott Alexander reviewed a book from Stephan Guyenet about this hypothesis, and I find it quite compelling.

Now, I am probably not the first person to think about this -- if it is about lifestyle, then perhaps we should see clear connection between obesity and profession. To put it bluntly, are people working in offices more fat than people doing hard physical work? I admit I never actually paid attention to this.

That's a good question. I haven't looked into this, and may soon. My guess is that you'd probably have to adjust for cognitive confounders, but after doing so I'd predict that people in highly physically demanding professions tend to be thinner and more fit (in the sense of body fat percentage, not necessarily BMI). However, I'd also suspect that the causality may run in the reverse direction; it's a lot easier to exercise if you're thin.

Comment by Matthew Barnett (matthew-barnett) on Matthew Barnett's Shortform · 2021-08-17T20:46:50.776Z · LW · GW

To clarify, there are two related but separate questions about obesity that are worth distinguishing,

  1. What explains why people are more obese than 50 years ago? And what can we do about it?
  2. What explains why some people are more obese than others, at a given point of time? And what can we do about it?

In my argument, I was primarily saying that CICO was important for explaining (1). For instance, I do not think that the concept of metabolic privilege can explain much of (1), since 50 years is far too little of time for our metabolisms to evolve in such a rapid and widespread manner. So, from that perspective, I really do think that overconsumption and/or lack of exercise are the important and relevant mechanisms driving our current crisis. And further, I think that our overconsumption is probably related to processed food.

I did not say much about (2), but I can say a little about my thoughts now. I agree that people vary in how "fast" their metabolisms expend calories. The most obvious variation is, as you mentioned, the difference between the youthful metabolism and the metabolism found in older people.

However...

Then you have the opposite type of people, whose metabolism stubbornly refuses to release the fat from fat cells, no matter how much they starve or how much they try to exercise... (In extreme cases, if they try to starve, they will just get weak and maybe fall in coma, but they still won't lose a single kilogram.)

I don't think these people are common, at least in a literal sense. Obesity is very uncommon in pre-industrialized cultures, and in hunter-gatherer settings. I think this is very strong evidence that it is feasible for the vast majority of people to be non-obese under the right environmental circumstances (though feasible does not mean easy, or that it can be done voluntarily in our current world). I also don't find personal anecdotes from people about the intractability of losing weight compelling, given this strong evidence.

Furthermore, in addition to the role of metabolism, I would also point to the role of cognitive factors like delayed gratification in explaining obesity. You can say that this is me just being "smug" or "blaming fat people for their own problems" but this would be an overly moral interpretation of what I view as simply an honest causal explanation. A utilitarian might say that we should only blame people for things that they have voluntary control over. So in light of the fact that cognitive abilities are largely outside of our control, I would never blame an obese person for their own condition.

Instead of being moralistic, I am trying to be honest. And being honest about the cause of a phenomenon allows us to invent better solutions than the ones that exist. Indeed, if weight loss is a simple matter of overconsumption, and we also admit that people often suffer from problems of delayed gratification, then I think this naturally leads us to propose medical interventions like bariatric surgery or weight loss medication—both of which have a much higher chance of working than solutions rooted in a misunderstanding of the real issue. 

Comment by Matthew Barnett (matthew-barnett) on Matthew Barnett's Shortform · 2021-08-16T21:17:45.386Z · LW · GW

What specifically is the advantage of the version with calories?

Well, let me consider a recent, highly upvoted post on here: A Contamination Theory of the Obesity Epidemic. In it, the author says that the explanation for the obesity crisis can't be CICO,

“It's from overeating!”, they cry. But controlled overfeeding studies (from the 1970's—pre-explosion) struggle to make people gain weight and they loose it quickly once the overfeeding stops. (Which is evidence against a hysteresis theory.)

“It's lack of exercise”, they yell. But making people exercise doesn't seem to produce significant weight loss, and obesity is still spreading despite lots of money and effort being put into exercise.

If CICO is literally true, in the same way that the "atoms in, atoms out" theory is true, then this debunking is very weak. The obesity epidemic must be due to either overeating or lack of exercise, or both.

The real debate is, of course, over which environmental factors caused us to eat more, or exercise less. But if you don't even recognize that the cause must act through this mechanism, then you're not going to get very far in your explanation. That's how you end up proposing that it must be some hidden environmental factor, as this post does, rather than more relevant things related to the modern diet.

My own view is that the most likely cause of our current crisis is that modern folk have access to more and a greater variety of addicting processed food, so we end up consistently overeating. I don't think this theory is obviously correct, and of course it could be wrong. However, in light of the true mechanism behind obesity, it makes a lot more sense to me than many other theories that people have proposed, especially any that deny we're overeating from the outset.

Comment by Matthew Barnett (matthew-barnett) on Matthew Barnett's Shortform · 2021-08-16T01:03:20.101Z · LW · GW

There have been a few posts about the obesity crisis here, and I'm honestly a bit confused about some theories that people are passing around. I'm one of those people thinks that the "calories in, calories" (CICO) theory is largely correct, relevant, and helpful for explaining our current crisis. 

I'm not actually sure to what extent people here disagree with my basic premises, or whether they just think I'm missing a point. So let me be more clear.

As I understand, there are roughly three critiques you can have against the CICO theory. You can think it's,

(1) largely incorrect
(2) largely irrelevant
(3) largely just smugness masquerading as a theory

I think that (1) is simply factually wrong. In order for the calorie intake minus expenditure theory to be factually incorrect, scientists would need to be wrong about not only minor details, but the basic picture concerning how our metabolism works. Therefore, I assume that the real meat of the debate is in (2) and (3).

Yet, I don't see how (2) and (3) are defensible either. As a theory, CICO does what it needs to do: compellingly explains our observations. It provides an answer to the question, "Why are people obese at higher rates than before?", namely, "They are eating more calories than before, or expending fewer calories, or both."

I fully admit that CICO doesn't provide an explanation for why we eat more calories before, but it never needed to on its own. Theories don't need to explain everything to be useful. And I don't think many credible people are claiming that "calories in, calories out" was supposed to provide a complete picture of what's happening (theories rarely explain what drives changes to inputs in the theory). Instead, it merely clarifies the mechanism of why we're in the current situation, and that's always important.

It's also not about moral smugness, any more than any other epistemic theory. The theory that quitting smoking improves one's health does not imply that people who don't quit are unvirtuous, or that the speaker is automatically assuming that you simply lack willpower. Why? Because is and ought are two separate things. 

CICO is about how obesity comes about. It's not about who to blame. It's not about shaming people for not having willpower. It's not about saying that you have sinned. It's not about saying that we ought to individually voluntarily reduce our consumption. For crying out loud, it's an epistemic theory not a moral one!

To state the obvious, without clarifying the basic mechanism of how a phenomenon works in the world, you'll just remain needlessly confused.

Imagine if people all around the world people were getting richer (as measured in net worth), and we didn't know why. To be more specific, suppose we didn't understand the "income minus expenses" theory of wealth, so instead we went around saying things like, "it could be the guns", "it could be factories", "it could be the that we have more computers." Now, of course, all of these explanations could play a role in why we're getting richer over time, but none of them make any sense without connecting them to the "income minus expenses theory."

To state "wealth is income minus expenses" does not in any way mean that you are denying how guns, factories, and computers might play a role in wealth accumulation. It simply focuses the discussion on ways that those things could act through the basic mechanism of how wealth operates.

If your audience already understands that this is how wealth works, then sure, you don't need to mention it. But in the case of the obesity debate, there are a ton of people who don't actually believe in CICO; in other words, there are a considerable number of people who firmly believe critique (1). Therefore, refusing to clarify how your proposed explanation connects to calories, in my opinion, generates a lot of unnecessary confusion.

As usual, the territory is never mysterious. There are only brains who are confused. If you are perpetually confused by a phenomenon, that is a fact about you, and not the phenomenon. There does not in fact need to be a complicated, clever mechanism that explains obesity that all researchers have thus far missed. It could simply be that the current consensus is correct, and we're eating too many calories. The right question to ask is what we can do to address that.

Comment by Matthew Barnett (matthew-barnett) on Obesity Epidemic Explained in 0.9 Subway Cookies · 2021-08-16T00:35:07.308Z · LW · GW

While "eat less exercise more" may be fine advice at a personal level, it isn't a sufficient answer to the society-wide epidemic of obesity that we currently face.

I disagree, although I found the evidence in the linked posts to be rather weak. When I think of "calories in, calories out" I don't immediately think, "Therefore we should push for voluntary dieting changes." There are likely involuntary and external factors related to our consumption, which act through the CICO mechanism, driving the obesity crisis.

It's a bit like if we were debating the "income minus spending theory of wealth." One hand, you're right that "just spend less" is unhelpful advice for people. However, the theory is not actually an incorrect one. If people are actually getting poorer, then it's either because their incomes went down, or their spending went up, or both. Denying the basic mechanism won't lead to a solution. It will just make you confused forever.

Comment by Matthew Barnett (matthew-barnett) on A Response to A Contamination Theory of the Obesity Epidemic · 2021-08-14T04:45:34.651Z · LW · GW

I notice I Am Confused about this. So are you.

Never be too sure.

I don't think I can respond to everything in your comment, but let me try to address the main point. As I understand, you say that something is left unexplained by the "Calories in, Calories out" paradigm. That something is explained by your question,

Why did the "set point" go up, permanently?

I think the most likely explanation is simply that modern food is tastier than the more bland food eaten in the past. "Taste" here should be interpreted as capturing all dimensions of the satisfaction of eating, including texture, mouthfeel, and aftertaste. There is also a simple explanation for why typical food has gotten tastier over time; namely, food science has gotten better, corporations have become more efficient at producing and marketing processed foods, and consumer incomes have gotten higher—thus enabling more access and greater choice.

This explanation also perfectly predicts your long paragraph addressing possible causes. Is it fat? Carbs? High-glycemic index foods? Not enough grains? I ask: why couldn't it be all those things at once?

If the reason why we eat more is because food has gotten tastier, then we should also expect the "cause" to be multifaceted. After all, most people don't think that there's only "one thing" that makes food taste good. Taste is more complicated than that, and varies between people.

Reducing the question of "why are we getting obese?" to "why do foods taste good?" doesn't solve the problem, of course—we still don't have a full theory of why food tastes good. But, in my opinion, if it's correct, then it totally deconfuses the proximate mechanism here.

In that sense, I think my CO2 analogy holds quite well. There are many reasons why people omit CO2: electricity, temperature control, transportation etc. And as we've gotten richer, those justfications have become more salient, as people can afford to purchase service that provide those benefits, omitting CO2 as a byproduct. Simply knowing this doesn't mean you've solved climate change, of course, but it gets you a lot further than "Why are people burning more CO2 than before? I don't know; could be anything."

Comment by Matthew Barnett (matthew-barnett) on A Response to A Contamination Theory of the Obesity Epidemic · 2021-08-14T02:25:58.230Z · LW · GW

My primary claim was that we already understand the main proximate cause of the obesity pandemic. It has something to do with people maintaining calorie a calorie surplus—the difference between our calorie intake and our energy expenditure.

If I understand your reply correctly, you are essentially saying, "vegetable oils likely cause our bodies to retain extra calories than they otherwise would." A reasonable conjecture, but let me offer another.

Assume that our bodies do not retain more or fewer calories depending on what we eat. Instead, our calorie surplus is measured reasonably well by the number on the nutrition label. Then, naturally, the problem is simply that we're eating too much

Of course, this theory leaves a lot to be explained, such as why we're eating so much in the first place. However, we also have a simple answer for that: modern processed food generally tastes good, gets people hooked, and causes us to have more frequent and more intense food cravings. As far as Occam is concerned, I don't see why we need much more than this theory.

Comment by Matthew Barnett (matthew-barnett) on A Response to A Contamination Theory of the Obesity Epidemic · 2021-08-14T00:37:06.352Z · LW · GW

Suppose someone wrote an essay that sought to uncover the cause of climate change. Admitting that the explanation may be multiplex, they mention automobiles, TVs, and light-bulbs. "Any of these could be the real explanation for why the globe has warmed," they explain, "but personally, my bet is on automobiles," before laying the striking correlation between automobile adoption and global temperature.

Now you check and—indeed—they never once mentioned the role of greenhouse gasses, which is the factor connecting all of the candidate causes together. In that case, I would be quite exasperated by such an essay, as it would seem to distort, rather than clarify, the underlying mechanism driving climate change.

Similarly, I am exasperated by this essay in your neglect to mention the role of calories. We have overwhelming evidence that weight is determined directly by caloric intake and energy expenditure. There are, of course, sensible questions pertaining to how much of one's weight can be explained these factors, but no doubt about whether the answer is at least "a great deal."

The simple theory, as in the case of climate change, would begin by identifying how any potential cause connected to caloric intake and energy expenditure. Now, I may have missed something, but I did not see you do this. From the point of view of someone who already understands this background mechanism well, perhaps this omission makes sense. But as a casual reader, I didn't quite understand why you believed this theory to be plausible.

Comment by Matthew Barnett (matthew-barnett) on Rob B's Shortform Feed · 2021-07-28T02:45:50.494Z · LW · GW

I'd personally like to find some cruxes between us some time, though I don't yet know the best format to do that. I think I'll wait to see your responses to Issa's question first.

Comment by Matthew Barnett (matthew-barnett) on Black ravens and red herrings · 2021-07-27T20:52:59.412Z · LW · GW

Nice! I. J. Good himself pointed to another example of how this rule might break in his paper appropriately titled (for my post anyway), "The White Shoe is a Red Herring",

Suppose that we know we are in one or other of two worlds, and the hypothesis, H, under consideration is that all the ravens in our world are black. We know in advance that in one world there are a hundred black ravens, no non-black ravens, and a million other birds; and that in the other world there are a thousand black ravens, one white raven, and a million other birds. A bird is selected equiprobably at random from all the birds in our world. It turns out to be a black raven. This is strong evidence (a Bayes-Jefrreys-Turing factor of about 10) that we are in the second world, wherein not all ravens are black.

From Wikipedia,

Hempel rejected this as a solution to the paradox, insisting that the proposition 'c is a raven and is black' must be considered "by itself and without reference to any other information", and pointing out that it "... was emphasized in section 5.2(b) of my article in Mind ... that the very appearance of paradoxicality in cases like that of the white shoe results in part from a failure to observe this maxim."

ETA: I now see that the paper you linked cites this example. Cool.

Comment by Matthew Barnett (matthew-barnett) on Matthew Barnett's Shortform · 2021-07-27T18:43:06.722Z · LW · GW

[ETA: Apparently this was misleading; I think it only applied to one company, Alienware, and it was because they didn't get certification, unlike the other companies.]

In my post about long AI timelines, I predicted that we would see attempts to regulate AI. An easy path for regulators is to target power-hungry GPUs and distributed computing in an attempt to minimize carbon emissions and electricity costs. It seems regulators may be going even faster than I believed in this case, with new bans on high performance personal computers now taking effect in six US states. Are bans on individual GPUs next?

Comment by Matthew Barnett (matthew-barnett) on Matthew Barnett's Shortform · 2021-07-16T23:37:15.862Z · LW · GW

Yes, I would make this distinction too. Yet, I submit that few people actually believe, or even say they believe, that the main problems in the world are caused by people being gratuitously or sadistically evil. There are some problems that people would explain this way: violent crime comes to mind. But I don't think the evil hypothesis is the most common explanation given by non-rationalists for why we have, say, homelessness and poverty.

That is to say that, insofar as the common rationalist refrain of "problems are caused by incentives dammit, not evil people" refers to an actual argument people generally give, it's probably referring to the argument that people are selfish and greedy. And in that sense, the rationalists and non-rationalists are right: it's both the system and the actors within it.

Comment by Matthew Barnett (matthew-barnett) on Matthew Barnett's Shortform · 2021-07-16T23:16:27.188Z · LW · GW

Rationalists are fond of saying that the problems of the world are not from people being evil, but instead a result of the incentives of our system, which are such that this bad outcome is an equilibrium. There's a weaker thesis here that I agree with, but otherwise I don't think this argument actually follows.

In game theory, an equilibrium is determined by both the setup of the game, and by the payoffs for each player. The payoffs are basically the values of the players in the game—their utility functions. In other words, you get different equilibria if players adopt different values.

Problems like homelessness are caused by zoning laws, yes, but they're also caused by people being selfish. Why? Because lots of people could just voluntarily donate their wealth to help homeless people. Anyone with a second house could decide to give it away. Those with spare rooms could simply rent them out for free. There are no laws saying you must spend your money on yourself.

A simple economic model would predict that if we redistributed everyone's extra housing, then this would reduce the incentive to create new housing. But look closer at the assumptions in that economic model. We say that the incentives to build new housing are reduced because few people will pay to build a house if they don't get to live in it or sell it to someone else. That's another way of assuming that people value their own consumption more than that of others—another way of saying that people are selfish.

More fundamentally, what it means for something to be an incentive is that it helps people get what they want. Incentives, therefore, are determined by people's values; they are not separate from them. A society of saints would have different equilibria than a society of sinners, even if both are playing the same game. So, it really is true that lots of problems are caused by people being bad.

Of course, there's an important sense in which rationalists are probably right. Assume that we can change the system but we can't change people's values. Then, pragmatically, the best thing would be to change the system, rather than fruitlessly try to change people's values.

Yet it must be emphasized that this hypothesis is contingent on the relative tractability of either intervention. If it becomes clear that we can genuinely make people less selfish, then that might be a good thing to try. 

My main issue with attempts to redesign society in order to make people less selfish or more cooperative is that you can't actually change people's innate preferences by very much. The most we can reasonably hope for is to create a system in which people's selfish values are channeled to produce social good. That's not to say it wouldn't be nice if we could change people's innate preferences. But we can't (yet).

(Note that I wrote this as a partial response to jimrandomh's shortform post, but the sentiment I'm responding to is more general than his exact claim.)

Comment by Matthew Barnett (matthew-barnett) on What will the twenties look like if AGI is 30 years away? · 2021-07-15T18:51:00.227Z · LW · GW

It sounds like we might not disagree a lot? We've exchanged a few responses to each other in the past that may have given the impression that we disagree strongly on AI timelines, but plausibly we just frame things differently.

Roughly speaking, when I say "AI timelines" I'm referring to the time during which AI fundamentally transforms the world, not necessarily when "an AGI" is built somewhere. I think this framing is more useful because it tracks more closely what EAs actually care about when they talk about AI.

I also don't think that the moment GDP accelerates is the best moment to intervene, though my framing here is different. I'd be inclined to discard a binary model of intervention during which all efforts after some critical threshold are wasted. Rather, intervention is a lot like the time-value of money in finance. In general, it's better to have money now rather than later; similarly, it's better to intervene earlier rather than later. But the value of interventions diminish continuously as time goes onwards, eventually getting near zero.

The best way to intervene also depends a lot on how far we are from AI-induced growth. So, for example, it might not be worth trying to align current algorithms, because that sort of work will be relatively more useful when we know which algorithms are actually being used to build advanced AI. Relatively speaking, it might be worth more right now to build institutional resilience, in the sense of creating incentive structures for actors to care about alignment. And hypothetically, if I knew about the AI alignment problem in, say, the 1920s, I might have recommended investing in the stock market until we have a better sense as to what form AI will take.

Comment by Matthew Barnett (matthew-barnett) on What will the twenties look like if AGI is 30 years away? · 2021-07-14T23:46:54.729Z · LW · GW

My best guess is that the next 10 (or maybe 20) years in AI will look a bit like the late 1990s and early 2000s when a lot of internet companies were starting up. If you look at the top companies in the world by market cap, most of them are now internet companies. We can compare that list to the top companies in 1990, which were companies like General Motors, Ford, and General Electric. In other words, internet companies totally rocked the boat in the last few decades.

From a normal business standpoint, the rise of the internet was a massive economic shift, and continues to be the dominant engine driving growth in the US financial markets. Since 2004, the Vanguard Information Technology ETF went up about 734%, compared to a rise of only 294% in the S&P 500 during the same time period.

And yet, overall economic growth is still sluggish. Despite the fact that we went from a world in which almost no one used computers, to a world in which computers are an essential part of almost everyone's daily lives, our material world is surprisingly still pretty similar. The last two decades of growth have been the slowest decades in over a century

If you just focused on the fact that our smartphones are way better than anything that came before (and super neat), you'd miss the fact that smartphones aren't making the world go crazy. Likewise, I don't doubt that we will get a ton of new cool AI products that people will use. I also think it's likely that AI is going to rock the boat in the financial markets, just like internet companies did 20 years ago. I even think it's likely that we'll see the rise of new AI products that become completely ubiquitous, transforming our lives. 

For some time, people will see these AI products as a big deal. Lots of people will speculate about how the next logical step will be full automation of all labor. But I still think that by the end of the decade, and even the next, these predictions won't be vindicated. People will still show up to work to get paid, the government will still operate just as it did before, and we'll all still be biological humans.

Why? Because automating labor is hard. To present just one illustration, we still don't have the ability to automate speech transcription. Look into current transcription services and you'll see why. Writing an AI that can transcribe some 75% of your words correctly turned out to be relatively easy. But it's been much harder to do the more nuanced stuff, like recognize which speakers are saying what, correctly transcribe the uhhs and ahhs and made-up names like, say, "Glueberry". 

My impression is that when people look at current AI tech, are impressed, and then think in their heads, "Well, if we're already here, then we'll probably be able to automate all human labor within, say, 10 years" they just aren't thinking through all the very complicated difficulties that are actually needed to fully replace labor. And that makes sense, since that stuff is less salient in our minds: we can see the impressive feats that AI can do already, since we have it and the demos are there for us to marvel at. It's harder for us to see all the stuff AI can't yet do.

Comment by Matthew Barnett (matthew-barnett) on You are allowed to edit Wikipedia · 2021-07-05T21:39:22.914Z · LW · GW

For the type of things people are likely to complain about, there will tend to be admins who watch the page and will revert your edits if you put something they disagree with. So, it's probably best to edit Wikipedia only if you either (1) are already powerful there, or (2) are editing uncontroversial pages.

Comment by Matthew Barnett (matthew-barnett) on Could Advanced AI Drive Explosive Economic Growth? · 2021-07-01T04:36:35.431Z · LW · GW

Nuclear fission is often compared to AI development, but I think it's probably a bad comparison.

Nuclear bombs weren't built by successively making bombs more efficient or bigger. Rather, they were a result of a particular quirk of physics works at the subatomic level. Once that quirk was understood, people quickly realized that it was possible, and governments began enriching uranium. 

By contrast, machine learning and AI research builds on itself. Intelligence looks more like a continuum, and will be built from a culmination of ideas and knowledge. There are rarely large theoretical leaps made in the AI field; at least, none which lend themselves rapidly to practical applications. Most of the best ideas in AI usually have ancestry in precursor ideas which were almost as good. For example, almost all of the ideas in modern ML are downstream of probability theory, learning theory, and backpropagation, which were developed many decades ago.

Comment by Matthew Barnett (matthew-barnett) on ChristianKl's Shortform · 2021-06-29T21:38:35.689Z · LW · GW

current debates bring people who [...] attack modern tools of evidence-based medicine like meta-reviews as flawed.

Can you present an example of what you were thinking about here?

Comment by Matthew Barnett (matthew-barnett) on How much interest would there be in a fringe theories wiki? · 2021-06-29T19:20:35.507Z · LW · GW

Yeah, that's reasonable. FWIW I meant "think for five minutes about the specific objection" rather than "think for five minutes about wiki systems."

Comment by Matthew Barnett (matthew-barnett) on How much interest would there be in a fringe theories wiki? · 2021-06-29T18:41:21.313Z · LW · GW

I'm having trouble keeping track of the central objection. First, it was you won't be able to filter out bad content --> I'm not aware of platforms that use invite-only --> I'm not aware of platforms that use invite only AND pay people for non-altruistic content. Anyway, it would be too curt to just say "Have you tried thinking for five minutes first?" so I'll try to explain my perspective. :)

There are vast numbers of websites filled with people writing content about various things. This website is even one of them. If people think this website is an interesting concept, then it would be more productive to search over the space of possible site designs and see if one makes sense. I agree that most immediate solutions that come to mind don't seem viable, but that's typical of new ideas.

Comment by Matthew Barnett (matthew-barnett) on How much interest would there be in a fringe theories wiki? · 2021-06-29T17:59:05.203Z · LW · GW

Good idea!

Comment by Matthew Barnett (matthew-barnett) on How much interest would there be in a fringe theories wiki? · 2021-06-29T16:34:00.786Z · LW · GW

It seems like you’re just asking me the question I asked in this post?

Anyway, in a comment above you said that interest in consumption was not the same as interest in contributions. But typically, interest in consumption represents weak evidence of willingness to pay for a good!

Comment by Matthew Barnett (matthew-barnett) on rohinmshah's Shortform · 2021-06-29T08:10:05.085Z · LW · GW

IMO, the argument for a Pigouvian tax in this context is that for a given amount of CO2 reduction that you want, the tax is a cheaper way of getting it

Since the argument about internalizing externalities fails in this case (as the tax is local), arguably the best way of modeling the problem is viewing each community as having some degree of altruism. Then, just as EAs might say “donate 10% of your income in a cause neutral way” the argument is that communities should just spend their “climate change money” reducing carbon in the way that’s most effective, even if it’s not rationalized in some sort of cost internalization framework. And Noah pointed out in his article (though not in the part I quoted) that R&D spending is probably more effective than imposing carbon taxes.

Comment by Matthew Barnett (matthew-barnett) on How much interest would there be in a fringe theories wiki? · 2021-06-28T23:09:59.066Z · LW · GW

Is there any Wiki that's invite-only that works?

Depends on what you mean by "works" but there's the Timelines wiki and the Cause Prioritization Wiki which says that you must "Contact Issa Rice to obtain an account on the wiki." I'm not aware of other examples (but I'm sure there are at least a few).

Generally, you have to think about incentives. Publishing content on a Wiki generally means getting less personal recognition for having written the content. Publishing things on Wikipedia is useful because while you get little personal recognition you get a lot of reach. 

I think the main reason why people post on wikis is that they find it fun, or satisfying in some way, rather than the reach it gets. It's easier than blogging because you can be satisfied with contributing just one line, rather than requiring a fully written article every time you want to publish something. Also I doubt anyone would want to contribute to this project if it were organized as a set of blogs.

Comment by Matthew Barnett (matthew-barnett) on How much interest would there be in a fringe theories wiki? · 2021-06-28T21:45:09.931Z · LW · GW

If it isn't backed by a surprisingly effective system for filtering out low-quality content, then most of the content on it will be crap

I feel like it won't be that hard. My current idea is to make editing privileges invite-only.

I'm already cringing pretty hard at your list of examples; putting "cryonics works" and "there was an industrial civilization of birds in Antarctica" into an epistemic-status category together, comes across as a smear against cryonics, because the epistemic statuses of these two ideas is wildly different.

I didn't look into the industrial civilization of birds hypothesis too much, but it seems like a perfectly natural theory that is consistent with how scientists usually think about the world. Obviously, specific geological evidence probably doesn't back the theory up, but I'm not really trying to judge the plausibility of my examples right now. I'm more interested in whether they're the type of things I'd want to read and think about. Judging from the upvotes, it seems like a lot of people enjoyed thinking about CellBioGuy's theory.

Comment by Matthew Barnett (matthew-barnett) on What precautions should fully-vaccinated people still be taking? · 2021-06-28T20:02:44.373Z · LW · GW

They said "No, Covid isn’t flu, but when you are fully vaccinated then the risk level becomes comparable." So, what's the evidence that long term heart and brain damage of COVID-19 is worse than the flu (or the cold for that matter) after you're already vaccinated?

Comment by Matthew Barnett (matthew-barnett) on rohinmshah's Shortform · 2021-06-28T19:06:38.474Z · LW · GW

Fwiw, at < $6000 per person that seems like a bargain to me

Possibly my intuition here comes from seeing COVID-19 risks as not too dissimilar from other risks for young people, like drinking alcohol or doing recreational drugs, accidental injury in the bathroom, catching the common cold (which could have pretty bad long-term effects), kissing someone (and thereby risk getting HSV-1 or the Epstein–Barr virus), eating unhealthily, driving, living in an area with a high violent crime rate, insufficiently monitoring one's body for cancer, etc. I don't usually see people pay similarly large costs to avoid these risks, which naturally makes me think that people don't actually value their time or their life as much as they say. 

One possibility is that everyone would start paying more to avoid these risks if they were made more aware of them, but I'm pretty skeptical. The other possibility seems more likely to me: value of life estimates are susceptible to idealism about how much people actually value their own life and time, and so when we focus on specific risk evaluations, we tend to exaggerate.

ETA: Another possibility I didn't mention is that rationalists are just rich. But if this is the case, then why are they even in a group house? I understand the community aspect, but living in a group house is not something rich people usually do, even highly social rich people.

Still, I think you shouldn't ask about paying large sums of money -- the utility-money curve is pretty sharply nonlinear as you get closer to 0 money, so the amount you'd pay to avoid a really bad thing is not 100x the amount you'd pay to avoid a 1% chance of that bad thing.

Makes sense.

Comment by Matthew Barnett (matthew-barnett) on rohinmshah's Shortform · 2021-06-28T07:22:54.317Z · LW · GW

I think $30k-$40k might make sense.

I'm kind of confused right now. At a mere $15k, you could probably get a pretty good software engineer to work for a month on any altruistic project you wish. I'm genuinely curious about why you think your work is so irreplaceable (and I'm not saying it isn't!).

Comment by Matthew Barnett (matthew-barnett) on rohinmshah's Shortform · 2021-06-28T05:50:59.750Z · LW · GW

The estimate was largely driven by fear of long covid + a much higher value per hour of time

Makes sense, though FWIW I wasn't estimating their wage at $20 an hour. Most cases are mild, and so productivity won't likely suffer by much in most cases. I think even if the average wage there is $100 after taxes, per hour (which is pretty rich, even by Bay Area standards), my estimate is near the high end of what I'd expect the actual loss of productivity to be. Though of course I know little about who is there.

ETA: One way of estimating "altruistic benefits from housemate's work that aren't captured by the market price of their salary" is to ask at what after-tax wage you'd be willing to work for a completely pointless project, like painting a wall, for 2 weeks. If it's higher than $100 an hour I commend those at Event Horizon for their devotion to altruism!

Comment by Matthew Barnett (matthew-barnett) on rohinmshah's Shortform · 2021-06-28T05:28:57.386Z · LW · GW

At Event Horizon we had a policy for around 6-9 months where if you got a microcovid, you paid $1 to the house

That gives an implied cost of $1 million dollars for someone getting COVID-19, which seems way overpriced to me. I thought I'd do a quick Fermi estimate to verify my intuitions.

I don't know how many people are in Event Horizon, but I'll assume 15. Let's say that on average about 10 people will get COVID-19 if one person gets it, due to some people being able to isolate successfully. I'm going to assume that the average age there is about 30, and the IFR is roughly 0.02% based on this paper. That means roughly 0.002 expected deaths will result. I'll put the price of life at $10 million. I'll also assume that each person loses two weeks of productivity equivalent to a loss of $20 per hour for 80 hours = $1600, and I'll assume a loss of well-being equivalent to $10 per hour for 336 hours = $3360. Finally, I'll assume the costs of isolation are $1,000 per person. Together, this combines to $10M x 0.002 + ($1600 + $3360) x 10 + $1000 x 15 = $84,600.

However, I didn't include the cost of long-covid, which could plausibly raise this estimate radically depending on your beliefs. But personally I'm already a bit skeptical that 15 people would be willing to collectively pay $86,400 to prevent an infection in their house with certainty, so I still feel my initial intuition was mostly justified.

Comment by Matthew Barnett (matthew-barnett) on We need a standard set of community advice for how to financially prepare for AGI · 2021-06-27T19:37:39.432Z · LW · GW

This would only be true if there were an indefinitely scalable way to convert dollars into happiness/suffering-reduction

I don't agree, but I think my argument assumed a different foundation than what you have in mind. Let me try to explain.

Assume for a moment that at some point we are will exist in a world where the probability of existential risk changes negligibly as a result of marginal dollars thrown at attempts to mitigate it. This type of world seems plausible post-AGI, since if we already have superintelligences running around, so to speak, then it seems reasonably likely that we will have already done all we can do about existential risk.

The type of altruism most useful in such a world would probably be something like, producing happy beings (if you're a classical utilitarian, but we can discuss other ethical frameworks too). In that case, the number of dollars you own would scale nearly linearly with the number of happy beings you can produce. Why? Because your total wealth will likely be tiny compared to the global wealth, and so you aren't likely to hit large diminishing returns even if you spend all of your wealth towards that pursuit.

Quick intuition pump: suppose you were a paperclip maximizer in the actual world (this one, in 2021) but you weren't superintelligent, weren't super-rich, and you there was no way you could alter existential risk. What would be the best action to take? Well, one obvious answer would be to use all of your wealth to buy paperclips (and by "all of your wealth" I mean, everything you own, including the expected value of your future labor). Since your wealth is tiny compared to the overall paperclip market, your actions aren't likely to increase the price of paperclips by much, and thus, the number of paperclips you cause to exist will be nearly linear in the number of dollars you own.

ETA: After thinking more, perhaps your objection is that in the future, we will be super-rich, so this analogy does not apply. But I think the main claims remain valid insofar as your total wealth is tiny compared to global wealth. I am not assuming that you are poor in some absolute sense, only that you literally don't control more than say, 0.01% of all wealth.

ETA2: I also noticed that you were probably just arguing that money spent pre-AGI goes further than money spent post-AGI. Seems plausible, so I might have just missed the point. I was just  arguing a claim that inflation adjusted dollars shouldn't have strongly diminishing marginal utility in the future to altruistic ethics systems.

Comment by Matthew Barnett (matthew-barnett) on We need a standard set of community advice for how to financially prepare for AGI · 2021-06-26T23:57:05.159Z · LW · GW

I would add that money is probably much less valuable after AGI than before, indeed practically worthless.

Depending on your system of ethics, there shouldn't be large diminishing returns to real wealth in the future. Of course, personally, if you're a billionaire, then $10,000 doesn't make much of a difference to you, whereas to someone who owns nothing, it could be life-saving.

But in inflation adjusted terms, dollars represent the amount of resources that you control, and stuff that you can produce. For instance, if you care about maximizing happiness, and your utility function is linear in the number of happy beings, then each dollar you have goes just as far whether you are a billionaire or trillionaire. It also makes sense from a perspective of average utilitarianism. From that perspective, what matters most is plausibly what fraction of beings you can cause to be happy, which implies that the fraction of global wealth you control matters immensely.

Comment by Matthew Barnett (matthew-barnett) on We need a standard set of community advice for how to financially prepare for AGI · 2021-06-26T23:47:58.130Z · LW · GW

Because for the intermediate timeframe until the market begins to take the concept seriously, the value of your investments will be determined by all the other factors which you’re ignoring in favour of focusing on AGI, so unless you want your investment results to be meh for years-to-decades, then don‘t go for some all-out bet on AI.

For most people here, the choice is not between choosing AI stocks based on fundamental value, and choosing another set of stocks based on fundamental value. Rather, it's between choosing AI stocks based on their fundamental value, and choosing an index fund. If we assume that whatever short term variables affect AI stocks are just as unpredictable as those that affect index funds, the only real risk to investing in AI stocks is that you might not be diversified enough.

In other words, the main argument against investing in AI stocks (conditional on AGI being a real force later this century), is that you don't want to expose yourself to that much risk. But plenty of people are OK with that level of risk, so I don't see a problem with it.

Comment by Matthew Barnett (matthew-barnett) on What will be the aftermath of the US intelligence lab leak report? · 2021-06-26T23:33:25.193Z · LW · GW

These events make it quite likely that we have a huge crisis of faith in the medical establishment.

I am less confident than you in the lab leak hypothesis, but am even less confident that this development will shake confidence in the medical establishment. People rarely care much when experts get things wrong. At the most, I expect that there will be medium-sized media outlets telling people that the medical establishment is unreliable, and their audience -- the vast majority of whom had already agreed with that thesis -- will nod their heads in agreement. 

I'm unsure about how to operationalize a crisis of faith here, but I'm willing to bet against it.

Comment by Matthew Barnett (matthew-barnett) on What will be the aftermath of the US intelligence lab leak report? · 2021-06-26T23:28:46.171Z · LW · GW

GJOpen community is at 9% that the report will conclude that lab leak is more likely than not, I’m at 12%.

This Metaculus question with a slightly different operationalization, and a longer timeframe, says 39% right now.

Comment by Matthew Barnett (matthew-barnett) on rohinmshah's Shortform · 2021-06-26T23:21:16.939Z · LW · GW

My real question though is: did people actually do this? Did they consider the possibility of a tax, discuss it, realize they couldn't come to an agreement on price, and then implement something else?

Probably not, although they lived in a society in which the response "just use Pigouvian taxes" was not as salient as it otherwise could have been in their minds. This reduced saliency was, I believe, at least partly due to fact that Pigouvian taxes have standard implementation issues. I meant to contribute one of these issues as a partial explanation, rather than respond to your question more directly.

Comment by Matthew Barnett (matthew-barnett) on rohinmshah's Shortform · 2021-06-26T18:33:48.356Z · LW · GW

Part of the issue is that there's rarely a natural way of pricing Pigouvian taxes. You can make price estimates based on how people hypothetically judge the harm to themselves, but there's always going to be huge disagreements. 

This flaw is a reasonable cause for concern. Suppose you were in a group house where half of the people worked remotely and the other half did not. The people who worked remotely might be biased (at least rhetorically) towards the proposition that the Pigouvian tax should be high, and the people who work in-person might be biased in the other direction. Why? Because if someone doesn't expect to have to pay the tax, but does expect to receive the revenue, they may be inclined to overestimate the harm of COVID-19, as a way of benefiting from the tax, and vice versa.

In regards to carbon taxes, it's often true that policies sound like the "obvious" thing to do, but actually have major implementation flaws upon closer examination. This can help explain why societies don't do it, even if it seems rational. Noah Smith outlines the case against a carbon tax here,

This isn’t just politics; economists have forgotten basic Econ 101. Voters instinctively know what economists, for some mystifying reason, have seemed to ignore — the people who pay the costs of a carbon tax don’t reap the benefits. Carbon taxes are enacted locally, but climate change is a global phenomenon. That means that if Washington state taxes carbon, its own residents pay, but most of the benefit is reaped by people in other countries and other states. Thus, jurisdictions that choose not to enact carbon taxes can simply hope that someone else shoulders the cost of combating climate change. So no one ends up paying the cost.

Of course, this argument shouldn't stop a perfectly altruistic community from implementing a carbon tax. But if the community was perfectly altruistic, the carbon tax would be unnecessary.

Comment by Matthew Barnett (matthew-barnett) on ELI12: how do libertarians want wages to work? · 2021-06-24T23:52:34.161Z · LW · GW

My question asks for a combination of reasoning like in those theorems, with empirical information about parameters like bounded reasoning, imperfect information, mortal agents, agents with complicated boundaries, etc. AFAIK in those conditions, there could be weird attractors that mean dumb-seeming stuff like government actually makes sense.

That seems like a tall order. Instead of making very general assumptions about agents (which limits the usefulness of mathematical results), it would probably make more sense in this context to talk about empirical analysis. We can ask questions like, do societies that use democratic governments to solve certain types of coordination problems X, generally do better across some metric Y? Economists have been studying problems like this for centuries.

I'm pretty confident that we could describe a coherent world that's vastly superior just by changing the (hypothetical) government away from our actual one.

Sure. The field of mechanism design tries to build hypothetical economic systems that would work better than the ones we currently have.

Comment by Matthew Barnett (matthew-barnett) on ELI12: how do libertarians want wages to work? · 2021-06-24T20:15:47.663Z · LW · GW

It seems like unions and minimum wages are a form of bargaining, using the government as a way to overcome coordination problems given the cognitive costs of doing distributed game theory. Why isn't that considered by libertarians to be a legitimate (i.e., reasonably endorsed as part of a globally coherent notion of justice) form of bargaining / decision-making?

As for myself, the question is not "Is this a legitimate coordination mechanism?" but rather "Does this coordination mechanism effectively solve coordination problems?" In other words, rather than viewing government as some sort of privileged system that solves all the problems in the private market, we can view it from the same perspective that we view systems in the private market. We can do this through the lens of game theory: model the government as being composed of a bunch of different economic agents who each try to achieve their own goals in light of the incentive structures in government, and then ask whether the aggregate behavior of these agents adds up to something we prefer to alternatives.

When we start getting specific about what we mean by governance, then the flaws in using government as a device for collective action become more apparent. The flaws in first past the post representative democracy are fairly well known, but I'll repeat some arguments here:

  1. Given typical election margins, the expected selfish value of voting is very small. Therefore, voters have very little incentive to be informed about which candidates might benefit them.
  2. Candidate media coverage is a very powerful lever for getting politicians elected. Therefore, rich special interests can sway elections to their desired ends by paying for targeted media coverage.
  3. If a candidate wins by even a single vote, they win 100% of the effective power of the office for their term. Therefore, minority interests are given almost no weight during democratic deliberations, unless those interests could plausibly lead to the incumbent losing a future election (for example, by people in the majority being sympathetic to minority interests).
  4. First past the post voting doesn't take into account the intensity of preferences. For example, the weak interests of 9 voters who are OK with locking up marijuana smokers outweigh the strong interest of the 1 voter who doesn't want to be locked up themselves.

Given these flaws in the way democracy actually functions (as opposed to the way people poetically imagine democracy), there's no guarantee that coordinating through it will actually lead to outcomes that maximize utility, like happiness or preference satisfaction. By contrast, libertarians generally share the intuition that market structures usually do a better job at solving coordination problems than government (and sometimes back it up with results like the fundamental theorems of welfare economics).

Comment by Matthew Barnett (matthew-barnett) on The Point of Trade · 2021-06-23T20:59:33.435Z · LW · GW

Part of the confusion here is over what the word "trade" means. I'm actually pretty sympathetic to the proto-justification given that, "The point of trade is that sometimes people get different amounts of value from things, so they can get more value by trading them." It's just that, even in a society where everyone shares the same values, we get efficiency from specialization.

It seems like a further stipulation to refer to "specialization and transportation of resources" as "trade" if every party involved has the exact same values. Consider a colony of ants. There are various types of worker ants, and the queen ant, and each uses special behavioral tools to optimize their particular niche within the colony. Still, it would seem weird to say that worker ants are "trading" with the queen ant, given that it makes a lot of sense to view the colony as a single unit, working towards a common aim.

Comment by Matthew Barnett (matthew-barnett) on Matthew Barnett's Shortform · 2021-06-19T06:14:32.682Z · LW · GW

It's now been about two years since I started seriously blogging. Most of my posts are on Lesswrong, and the most of the rest are scattered about on my substack and the Effective Altruist Forum, or on Facebook. I like writing, but I have an impediment which I feel impedes me greatly.

In short: I often post garbage.

Sometimes when I post garbage, it isn't until way later that I learn that it was garbage. And when that happens, it's not that bad, because at least I grew as a person since then.

But the usual case is that I realize that it's garbage right after I'm done posting it, and then I keep thinking, "oh no, what have I done!" as the replies roll in, explaining to me that it's garbage.

Most times when this happens, I just delete the post. I feel bad when this happens because I generally spend a lot of time writing and reviewing the posts. Some of the time, I don't delete the post because I still stand by the main thesis, although the delivery or logical chain of reasoning was not very good and so I still feel bad about it.

I'm curious how other writers deal with this problem. I'm aware of "just stop caring" and "review your posts more." But, I'm sometimes in awe of some people who seem to consistently never post garbage, and so maybe they're doing something right that can be learned.

Comment by Matthew Barnett (matthew-barnett) on AI-Based Code Generation Using GPT-J-6B · 2021-06-17T04:16:26.304Z · LW · GW

Have you tried, y'know, testing your belief? ;)

You can Google its answers. I've been googling its answers and am not generally finding direct copy-pastes for each, though I'm also a bit confused about why Google is obtaining no results for short strings such as "(s[0:length] == s[length::-1])".

ETA: even if it's copying code but modifying it slightly so that the variable names match, it seems like (1) this is itself pretty impressive if it actually works reliably, and (2) I don't think the claim is that current tech is literally shovel ready to replace programmers. That would be a strawman. It's about noticing the potential of this tech before it reaches its destination.

Comment by Matthew Barnett (matthew-barnett) on AI-Based Code Generation Using GPT-J-6B · 2021-06-17T04:05:33.230Z · LW · GW

I no longer consider agents with superhuman performance in competitive programming to be a ridiculous thing to pursue. 

Dan Hendrycks and Steven Basart et al. recently released APPS, an ML benchmark for measuring the performance of ML models at the task of writing code. One part of their benchmark measures the performance of code on competitive programming questions. I wrote a Metaculus question on when people expect this benchmark to be solved -- operationalized as getting above 80% strict accuracy on the competitive programming section.

Initial results are encouraging. GPT-Neo 2.7B passes nearly 20% of test cases on average for introductory coding problems, when the model is allowed to give 5 attempts (see Table 4 in the paper). A fine-tuned GPT-J-6B is likely to be even better.

Comment by Matthew Barnett (matthew-barnett) on Three reasons to expect long AI timelines · 2021-04-24T18:39:02.023Z · LW · GW

Wow, that chart definitely surprised me. Yes, this caused me to update.

Comment by Matthew Barnett (matthew-barnett) on Three reasons to expect long AI timelines · 2021-04-24T18:35:38.915Z · LW · GW

Nuclear power is not 10x cheaper.  It carries large risks so some regulation cannot be skipped.  I concur that there is some unnecessary regulation, but the evidence such as the linked source just doesn't leave "room" for a 10x gain.  Currently the data suggests it doesn't provide an economic gain over natural gas unless the carbon emissions are priced in, and they are not in most countries.

I recommend reading the Roots of Progress article I linked to in the post. Most of the reason why nuclear power is high cost is because of the burdensome regulations. And of course, regulation is not uniformly bad, but it seems from the chart Devanney Figure 7.11 in the article that we could have relatively safe nuclear energy for a fraction of its current price.

Comment by Matthew Barnett (matthew-barnett) on Three reasons to expect long AI timelines · 2021-04-23T20:16:13.249Z · LW · GW

Thanks for the useful comment.

You might say "okay, sure, at some level of scaling GPTs learn enough general reasoning that they can manage a corporation, but there's no reason to believe it's near".

Right. This is essentially the same way we might reply to Claude Shannon if he said that some level of brute-force search would solve the problem of natural language translation.

one of the major points of the bio anchors framework is to give a reasonable answer to the question of "at what level of scaling might this work", so I don't think you can argue that current forecasts are ignoring (2).

Figuring out how to make a model manage a corporation involves a lot more than scaling a model until it has the requisite general intelligence to do it in principle if its motivation were aligned.

I think it will be hard to figure out how to actually make models do stuff we want. Insofar as this is simply a restatement of the alignment problem, I think this assumption will be fairly uncontroversial around here. Yet, it's also a reason to assume that we won't simply obtain transformative models the moment they become theoretically attainable.

It might seem unfair that I'm inputting safety and control as an input in our model for timelines, if we're using the model to reason about the optimal time to intervene. But I think on an individual level it makes sense to just try to forecast what will actually happen.

Comment by Matthew Barnett (matthew-barnett) on Three reasons to expect long AI timelines · 2021-04-23T07:08:53.804Z · LW · GW

These arguments prove too much; you could apply them to pretty much any technology (e.g. self-driving cars, 3D printing, reusable rockets, smart phones, VR headsets...).

I suppose my argument has an implicit, "current forecasts are not taking these arguments into account." If people actually were taking my arguments into account, and still concluding that we should have short timelines, then this would make sense. But, I made these arguments because I haven't seen people talk about these considerations much. For example, I deliberately avoided the argument that according to the outside view, timelines might be expected to be long, since that's an argument I've already seen many people make, and therefore we can expect a lot of people to take it into account when they make forecasts.

I agree that the things you say push in the direction of longer timelines, but there are other arguments one could make that push in the direction of shorter timelines

Sure. I think my post is akin to someone arguing for a scientific theory. I'm just contributing some evidence in favor of the theory, not conducting a full analysis for and against it. Others can point to evidence against it, and overall we'll just have to sum over all these considerations to arrive at our answer.