Posts

Conditional prediction markets are evidential, not causal 2024-02-07T21:52:47.476Z
London rationalish meetup: Feedbackloop-first rationality 2024-02-04T23:47:14.899Z
When and why should you use the Kelly criterion? 2023-11-05T23:26:38.952Z
Headphones hook 2023-09-29T22:50:04.962Z
London rationalish meetup - Thinking Physics 2023-09-03T22:47:05.391Z
Ruining an expected-log-money maximizer 2023-08-20T21:20:02.817Z
Three configurable prettyprinters 2023-08-10T23:10:02.000Z
How tall is the Shard, really? 2023-06-23T08:10:02.124Z
London rationalish meetup - relinquishing curiosity 2023-05-16T21:42:12.487Z
Does descaling a kettle help? Theory and practice 2023-05-02T20:20:04.401Z
London rationalish meetup 2023-02-08T21:17:36.417Z
Life Has a Cruel Symmetry 2023-01-23T23:40:04.161Z
When to mention irrelevant accusations? 2023-01-14T21:58:08.015Z
Smallpox Eradication Day Unconference 2022-12-02T19:25:05.532Z
On Kelly and altruism 2022-11-24T23:40:04.095Z
A list of Petrov buttons 2022-10-26T20:50:04.025Z
London Rationalish meetup 2022-10-06T23:19:30.042Z
London Rationalish meetup 2022-09-10T11:18:45.700Z
London Rationalish meetup 2022-08-07T23:00:54.924Z
PD-alikes in two dimensions 2022-04-23T09:52:38.765Z
Variadic functions in Hindley Milner 2022-04-02T20:00:03.972Z
London rationalish meta-meetup 2022-03-10T23:04:00.845Z
London Rationalish meetup 2022-02-11T22:22:21.160Z
Walkthrough: Filing a UK self-assessment tax return 2021-12-29T23:10:03.764Z
Ten Hundred Megaseconds 2021-09-20T21:30:05.464Z
Against "blankfaces" 2021-08-08T23:00:04.126Z
Book Review: Order Without Law 2021-07-10T21:50:03.589Z
99% shorter 2021-05-27T19:50:03.532Z
philh's Shortform 2021-04-22T20:50:46.556Z
A command-line grammar of graphics 2021-03-30T20:30:03.071Z
Haskenthetical update: user-defined macros 2021-03-14T21:20:04.108Z
Specialized Labor and Counterfactual Compensation 2020-11-14T18:13:43.044Z
Against boots theory 2020-09-14T13:20:04.056Z
Classifying games like the Prisoner's Dilemma 2020-07-04T17:10:01.965Z
Short essays on various things I've watched 2020-06-12T22:50:01.957Z
Haskenthetical 2020-05-19T22:00:02.014Z
Chris Masterjohn on Coronavirus, Part 2 2020-04-28T21:50:01.430Z
In my culture: the responsibilities of open source maintainers 2020-04-13T13:40:01.174Z
Chris Masterjohn on Coronavirus, Part 1 2020-03-29T11:00:00.819Z
My Bet Log 2020-03-19T21:10:00.929Z
Tapping Out In Two 2019-12-05T23:10:00.935Z
The history of smallpox and the origins of vaccines 2019-12-01T20:51:29.618Z
The Effect pattern: Transparent updates in Elm 2019-10-20T20:00:01.101Z
London Rationalish meetup (part of SSC meetups everywhere) 2019-09-12T20:32:52.306Z
Is this info on zinc lozenges accurate? 2019-07-27T22:05:11.318Z
A reckless introduction to Hindley-Milner type inference 2019-05-05T14:00:00.862Z
"Now here's why I'm punching you..." 2018-10-16T21:30:01.723Z
Pareto improvements are rarer than they seem 2018-01-27T22:23:24.206Z
2017-10-08 - London Rationalish meetup 2017-10-04T14:46:50.514Z
Authenticity vs. factual accuracy 2016-11-10T22:24:38.810Z

Comments

Comment by philh on philh's Shortform · 2024-03-08T23:41:24.909Z · LW · GW

Oh, huh. Searle's original Chinese room paper (first eight pages) doesn't say machines can't think.

"OK, but could a digital computer think?"

If by "digital computer" we mean anything at all that has a level of description where it can correctly be described as the instantiation of a computer program, then again the answer is, of course, yes, since we are the instantiations of any number of computer programs, and we can think.

"But could something think, understand, and so on solely in virtue of being a computer with the right sort of program? Could instantiating a program, the right program of course, by itself be a sufficient condition of understanding?"

This I think is the right question to ask, though it is usually confused with one or more of the earlier questions, and the answer to it is no.

"Why not?"

Because the formal symbol manipulations by themselves don't have any intentionality; they are quite meaningless; they aren't even symbol manipulations, since the symbols don't symbolize anything. In the linguistic jargon, they have only a syntax but no semantics. Such intentionality as computers appear to have is solely in the minds of those who program them and those who use them, those who send in the input and those who interpret the output.

I can't say I really understand what he's trying to say, but it's different from what I thought it was.

Comment by philh on The Gemini Incident · 2024-02-27T00:33:51.554Z · LW · GW

Yeah. It's still possible to program in such a way that that works, and it's always been possible to program in such a way that it doesn't work. But prepared statements make it easier to program in such a way that it doesn't work, by allowing the programmer to pass executable code (which is probably directly embedded as a literal in their application language) separately from the parameters (which may be user-supplied).

(I could imagine a SQL implementation forbidding all strings directly embedded in queries, and requiring them to be passed through prepared statements or a similar mechanism. That still wouldn't make these attacks outright impossible, but it would be an added layer of security.)

Comment by philh on Monthly Roundup #15: February 2024 · 2024-02-27T00:15:21.940Z · LW · GW

A large majority of empirical evidence reported in leading economics journals is potentially misleading. Results reported to be statistically significant are about as likely to be misleading as not (falsely positive) and statistically nonsignificant results are much more likely to be misleading (falsely negative). We also compare observational to experimental research and find that the quality of experimental economic evidence is notably higher.

I'm confused by this "falsely negative". Like, without that, that part sounds like it's saying something like

when a result is reported as "we observed a small effect here, but it wasn't statistically significant", then more often than not, there's no real effect there

but that's a false positive. If they're saying it's a false negative, it suggests something like

when a result is reported as statistically insignificant, that makes it sound like there's no effect there, but more often than not there actually is an effect

...but that's (a) not a natural reading of that part and (b) surely not true.

Comment by philh on The Gemini Incident · 2024-02-26T23:43:37.952Z · LW · GW

Were SQL a better language this wouldn’t be possible, all the command strings would separated somehow

SQL does support prepared statements which forbid injection. Maybe you're thinking of something stronger than this? I'm not sure how long they've been around for, but wikipedia's list of SQL injection examples only has two since 2015 which hints that SQL injection is much less common than it used to be.

(Pedantic clarification: dunno if this is in any SQL standard, but it looks like every SQL implementation I can think of supports them.)

Comment by philh on philh's Shortform · 2024-02-19T23:56:18.532Z · LW · GW

Planet Money #902 (28 Mar 2019): The Phoebus Cartel

Listened to this one a few weeks ago and don't remember most of it. But half the episode was about the phoebus cartel, a case of planned obsolesence when lightbulb manufacturers decided that no light bulb should be allowed to last more than 1000 hours.

Writing this for Gell-Mann amnesia reasons: in the episode someone says there was no benefit to consumers from this, but I'd recently seen a technology connections episode on the subject saying that longer lasting incandescent light bulbs are less energy efficient (i.e. more heat less light) for physics reasons, to the extent that they could easily be more expensive over their lifetime. Seems like an important caveat that PM missed!

The other half was about psychological obsolesence, where manufacturers make long-lasting goods like cars cosmetically different to convince you you need a new one.

Comment by philh on philh's Shortform · 2024-02-19T23:48:04.288Z · LW · GW

Planet Money #1717 (9 Feb 2024): A Lawsuit for your broken heart

Keith met woman, fell in love, got married, had kids. She helped with his BMX company and she'd post sickeningly cute things on facebook about how she had the best family.

Then Keith saw some very messages she was exchanging with some other guy (from him: «do you like how tall I am», «show me a bikini pic», that kind of thing). He got mad, called him, said «never fucking talk to my wife again» and thought that would be the end of it.

It was not the end of it. She had affair, they got divorced. A bit later he was catching up with an old school friend who'd been in a similar situation, and she told him she was suing the woman her husband had cheated with. You can do that?

These are heartbalm laws and they're kind of archaic. In the past if a woman got engaged and the man broke things off, she could be ruined, so she got to sue him for breach of promise. There's also seduction, where she could sue someone for lying her into bed. And criminal conversation, which is adultery. And the one relevant to the show, alienation of affections, where you can sue someone for damaging your marriage.

Most states have abolished these, partly because public perception moved towards women using these in ways that were unpopular, this is where the term "gold digger" took off. There were also a bunch of famous people who got sued.

But a few states still have alienation of affections, including North Carolina, which is where most of the suits are. Possibly because that's where most of the legal experts in them are.

Keith presents evidence that his marriage would otherwise have been happy: the sickeningly cute facebook posts, messages between him and his ex, messages from her to her girlfriends saying the marriage would have been fine if not for this other guy. (She subsequently married him.)

And because marriage is in part an economic arrangement, his lawyer also talks about the work that the ex had been doing for the company, and all the unpaid labor she was doing like childcare and washing dishes. The hosts point out it's kinda weird that Keith is suing some other guy for the unpaid labor his ex wife used to do. But that's what's happening, and Keith wins the suit and is awarded $8 million.

Other guy files for bankruptcy. Keith probably won't get anything from him, and still owes his lawyers thousands of dollars in fees. But he says it was worth it.

Comment by philh on Epistemic Hell · 2024-02-13T00:23:57.425Z · LW · GW

Nice!

Wikipedia says his mission began on 08/29/1498 and ended on 01/07/1499 (so about 3 months).

It looks like this is just one leg of the return journey. In total the outward journey was about 10 months and the return was about 11, and both spent 3+ months without landing.

Comment by philh on Conditional prediction markets are evidential, not causal · 2024-02-09T23:44:39.233Z · LW · GW

Hm, do you want to go into more depth? Intuitively I agree this is obviously distortionary, but I'm finding it awkward to think through the details of the distortion.

One thing that comes to mind is "if the market is at 10% but you think 5% is "correct" according to what seems like the spirit of the question, you're going to expect that the market just doesn't get resolved, so why bother betting it down". But I feel like there's probably more than that. (E: oh, the dynomight essay linked above mentions this one as well.)

Comment by philh on Conditional prediction markets are evidential, not causal · 2024-02-07T23:48:37.680Z · LW · GW

Thanks! Yeah, I think that's making the same basic point with a different focus.

And that makes me more confident in changing the title, so doing that now. (Original title: "Conditional prediction markets are, specifically, conditioned on the thing happening".)

Comment by philh on Conditional prediction markets are evidential, not causal · 2024-02-07T22:13:50.385Z · LW · GW

Maybe a pithier title would be "Conditional prediction markets are evidential, not causal"?

Comment by philh on Epistemic Hell · 2024-02-05T18:50:06.128Z · LW · GW

The problem was so common that shipowners and governments assumed a 50% death rate from scurvy for their sailors on any major voyage.

Sticking my neck out: roll to disbelieve that 50% of sailors on major voyages died in general, let alone specifically of scurvy.

Ways to change the claim that I'd find much more believable:

  • 50% of those who got scurvy died of it
  • A 50% death rate was considered plausible, and the possibility was planned for, but it wasn't common
  • "Major voyage" here is a much smaller category than I expect; think Magellan rather than Columbus.
Comment by philh on Gender Exploration · 2024-01-26T01:34:48.153Z · LW · GW

Note that in general there's no contradiction between

"X is a very not-Y trait, in fact just about the least Y trait there is"

And

"Nevertheless, one can be X and also overall very Y"

Comment by philh on philh's Shortform · 2024-01-21T20:01:07.456Z · LW · GW

Planet Money: The Maine Potato War

The standard potato today is the russet, mostly associated with Idaho and Washington, but in the... 70s? 80s?... it was some other kind from Maine.

These potatos were sold on the New York Mercantile Exchange, and in particular you could buy and sell futures there. Futures are good for hedging, farmers could lock in the price early to avoid risk of a crash, and buyers could lock it in early to avoid risk of a rise. You could also just speculate, with no intention of ever seeing a potato. Speculators are generally good for markets because they put a bunch of money in there which helps them flow better.

The big Western potato farmers didn't like that Maine potatos were on the exchange, because it gave them an advantage. So one year they entered the market with like a million dollars worth of contracts. (I vaguely recall that these were buy contracts, but I think it fits better with the rest of the story if they were sell.) Trading is weird for the year, and when things finally play out after close date, the Western farmers collectively owe the market millions of pounds of Maine potatoes, possibly more than existed in the entire state of Maine.

They try to get around it by offering russet potatoes instead, but buyers say no dice. So in the end they just default. They have to pay back the buyers plus pay heavy fines. I think banned from the exchange for a bit? Short term expensive for them.

But long term very good for them! Markets don't like defaults, so the Maine potatoes are removed from the exchange a bit later.

Comment by philh on If Clarity Seems Like Death to Them · 2024-01-19T22:30:52.402Z · LW · GW

I think he is using the argument as a soldier.

I see. That's not a sense I pick up on myself, but I suppose it's not worth litigating.

To be clear, skimming my previous posts, I don't see anything that I don't endorse when it comes to literary criticism. Like, if I've said something that you agree with most of the time, but disagree with for literary criticism, then we likely disagree. (Though of course there may be subtleties e.g. in the way that I think something applies when the topic is literary criticism.)

You mention that “awesome” and “terrible” are very subjective words, unlike “blue”, and this is relevant. I agree. Similarly, media criticism is very subjective, unlike dress colors.

Media criticism can be very subjective, but it doesn't have to be. "I love Star Wars" is more subjective than "Star Wars is great" is more subjective than "Star Wars is a technical masterpiece of the art of filmmaking" is more subjective than "Star Wars is a book about a young boy who goes to wizard school". And as I said above:

I’m comfortable with “Luke is a Jedi”, and I think it’s importantly different from, say, “Yoda is wise” or “the Death Star is indestructible” or “the Emperor has been defeated once and for all”.

And I think the ways it’s different are similar to the differences between claims about base-level reality like “Tim Cook is a CEO” versus “the Dalai Lama is wise” or “the Titanic is unsinkable” or “Napoleon has been defeated once and for all”.

Comment by philh on philh's Shortform · 2024-01-18T18:07:13.828Z · LW · GW

Planet money #904 (6 Apr 2019): Joke Theft

Meg is an Internet comedian. After a picture of Kanye kissing Kim at some ceremony, she photoshopped it to be a pic of Kanye kissing Kanye. It went viral. But her own tweet only got a couple hundred likes. Most of the viral came from an Instagram account called Fuck Jerry which has 14 million followers and reposted it without credit.

And Fuck Jerry does that a lot, and gets a lot of ad money, including from Comedy Central who should know better than to support someone who keeps stealing from comedians. So that made Meg kinda mad.

Stand up comedy started after the death of vaudeville. At first it was just throwing out one-liner after one-liner, the jokes didn't have much effort put into them so no one really cared if you stole them. Later the acts and the jokes got more sophisticated and comedians did care.

You can copyright a joke, but it costs $35 to register, it can't be too short, and just changing the wording gets around it. This is partly to stop stuff like "don't you hate it when..." from being taken out of the commons. A comedy lawyer couldn't find any case of one stand up comedian suing another for joke theft.

What do they do instead? A comedian gives three strategies.

  • Violence: every time you steal one of my jokes, I'll damage your car.
  • Warnings: if you as a friend of the comic see a known joke thief in the audience, write a message on a napkin and have the waitress deliver it to the comic who can then finish early or avoid using their more precious material. (Robin Williams was a known joke thief. He said he just absorbed stuff and couldn't remember where it was from. Comedian says Robin once stole one from him, but at least when he called him out he cut him a cheque. It said "sorry for the inconvenience", not "sorry for stealing". But it was $200 which was a lot of money in the mid 80s.)
  • Organizing: get venues to blacklist them.

Meg makes a campaign "fuck Fuck Jerry" trying to get people to unfollow. A bunch of famous comedians join in (I think Amy Schumer was mentioned). It works at least a bit, they lose 300,000 followers. That doesn't sound like much, maybe there was more?

PM speaks to Elliot who is behind Fuck Jerry. He considers himself a curator, not thief. He wishes people would know he's not a bad person. PM is not very sympathetic. He says he now asks for permission; Meg says that's not enough. He says he's thinking of starting to pay; PM says it's unclear how that would work but it's a bold thing to put on the air.

Comment by philh on If Clarity Seems Like Death to Them · 2024-01-16T11:22:02.763Z · LW · GW

Thanks for replying. I’m going to leave aside non-fictional examples (“The Dress”) because I intended to discuss literary criticism.

So uh. Fair enough but I don't think anything else in your comment hinged on examples being drawn from literary criticism rather than reality? And I like the dress as an example a lot, so I think I'm gonna keep using it.

I’m not sure exactly what you mean, see Taboo “Outside View”. My best guess is that you mean that “X seems Y to me” implies my independent impression, not deferring to the views of others, whereas “X is Y” doesn’t.

From a quick skim, I'd say many of the things in both the inside-view and outside-view lists there could fit. Like if I say "the dress looks white to me but I think it's actually blue", some ways this could fit inside/outside view:

  • Inside is one model available to me (visual appearance), outside is all-things-considered (wikipedia).
  • Inside is my personal guess, outside is taking a poll (most people think it's blue, they're probably right).
  • Inside is my initial guess, outside is reference class forecasting (I have a weird visual processing bug and most things that look white to me turn out to be blue).

If so, I don’t think I am missing this.

I don't really know how to reply to this, because it seems to me that you listed "acknowledging or changing social reality", I said "I think you're missing inside versus outside view", and you're saying "I don't think I am missing that" and elaborating on the social reality thing. I claim the two are different, and if they seem the same to you, I don't really know where to proceed from there.

Again, I don’t think I am missing this. I agree that “X seems Y to me” implies something like a gut reaction or a hot take. I think this is because “X seems Y to me” expresses lower confidence than “X is Y”, and someone reporting a gut reaction or a hot take would have lower confidence than someone who has studied the text at length and sought input from other authorities.

I think you have causality backwards here. I'd buy "it seems low confidence because it suggests a gut reaction" (though I'm not gonna rule out that there's more going on). I don't buy "it suggests a gut reaction because it seems low confidence".

So I claim the gut-reaction thing is more specific than the low-confidence thing.

Well, that isn’t his stated goal.

Right. Very loosely speaking, Eliezer said to do it because it was kind to authors; Zack objected because it was opposed to truth; I replied that in fact it's pro-truth. (And as you point out, Eliezer had already explained that it's pro-truth, differently but compatibly with my own explanation.)

Yudkowsky doesn’t advise critics to say: “mileage varied, I thought character X seemed clever to me”, he doesn’t say “please don’t tell me what good things the author was thinking unless the author plainly came out and said so”.

Well, I can't speak for Eliezer, and what Eliezer thinks is less important than what's true. For myself, I think both of those would be good advice for the purpose of saying true and informative things; neutral advice for the purpose of being kind to authors.

Given the one-sided application of the advice, I don’t take it very seriously.

I'm not sure what you mean by not taking it very seriously.

Applying a rule in one situation is either good advice for some purpose, or it's not. Applying a rule in another situation is either good advice for some purpose, or it's not. If someone advises applying the rule in one situation, and says nothing about another situation... so what?

My vague sense here is that you think he has hidden motives? Like "the fact that he advises it in this situation and not that situation tells us... something"? But:

  • I don't think his motives are hidden. He's pretty explicitly talking about how to be kind to authors, and the rule helps that purpose more in one situation than another.
  • You can just decide for yourself what your purposes are and whether it's good advice for them in any given situation. If he makes arguments that are only relevant to purposes you don't share, you can ignore them. If he makes bad arguments you can point them out and/or ignore them. If he makes good arguments that generalize further than he takes them, in ways that you endorse but you think he wouldn't, you can follow the generalization anyway.

I claim that this text would not be more true and informative with “mileage varies, I think x seems y to me”. What do you think?

Eliezer described it as his opinion before saying it, and to me that does the same work.

If it weren't flagged as opinion, then yes, I think a "seems" or "to me" or something would make it slightly more true and informative. Not loads in this case - "awesome" and "terrible" are already very subjective words, unlike "blue" or "indestructible".


This feels like the type of conversation that takes a lot of time and doesn't help anyone much. So after this I'm gonna try to limit myself to two more effortful replies to you in this thread.

Comment by philh on If Clarity Seems Like Death to Them · 2024-01-16T09:20:20.267Z · LW · GW

Fair enough! I did indeed miss that.

Comment by philh on If Clarity Seems Like Death to Them · 2024-01-15T18:43:45.708Z · LW · GW

Hm, I think I'm maybe somewhat equivocating between "the dress looks blue to me" as a statement about my state of mind and as a statement about the dress.

Like I think this distinction could be unpacked and it would be fine, I'd still endorse what I'm getting at above. But I haven't unpacked it as much as would be good.

Comment by philh on If Clarity Seems Like Death to Them · 2024-01-15T13:50:07.538Z · LW · GW

Two differences I think you're missing:

  • "seems to me" suggests inside view, "is" suggests outside view.
  • "seems to me" gestures vaguely at my model, "is" doesn't. This is clearer with the dress; if I think it's blue, "it looks blue to me" tells you why I think that, while "it's blue" doesn't distinguish between "I looked at the photo" and "I read about it on wikipedia and apparently someone tracked down the original dress and it was blue". With "X seemed stupid to me", it's a vaguer gesture, but I think something like "this was my gut reaction, maybe I thought about it for a few minutes". (If someone has spoken with the author and the author agrees "oops yeah that was stupid of X, they should instead have...", then "X was stupid" seems a lot more justifiable to me.)

In the specific case of responses to fiction there is no base reality, so we can’t write “x is y” and mean it literally. All these things are about how the fictional character seems. Still, I would write “Luke is a Jedi” not “Luke seems to be a Jedi”.

Eh... so I don't claim to fully understand what's going on when we talk about fictional universes. But still, I'm comfortable with "Luke is a Jedi", and I think it's importantly different from, say, "Yoda is wise" or "the Death Star is indestructible" or "the Emperor has been defeated once and for all".

And I think the ways it's different are similar to the differences between claims about base-level reality like "Tim Cook is a CEO" versus "the Dalai Lama is wise" or "the Titanic is unsinkable" or "Napoleon has been defeated once and for all".

Comment by philh on If Clarity Seems Like Death to Them · 2024-01-15T13:10:58.235Z · LW · GW

Nod, in that hypothetical I think you would have done nothing wrong.

I think the "obviously" is still false. Or, I guess there are four ways we might read this:

  1. "It is obvious to me, and should be obvious to you, that in general, talking about my own research interests does not violate these norms": I disagree, in general it can violate them.

  2. "It is obvious to me, but not necessarily to you, that in general...": I disagree for the same reason.

  3. "It is obvious to me, and should be obvious to you, that in this specific case, talking about my own research interests does not violate these norms": it's not obvious to the reader based on the information presented in the post.

  4. "It is obvious to me, but not necessarily to you, that in this specific case...": okay sure.

To me (1) is the most natural and (4) is the least natural reading, but I suppose you might have meant (4).

...not that this particularly matters. But it does seem to me like an example of you failing to track the distinction between what-is and what-seems-to-you, relevant to our other thread here.

Comment by philh on If Clarity Seems Like Death to Them · 2024-01-15T11:49:14.993Z · LW · GW

You probably don’t care about how it seems to me; you care about how it is.

Indeed, and as I argued above, a person who reliably tracks the distinction between what-is and what-seems-to-them tells me more about what-is than a person who doesn't.

I mean, I suppose that if someone happened to know that the dress was blue, and told me "the dress looks white to me" without saying "...but it's actually blue", that would be misleading on the subject of the color of the dress. But I think less misleading, and a less common failure mode, than a person who doesn't know that the dress is blue, who tells me "the dress is white" because that's how it looks to them.

I mean, in the specific case of the colors of objects in photographs, I think correspondence between what-is and what-seems is sufficiently high not to worry about it most of the time. The dress was famous in part because it's unusual. If you know that different people see the dress as different colors, and you don't know what's going on, then (according to me and, I claim, according to sensible rationalist discourse norms) you should say "it looks white to me" rather than "it's white". But if you have no reason to think there's anything unusual about this particular photograph of a dress that looks white to you, then whatever.

But I think this correspondence is significantly lower between "X was stupid" and "X seemed stupid". And so in this case, it seems to me that being careful to make the distinction:

  • Makes you better at saying true things;
  • Increases the information content of your words, on both the subjects what-is and what-seems-to-you;
  • Is kinder to authors.
Comment by philh on Dating Roundup #2: If At First You Don’t Succeed · 2024-01-10T14:07:19.823Z · LW · GW

Alyssa Vance goes into an epic, truly epic, amount of detail to debunk the implications of the ‘sexless epidemic’ and of this famous graph in particular

Link seems to be missing?

Comment by philh on Dating Roundup #2: If At First You Don’t Succeed · 2024-01-09T19:22:40.276Z · LW · GW

as the (let’s face it bad, but watchable if you don’t care, and worth watching if you need to learn this and other important related lessons) movie Ghosted illustrates

Note, wikipedia lists three movies named Ghosted. I guess you probably mean the 2023 one?

Comment by philh on If Clarity Seems Like Death to Them · 2024-01-07T16:47:17.365Z · LW · GW

But ... "I thought X seemed Y to me"[20] and "X is Y" do not mean the same thing!

And it seems to me that in the type of comment Eliezer's referring to, "X seemed stupid to me" is more often correct than "X was stupid".

Argument for this: it's unlikely that someone would say "X seemed stupid to me" if X actually didn't seem stupid to them, so it's almost always true when said; whereas I think it's quite common to misjudge whether X was actually stupid.

("X was stupid, they should have just used the grabthar device." / "Did you miss the part three chapters back [published eight months ago] where that got disabled?")

So we might expect that "more often true ⇒ less information content". We could rewrite "X was stupid" to "this story contained the letter E" and that would more often be true, too. But I don't think that holds, because

  • "X seemed stupid" is not almost-always true, unlike "this story contained the letter E";
  • But if someone said "X was stupid" I think it's almost-always also the case that X seemed stupid to them;
  • And in fact people don't reliably track this distinction.

I think people track it more than zero, to be clear. But if I see someone say "X was stupid", two prominent hypotheses are:

  1. This person reliably tracks the distinction between "X was stupid" and "X seemed stupid", and in this case they have sufficient confidence to make the stronger claim.
  2. This person does not reliably track that distinction.

And even on LessWrong, (2) is sufficiently common that in practice I often just rewrite the was-claim to the seemed-claim in my head.

(Actually, I think I'm imperfect at this. I think as a rule of thumb, the "was" claim updates me further than is warranted in the direction that X was stupid. My guess is that this kind of failure is pretty common. But that's separate from a claim about information content of people's words.)

So I think Eliezer is giving good advice for "how to be good at saying true and informative things", as well as good advice for "how to discuss an author's work in a way that leaves them motivated to keep writing".

Comment by philh on If Clarity Seems Like Death to Them · 2024-01-07T14:50:49.851Z · LW · GW

I have an inalienable right to talk about my own research interests, and talking about my own research interests obviously doesn't violate any norm against leaking private information about someone else's family, or criticizing someone else's parenting decisions.

I think you're violating a norm against criticizing someone's parenting decisions, to the extent that readers know whose decisions they are. I happen to know the answer, and I guess a significant number but far from a majority of readers also know. Which also means the parent or parents in question can't easily reply without deanonymizing themselves, which is awkward.

This isn't to take a stance on what you have a right to do or should have done. But I think it's false to say that you obviously haven't violated the norms you mentioned.

Comment by philh on E.T. Jaynes Probability Theory: The logic of Science I · 2024-01-02T22:57:53.623Z · LW · GW

Spoiler alert, using a lot of tedious math (that we honestly do not fully follow)

I had trouble with this too. I asked on math.stackexchange about part of the derivation of the product rule. Apparently I understood the answer at the time, but that was long ago.

the most extreme is the ‘A implies B’ which many people consider equivalent to ‘B implies A’ where the correct logical rule is ‘not B implies not A’. Jaynes argues this is an example of Bayesian reasoning. For example if all dogs are mammals (deduction) it also means that some mammals are dogs, concretely if only 20% of animals are mammals this information increased the likelihood of dogs by a factor of 5!. A implies B means that B increases the likelihood of A by a factor of 5.

Had trouble following this because "animals" are mentioned only once, so to elaborate: we're taking the background info to be X = "this thing is an animal", along with A = "this thing is a dog" and B = "this thing is a mammal". Then  is , and if  then

Jaynes' very physicist-oriented example concerns the emission of particles from a radioactive source and a sensor that can detect some portion of these particles. The radioactive source emits on average  particles per second drawn from a Poisson distribution and the sensor detects particles with an Bernoulli rate of . So if  is 0.1, the sensor picks up 10% of particles over the long run, but if you have just  particles, there is no guarantee that it will detect exactly 1 particle. Similarly, even though we might have  particles per second, any given second might have more or less than 100 particles emitted. This is where it gets complicated. If you use the MLE estimate, you will always get  particles as your estimate for each second of counts, because MLE ‘assumes’ that the 10:1 particle relationship is fixed and thus ignores the Poisson source variability. So let’s say you have a counter with  and have observed a count of 15 particles on this sensor for some second. How many particles, , have originated from the source during this second? MLE will get you 150 particles, as described above. But Jayne’s robot gives us 105 particles. What? This is a HUGE difference! This example also surprised experimental physicists. The reason the robot gets 105 and not 150 is because the source has lower variability than the detector, so a high number is weak evidence of an above average number of particles.

Had trouble following this too. I thought we were trying to estimate  and/or . But that's not it; we know s = 100 and φ = 0.1, and we know we detected 15 particles, and the question is how many were emitted.

And if we do an MLE, we'd say "well, the number-emitted that gives us the highest probability of detecting 15 is 150", so that's the estimate. We're throwing away what we know about the distribution of how-many-emitted.

And I guess we could instead ask "what fraction of emitted particles did we detect?" Presumably then we throw away what we know about the distribution of what-fraction-detected, and we say "well, the fraction-detected that gives us the highest probability of detecting 15 is 0.15", so that would be the MLE.

Which gives us another way to see that MLE is silly, because "what fraction did we detect" and "how many were emitted" are the same question, given that we detected 15.

Comment by philh on Expected utility and repeated choices · 2023-12-31T09:12:08.532Z · LW · GW

There's an unstated assumption here that you start with $0. Suppose instead you start with $0.5: then  while . So if you play game (a) first, you'd then prefer to play game (b) second.

But this doesn't fully resolve the question, because you'd still prefer (b, b) over (a, b).

Comment by philh on Here's the exit. · 2023-12-30T16:17:55.327Z · LW · GW

It’s not possible to take the downvotes as a signal of this if downvotes get used for a wide range of things.

Perhaps not in general, but I think it's often pretty clear. Like you've already said "I’m guessing the intention was to punish me for not caring", and yes, I think you're right. Seems to me the signal was recieved as intended.

Although if the person disagrees with whether it was bad, and the answer to that disagreement is to try to silence them… then that seems to me like a pretty anti-epistemic norm. At least locally.

Well, if someone comes here arguing for flat-earthism, I'm probably going to downvote without bothering to read their arguments. Is that anti-epistemic? Maybe, I guess? Certainly yes, if it turns out that the earth is flat (and that their arguments are correct). And "this practice isn't anti-epistemic as long as we only dismiss false ideas" is, um. Nevertheless, I endorse that practice.

If someone comes around here calling people names, and we downvote that rather than checking in "hey are you doing this because you think name calling is good actually? Would you like to dialogue about that?" is that anti-epistemic? Again, maybe yes? But I endorse it anyway.

The dynamic would be pretty simple:

  • After I downvote, I skim the replies to see if someone else already explained what had me do the downvote. If so, I upvote that explanation and agree-vote it too.
  • If there’s no such explanation, I write one.

Easy peasy.

I do not consider writing these explanations to be easy.

I seriously doubt the number of things needing downvotes on this site is so utterly overwhelming that this approach is untenable.

I can think of a few places we might disagree here: how many things deserve downvotes, how costly it is to explain them, how realistic it is for people to pay those costs. I'm not super enthusiastic about trying to drill down into this, though.

But I also think I'm less optimistic than you about the benefits of doing it. I can think of multiple conversations I've had where I wanted people to change what they're doing, I explained why I thought they were doing something bad, and they just keep on doing it. You yourself seem to understand what it is that many people dislike in many of your posts and comments, and yet you keep doing the thing. Surely there are cases where it does help, but I think they're a minority. (It seems plausible to me that the helpful cases actually do get explained more often than others. E.g. if someone explicitly asks why they're getting downvoted, that's evidence they're interested in improving, and also it makes them more likely to get an explanation.)

Another thing worth mentioning is that reacts reduce the cost of explaining downvotes. I dunno how much they're used, since I mostly use GreaterWrong which doesn't (yet?) support them. I believe they were only added to this post later, so they wouldn't have been helpful at the time. But yeah, if a comment gets downvoted a bunch with not even any reacts explaining why, that seems not ideal.

Comment by philh on Here's the exit. · 2023-12-27T00:36:20.943Z · LW · GW

But it's clear that some of what I said was heavily downvoted because I took a stance people didn't like. Saying things like "Yep, I could have phrased this post in a more epistemically accurate way… but for this post in particular I really don't care."

Well, that particular comment had a lot of other stuff going on, and yes I think it's a kind of comment that doesn't belong here and no I don't particularly feel like explaining that.

But also, yeah, I do kinda feel like "downvoting people when they admit they did something bad" is a thing we sometimes do here and that's not great incentives. If someone wants to avoid that kind of downvote, "stop admitting to the bad thing" seems like an obvious strategy. Oops! And like, I remember times when I asked someone a question and they got downvoted for their answer, and I did think it was a bad answer that in a vacuum deserved downvotes, but I still upvoted as thanks for answering.

I'm not sure it's so bad though. Some things that mitigate it as a strategy:

  • "This person strategically fails to answer certain questions" is a thing it's possible for someone to notice and point out.
  • Someone might not have realized the thing they did was bad-according-to-LW, and the downvotes help signal that. (Maybe better to instead upvote the admission and downvote the thing they did? But that's not always a thing that can be downvoted, or downvotes might not be specifically targetable to make it clear "this thing you did was bad".)
  • If someone did a bad thing and doesn't care, maybe we just don't want them here. Downvotes probably marginally push them away, as well as marginally push them towards not-admitting-things. Notably, I feel like we're more likely to downvote "I did a bad thing and don't care" than "I did a bad thing, oops, sorry".
  • Sometimes someone might take "not being able to say a thing" as a cost, and prefer the downvotes over the silence.

In general it seems like a hard problem, and it's not clear to me that downvoting this kind of thing is a mistake.

I'd also really like to see a return of the old LW cultural thing of, if you downvote then you explain why. There are some downvotes on my comments that I'm left scratching my head about and going "Okay, whatever." It's hard for downvotes to improve culture if the feedback amounts to "Bad."

I think there's currently too many things that deserve downvotes for that to be realistic.

Comment by philh on Bayesian Injustice · 2023-12-23T18:01:09.903Z · LW · GW

Notably, this effect could mean you differentially hire presties even if they're slightly worse on average than the normies. I feel like practicing my math, so let's look at the coin example.

If you flip once and get H, that's an odds ratio of  for a heads-bias, which combine with initial odds of  to give odds for [a  heads-bias] of  which is a probability of , and your expectation of the overall bias is .

Now suppose you flip twice and get HH. Odds for heads-bias is , probability , which gives expectation .

But suppose you flip twice and get HH, but instead of the coins being a mix of  and  heads-biased, they're  and , for some known . The odds ratio is now , probability of [a  heads-bias] is  and the expected bias is . I tried to expand this by hand but made a mistake somewhere, but plugging it into wolfram alpha we get that this equals  when

Where the two solutions are because an equal mix of  and  biases is the same as an equal mix of  and  biases.

So you'd prefer an HH with a most-likely bias of  over an H with a most-likely bias of .

Comment by philh on Nonlinear’s Evidence: Debunking False and Misleading Claims · 2023-12-21T12:39:26.203Z · LW · GW

Alice claimed to be making significant income from her side business.

To clarify further, my read of things is that you think the inaccurate claim would be

  • Alice was in fact making significant income from her side business.

but that you wouldn't dispute

  • Alice claimed to NL that she was making significant income from her side business.

Is that right? Or do you additionally think the second is inaccurate?

Comment by philh on How do you feel about LessWrong these days? [Open feedback thread] · 2023-12-15T20:50:54.352Z · LW · GW

I think I'm less open to weird ideas on LW than I used to be, and more likely to go "seems wrong, okay, next". Probably this is partly a me thing, and I'm not sure it's bad - as I gain knowledge, wisdom and experience, surely we'd expect me to become better at discerning whether a thing is worth paying attention to? (Which doesn't mean I am better, but like. Just because I'm dismissing more ideas, doesn't mean I'm incorrectly dismissing more ideas.)

But my guess is it's also partly a LW thing. It seems to me that compared to 2013, there are more weird ideas on LW and they're less worth paying attention to on average.

In this particular case... when you talk about "We don’t want you to say that" comments, it sounds to me like those comments don't want you to say your ideas. It sounds like Habryka and other commenters interpreted it that way too.

But my read of the the comment you're talking about here isn't that it's opposed to your ideas. Rather, it doesn't want you to use a particular style of argument, and I agree with it, and I endorse "we don't want bad arguments on LW". I downvoted that post of yours because it seemed to be arguing poorly. It's possible I missed something; I admittedly didn't do a close read, because while I've enjoyed a lot of your posts, I don't have you flagged in my brain as "if lsusr seems to be making a clear mistake, it's worth looking closely to see if the error is my own".

(I am sad that the "avoid paying your taxes" post got downvoted. It does seem to me like an example of the thing you're talking about here, and I upvoted it myself.)

Comment by philh on New LessWrong feature: Dialogue Matching · 2023-12-09T13:04:00.283Z · LW · GW

My stance towards almost everyone is "if you want to have a dialogue with me and have a topic in mind, great, I'm probably interested; but I have nothing specific in mind that I want to dialogue with you about". Which makes me not want to check the box, but some people have apparently checked it for me. I'd appreciate having two levels of interest, along the lines of "not interested/interested/enthusiastic" such that a match needs either interested/enthusiastic or enthusiastic/enthusiastic.

Comment by philh on Process Substitution Without Shell? · 2023-12-05T16:41:23.392Z · LW · GW

I'm curious how bash does it. Suppose you do cat <(cat), this gives you

  • One cat process with stdout writing to a pipe
  • One cat /proc/self/fd/11 process with file descriptors 3 and 11 both reading from this pipe.

So if I had to guess: bash calls pipe() to get two ends of a pipe. Let's say these have file descriptors 10 and 11. Then it forks twice, which copies the list of file descriptors. We now have three processes with file descriptors 10 (open for writing) and 11 (open for reading). Parent process closes both ends, one child closes the read end, one child closes the write end.

The writing child process uses dup2() or something to set its stdout to be the pipe, then calls exec("cat", []) or whatever.

The reading child process has open file descriptor 11, and can access it as a file at /proc/self/fd/11. So it calls exec("cat", ["/proc/self/fd/11"]). The process which is now cat still has that open file descriptor, but it doesn't know that, it just knows it's been given a path; it opens that path and gets file descriptor 3, also reading from that pipe.

So if you want to duplicate this in python:

  • You'll need to figure out how to access the pipe() function.
  • Do the various subprocess libraries close file descriptors? If not, you can maybe just pass them /proc/self/fd/(fileno) as the path to read/write. If they do, you might need to do the fork and exec manually?
  • Maybe you also need to figure out "use this specific file descriptor for the stdin/stdout of this process"? Not sure. At minimum a fork/exec would work if you do need to do this, but maybe there's something better.

None of this is very high confidence.

Comment by philh on When and why should you use the Kelly criterion? · 2023-11-11T13:12:28.574Z · LW · GW

You could argue that the latter is more important, since getting high expected utility in the end is the whole point. But on the other hand, when trying to decide on a bet size in practice, there's a limit to the precision with which it is possible to measure your edge, so the difference between optimal bet and Kelly bet could be small compared to errors in your ability to determine the Kelly bet size, in which case thinking about how optimal betting differs from Kelly betting might not be useful compared to trying to better estimate the Kelly bet.

So like, this seems plausible to me, but... yeah, I really do want to distinguish between

  • This maximizes expected utility
  • This doesn't maximize expected utility, but here are some heuristics that suggest maybe that doesn't matter so much in practice

If it doesn't seem important to you to distinguish these, then that's a different kind of conversation than us disagreeing about the math, but here are some reasons I want to distingish them:

  • I think lots of people are confused about Kelly, and speaking precisely seems more likely to help than hurt.
  • I think "get the exact answer in spherical cow cases" is good practice, even if spherical cow cases never come up. "Here's the exact answer in the simple case, and here are some considerations that mean it won't be right in practice" seems better than "here's an approximate answer in the simple case, and here are some considerations that mean it won't be right in practice".
    • Sometimes it's not worth figuring out the exact answer, but like. I haven't yet tried to calculate the utility-maximizing bet for those other utility functions. I haven't checked how much Kelly loses relative to them under what conditions. Have you? It seems like this is something we should at least try to calculate before going "eh, Kelly is probably fine".
  • I've spent parts of this conversation confused about whether we disagree about the math or not. If you had reliably been making the distinction I want to make, I think that would have helped. If I had reliably not made that distinction, I think we just wouldn't have talked about the math and we still wouldn't know if we agreed or not. That seems like a worse outcome to me.

Why specifically higher? You must be making some assumptions on the utility function that you haven't mentioned.

Well, we've established the utility-maximizing bet gives different expected utility from the Kelly bet, right? So it must give higher expected utility or it wouldn't be utility-maximizing.

Comment by philh on How to (hopefully ethically) make money off of AGI · 2023-11-10T09:36:38.228Z · LW · GW

And to take it one step further, holding long term debt at fixed rates is amazing in that situation, such as a long term mortgage.

(This is a typo that reverses the meaning, right? Should be "owing" long term debt, you want to owe a mortgage rather than to have issued a mortgage.)

Comment by philh on When and why should you use the Kelly criterion? · 2023-11-08T12:46:14.756Z · LW · GW

Thanks for clarifying! Um, but to clarify a bit further, here are three claims one could make about these examples:

  1. As , the utility maximizing bet at given wealth will converge to the Kelly bet at that wealth. I basically buy this.
  2. As , the expected utility from utility-maximizing bets at timestep  converges to that from Kelly bets at timestep . I'm unsure about this.
  3. For some finite , the expected utility at timestep  from utility-maximizing bets is no higher than that from Kelly bets. I think this is false. (In the positive: I think that for all finite , the expected utility at timestep  from utility-maximizing bets is higher than that from Kelly bets. I think this is the case even if the difference converges to 0, which I'm not sure it does.)

I think you're saying (2)? But the difference between that and (3) seems important to me. Like, it still seems that to a (non-log-money) utility maximizer, the Kelly bet is strictly worse than the bet which maximizes their utility at any given timestep. So why would they bet Kelly?


Here's why I'm unsure about 2. Suppose we both have log-money utility, I start with $2 and you start with $1, and we place the same number of bets, always utility-maximizing. After any number of bets, my expected wealth will always be 2x yours, so my expected utility will always be  more than yours. So it seems to me that "starting with more money" leads to "having more log-money in expectation forever".

Then it similarly seems to me that if I get to place a bet before you enter the game, and from then on our number of bets is equal, my expected utility will be forever higher than yours by the expected utility gain of that one bet.

Or, if we get the same number of bets, but my first bet is utility maximizing and yours is not, but after that we both place the utility-maximizing bet; then I think my expected utility will still be forever higher than yours. And the same for if you make bets that aren't utility-maximizing, but which converge to the utility-maximizing bet.

And if this is the case for log-money utility, I'd expect it to also be the case for many other utility functions.

...but something about this feels weird, especially with , so I'm not sure. I think I'd need to actually work this out.


Here's a separate thing I'm now unsure about. (Thanks for helping bring it to light!) In my terminology from on Kelly and altruism, making a finite number of suboptimal bets doesn't change how rank-optimal your strategy is. In Kelly's terminology from his original paper, I think it won't change your growth rate.

And I less-confidently think the same is true of "making suboptimal bets all the time, but the bets converge to the optimal bet".

But if that's true... what actually makes those bets suboptimal, in those two frameworks? If Kelly's justification for the Kelly bet is that it maximizes your growth rate, but there are other bet sizes that do the same, why prefer the Kelly bet over them? If my justification for the Kelly bet (when I endorse using it) is that it's impossible to be more rank-optimal than it, why prefer the Kelly bet over other things that are equally rank-optimal?

Comment by philh on When and why should you use the Kelly criterion? · 2023-11-08T08:32:29.150Z · LW · GW

Oh, right! I would have been able to give that proof ten years ago, but I've forgotten a lot since leaving university.

Comment by philh on When and why should you use the Kelly criterion? · 2023-11-07T07:23:58.531Z · LW · GW

Okay, "Kelly is close to optimal for lots of utility functions" seems entirely plausible to me. I do want to note though that this is different from "actually optimal", which is what I took you to be saying.

(The thing you actually wrote is the same as log utility, so I substituted what you may have meant)

Oops! I actually was just writing things without thinking much and didn't realize it was the same.

Comment by philh on When and why should you use the Kelly criterion? · 2023-11-06T16:03:18.928Z · LW · GW

Yes, but there's an additional thing I'd point out here, which is that at any finite timestep, Kelly does not dominate. There's always a non-zero probability that you've lost every bet so far.

When you extend the limit to infinity, you run into the problem "probability zero events can't necessarily be discounted" (though in some situations it's fine to), which is the one you point out; but you also run into the problem "the limit of the probability distributions given by Kelly betting is not itself a probability distribution".

Comment by philh on When and why should you use the Kelly criterion? · 2023-11-06T13:30:00.625Z · LW · GW

(Variance is "expected squared difference between observation and its prior expected value", i.e. variance as a concept is closely linked to the mean and not so closely linked to the median or mode. So if you're talking about "average" and "variance" and the average you're talking about isn't the mean, I think at best you're being very confusing, and possibly you're doing something mathematically wrong.)

Comment by philh on When and why should you use the Kelly criterion? · 2023-11-06T10:25:04.553Z · LW · GW

So I claim that Kelly won't maximize , or more generally  for any , or , or , or , or even  but it'll get asymptotically close when . Do you disagree?

Your "When to act like your utility is logarithmic" section sounds reasonable to me. Like, it sounds like the sort of thing one could end up with if one takes a formal proof of something and then tries to explain in English the intuitions behind the proof. Nothing in it jumps out at me as a mistake. Nevertheless, I think it must be mistaken somewhere, and it's hard to say where without any equations.

Comment by philh on Architects of Our Own Demise: We Should Stop Developing AI · 2023-11-01T17:41:24.595Z · LW · GW

Re timelines for climate change, in the 1970s, serious people in the field of climate studies started suggesting that there was a serious problem looming. A very short time later, the entire field was convinced by the evidence and argument for that serious risk—to the point that the IPCC was established in 1988 by the UN.

When did some serious AI researchers start to suggest that there was a serious problem looming? I think in the 2000s. There is no IPAIX-risk.

Nod. But then, I assume by the 1970s there was already observable evidence of warming? Whereas the observable evidence of AI X-risk in the 2000s seems slim. Like I expect I could tell a story for global warming along the lines of "some people produced a graph with a trend line, and some people came up with theories to explain it", and for AI X-risk I don't think we have graphs or trend lines of the same quality.

This isn't particularly a crux for me btw. But like, there are similarities and differences between these two things, and pointing out the similarities doesn't really make me expect that looking at one will tell us much about the other.

I think that if people like me are to be convinced it might be worth the attempt. As you say, there may be a more accessible to me domain in there somewhere.

Not opposed to trying, but like...

So I think it's basically just good to try to explain things more clearly and to try to get to the roots of disagreements. There are lots of ways this can look like. We can imagine a conversation between Eliezer and Yann, or people who respectively agree with them. We can imagine someone currently unconvinced having individual conversations with each side. We can imagine discussions playing out through essays written over the course of months. We can imagine FAQs written by each side which give their answers to the common objections raised by the other. I like all these things.

And maybe in the process of doing these things we eventually find a "they disagree because ..." that helps it click for you or for others.

What I'm skeptical about is trying to explain the disagreement rather than discover it. That is, I think "asking Eliezer to explain what's wrong with Yann's arguments" works better than "asking Eliezer to explain why Yann disagrees with him". I think answers I expect to the second question basically just consist of "answers I expect to the first question" plus "Bulverism".

(Um, having written all that I realize that you might just have been thinking of the same things I like, and describing them in a way that I wouldn't.)

Comment by philh on Architects of Our Own Demise: We Should Stop Developing AI · 2023-10-31T19:06:34.286Z · LW · GW

I don't know, I didn't intend it to relate to those things. It was a narrow reply to something in your comment, and I attempted to signal it as such.

(I'm not very invested in this conversation and currently intend to reply at most twice more.)

Comment by philh on Architects of Our Own Demise: We Should Stop Developing AI · 2023-10-31T16:01:12.544Z · LW · GW

I didn't know that, but not a crux. This information does not make me think it was obviously unreasonable to try both approaches simultaneously.

(Downvoted for tone.)

Comment by philh on Architects of Our Own Demise: We Should Stop Developing AI · 2023-10-31T10:05:57.335Z · LW · GW

If we look at climate change, for example, the vast majority of experts agreed about it quite early on—within 15 years of the Charney report.

So I don't know much about timelines of global warming or global warming science, but I note that that report came out in 1979, more than 100 years after the industrial revolution. So it's not clear to me that fifteen years after that counts as "quite early on", or that AI science is currently at a comparable point in the timeline. (If points in these timelines can even be compared.)

If all I am left with, however, is ‘smart person believes silly thing for silly reasons’ then it is not reasonable for me as a lay person to determine which is the silly thing.

FWIW I think even relatively-lay people can often detect silly arguments, even from people who know a lot more than them. Some examples where I think I've done that:

  • I remember seeing someone (possibly even Yann LeCun?) saying something along the lines of, AGI is impossible because of no free lunch theorems.
  • Someone saying that HPMOR's "you violated conservation of energy!" bit is dumb because something something quantum stuff that I didn't understand; and also because if turning into a cat violated conservation of energy, then so did levitating someone a few paragraphs earlier. I am confident this person (who went by the handle su3su2u1) knows a lot more about physics than me. I am also confident this second part was them being silly.
  • This comment.

So I'd suggest that you might be underestimating yourself.

But if you're right that you can't reasonably figure this out... I'm not sure there are any ways to get around that? Eliezer can say "Yann believes this because of optimism bias" and Yann can say "Eliezer believes this because of availability heuristic" or whatever, and maybe one or both of them is right (tbc I have not observed either of them saying these things). But these are both Bulverism.

It may be that Eliezer and Yann can find a double crux, something where they agree: "Eliezer believes X, and if Eliezer believed not-X then Eliezer would think AGI does not pose a serious risk. Yann believes not-X, and if Yann believed X then Yann would think AGI does pose a serious risk." But finding such Xs is hard, I don't expect there to be a simple one, and even if there was it just punts the question: "why do these two smart people disagree on X?" It's possible X is in a domain that you consider yourself better able to have an opinion on, but it's also possible it's in one you consider yourself less able to have an opinion on.

If AI does indeed pose unacceptable x-risk and there are good arguments/​good evidence for this, then there also has to be a good reason or set of reasons why many experts are not convinced.

I basically just don't think there does have to be this.

(Yann claims, for example, that the AI experts arguing for AI x-risk are a very small minority and Eliezer Yudkowsky seems to agree with this)

Fwiw my sense is that this is false, and that Yann might believe it but I don't expect Eliezer to. But I don't remember what I've seen that makes me think this. (To some extent it might depend on who you count as an expert and what you count as arguing for x-risk.)

Comment by philh on Architects of Our Own Demise: We Should Stop Developing AI · 2023-10-30T20:47:47.714Z · LW · GW

there needs to be some kind of explanation as to why there is such a difference of opinion by experts

Isn't this kind of thing the default? Like, for ~every invention that changed the world I'd expect to be able to find experts saying in advance that it won't work or if it does it won't change things much. And for lots of things that didn't work or didn't change the world, I'd expect to be able to find experts saying it would. I basically just think that "smart person believes silly thing for silly reasons" is pretty common.

Comment by philh on Architects of Our Own Demise: We Should Stop Developing AI · 2023-10-30T20:22:01.313Z · LW · GW

Without commenting on whether there was in fact much agreement or disagreement among the physicists, this doesn't sound like much evidence of disagreement. I think it's often entirely reasonable to try two technical approaches simultaneously, even if everyone agrees that one of them is more promising.

Comment by philh on A Good Explanation of Differential Gears · 2023-10-24T11:30:14.411Z · LW · GW

This was great.

Tangentially, this post reminded me of this GPS explainer which I also thought was excellent as explanation.

Comment by philh on Dishonorable Gossip and Going Crazy · 2023-10-20T10:42:25.121Z · LW · GW

I think if you take the EA worldview seriously then the obvious conclusion is that Earth life up until and including today has been net-negative because of animal suffering.

Nit: I don't consider "the EA worldview" to have any opinion on animal suffering. But (roughly speaking) I agree you can get this conclusion from the EA worldview plus some other stuff which is also common among EAs.