Posts

Humans, chimpanzees and other animals 2023-05-30T23:53:08.295Z
On "aiming for convergence on truth" 2023-04-11T18:19:18.086Z
Large language models learn to represent the world 2023-01-22T13:10:38.837Z
Suspiciously balanced evidence 2020-02-12T17:04:20.516Z
"Future of Go" summit with AlphaGo 2017-04-10T11:10:40.249Z
Buying happiness 2016-06-16T17:08:53.802Z
AlphaGo versus Lee Sedol 2016-03-09T12:22:53.237Z
[LINK] "The current state of machine intelligence" 2015-12-16T15:22:26.596Z
Scott Aaronson: Common knowledge and Aumann's agreement theorem 2015-08-17T08:41:45.179Z
Group Rationality Diary, March 22 to April 4 2015-03-23T12:17:27.193Z
Group Rationality Diary, March 1-21 2015-03-06T15:29:01.325Z
Open thread, September 15-21, 2014 2014-09-15T12:24:53.165Z
Proportional Giving 2014-03-02T21:09:07.597Z
A few remarks about mass-downvoting 2014-02-13T17:06:43.216Z
[Link] False memories of fabricated political events 2013-02-10T22:25:15.535Z
[LINK] Breaking the illusion of understanding 2012-10-26T23:09:25.790Z
The Problem of Thinking Too Much [LINK] 2012-04-27T14:31:26.552Z
General textbook comparison thread 2011-08-26T13:27:35.095Z
Harry Potter and the Methods of Rationality discussion thread, part 4 2010-10-07T21:12:58.038Z
The uniquely awful example of theism 2009-04-10T00:30:08.149Z
Voting etiquette 2009-04-05T14:28:31.031Z
Open Thread: April 2009 2009-04-03T13:57:49.099Z

Comments

Comment by gjm on Middle Child Phenomenon · 2024-03-16T18:19:57.911Z · LW · GW

Yes , I know what the middle-child phenomenon is in the more literal context. I just don't have any idea why you're using the term here. I don't see any similarities between the oldest / middle / youngest child relationships in a family and whatever relationships there might be between programmers / lawyers / alignment researchers.

(I think maybe all you actually mean is "these people are more important than we're treating them as". Might be true, but that isn't a phenomenon, it's just a one-off judgement that a particular group of people are being neglected.)

I still don't understand why the distribution of talent/success/whatever among law students is relevant. If your point is that very few of them are going to be in a position to make a difference to AI policy then surely that actually argues against your main claim that law students should be getting more attention from people who care about AI.

Comment by gjm on Middle Child Phenomenon · 2024-03-16T11:40:16.935Z · LW · GW

Having read this post, I am still not sure what "the Middle Child Phenomenon" actually is, nor why it's called that.

The name suggests something rather general. But most of the post seems like maybe the definition is something like "the fact that there isn't a vigorous effort to get law students informed about artificial intelligence".

Except that there's also all the stuff about the distribution of talent and interests among law students, and another thing I don't understand is what that actually has to do with it. If (as I'm maybe 75% confident) the main point of the post is that it would be valuable to have law students learn something about AI because public policy tends to be strongly influenced by lawyers, then it seems like this point would be equally strong regardless of how your cohort of 1000 lawyers is distributed between dropouts, nobodies, all-rounders, CV-chasers, and "golden children". (I am deeply unconvinced by this classification, by the way, but I am not a lawyer myself and maybe it's more accurate than it sounds.)

Comment by gjm on Constructive Cauchy sequences vs. Dedekind cuts · 2024-03-15T03:52:46.324Z · LW · GW

It looks as if you're taking a constructive Dedekind cut to involve a "set of real numbers" in the sense of a function for distinguishing left-things from right-things.

Is that actually how constructivists would want to define them? E.g., Bishop's "Foundations of Constructive Analysis", if I am understanding its definitions of "set" and "subset" correctly (which I might not be), says in effect that a set of rational numbers is a recipe for constructing elements of that set, along with a way of telling whether two things constructed in this way are equal. I'm pretty sure you can have one of those but not be able to determine explicitly whether a given rational number is in the set, in which case your central argument doesn't go through.

Are Cauchy sequences and Dedekind cuts equivalent if one thinks of them as Bishop does? There's an exercise in his book that claims they are. I haven't thought about this much and am very much not an expert on this stuff, and for all I know Bishop may have made a boneheaded mistake at this point. I'm also troubled by the apparent vagueness of Bishop's account of sets and subsets and whatnot.

More concretely, that exercise in Bishop's book says: a Dedekind cut is a pair of nonempty sets of rationals S,T such that we always have s<t and given rationals x<y either x is in S or y is in T. Unless I'm confused about Bishop's account of sets, all of this is consistent with e.g. S containing the negative rationals and T the positive rationals, and not being able to say that 0 is in either of them. And unless I'm confused about your "arbitration oracles", you can't build an arbitration oracle out of that setup.

(But, again: not an expert on any of this, could be horribly wrong.)

Comment by gjm on [deleted post] 2024-03-02T00:27:15.699Z

I did, in fact, read the post and the NYT articles, and I am not convinced that your description of what they do and what it means is correct. So, if my response to your article doesn't consist mostly of the gushing praise your first paragraph indicates you'd prefer, that's one reason why.

But, regardless of that: If you write something wrong, and someone points out that it's wrong, I don't think it's reasonable to respond with "how dare you point that out rather than looking only at the other parts of what I wrote?".

Scott is not using some weird eccentric definition of "lie". E.g., the main definition in the OED is: "An act or instance of lying; a false statement made with intent to deceive; a criminal falsehood." (Does that first clause soften it? Not really; it's uninformative, because they define the verb "lie" in terms of the noun "lie".) First definition in Wiktionary is " To give false information intentionally with intent to deceive". But, in any case, even with a very broad definition of "lie" the first four levels in his taxonomy are simply, uncontroversially, obviously not kinds of lying. Again, the first one is "reasoning well, and getting things right".

If I say "There are seven classes of solid objects in the solar system: dust motes, pebbles, boulders, mountains, moons, small planets, and large planets" and you identify something as a small planet, you should not call it "a Level 6 Planet, according to gjm's classification of planets".

And, while I understand a preference for being charitable and not leaping to calling things dishonest that aren't necessarily so ... I don't think you get to demand such treatment in the comments on an article that does the exact reverse to someone else.

Comment by gjm on [deleted post] 2024-03-01T01:57:29.215Z

Your justification seems to me almost completely non-responsive to the point I was actually making, which is not about whether it's reasonable to call what the NYT did in these cases "lying" but about whether it's reasonable to call something at level 6 in Scott's taxonomy a "level 6 lie".

Scott classifies utterances into seven types in ascending order of dishonesty. The first four are uncontroversially not kinds of lying. Therefore, something on the sixth level of Scott's taxonomy cannot reasonably be called a "level 6 lie", because that phrase will lead any reader who hasn't checked carefully to think that Scott has a taxonomy of levels of lying, where a "level 6 lie" is something worse than a level 5 lie, which is worse than a level 4 lie, ... than a level 1 lie, with all these things actually being kinds of lies.

Whereas in fact, even if we ignore Scott's own opinion that only "the most egregious cases of 6" (and also all of 7) deserve to be called lies at all, at the absolute worst a level-6 utterance is more-dishonest-than only one lower level of lie.

Further, you called these things "Scott Alexander's criteria for media lies", which is plainly not an accurate description because, again, more than half the levels in his taxonomy are completely uncontroversially not lying at all (and Scott's own opinion is that only the top level and "the most egregious cases of" the one below should be called lying).

So even if you were 100% sincere and reasonable in regarding what the NYT did as ("routinely and brazenly") lying, I do not see any way to understand your alleged application of Scott's taxonomy as a sincere and reasonable use of it. I do not find it plausible that you are really unable to understand that most of its levels are plainly not types of lie. I do not find it plausible that you really thought that something that begins with "reasoning well, and getting things right" followed by "reasoning well, but getting things wrong because the world is complicated and you got unlucky" can rightly be described as "criteria for media lies".

I could, of course, be wrong. Maybe you really are stupid enough not to understand that "according to X's criteria for media lies, Y is a level 6 lie" implies that what X presented is a classification of lies into levels, in which Y comes at level 6. Or maybe the stupidity is mine and actually most people wouldn't interpret it that way. (I would bet heavily against that but, again, I could be wrong.) Maybe you didn't actually read Scott's list, somehow. But you don't generally seem stupid or unable to understand the meanings and implications of words, so I still find it much much more plausible that you knew perfectly well that Scott was presenting a taxonomy of mostly-not-lies, and chose to phrase things as you did because it made what you were accusing the NYT of sound worse. Which is, I repeat, on at least level 6 of Scott's taxonomy.

And, again, none of this is about whether the NYT really did what you say, nor about whether it's reasonable to describe what you said the NYT did was lying. It's entirely about your abuse of Scott's taxonomy, which (1) is not a list of "criteria for media lies" and (2) is not something that justifies calling an utterance at its Nth level a "level N lie".

Comment by gjm on Intuition for 1 + 2 + 3 + … = -1/12 · 2024-02-19T02:50:23.939Z · LW · GW

It is not true that "no pattern that suggests a value suggests any other", at least not unless you say more precisely what you are willing to count as a pattern.

Here's a template describing the pattern you've used to argue that 1+2+...=-1/12:

We define numbers  with the following two properties. First, , so that for each  we can think of  as a sequence that's looking more and more like (1,2,3,...) as  increases. Second,  where , so the sums of these sequences that look more and more like (1,2,3,...) approach -1/12.

(Maybe you mean something more specific by "pattern". You haven't actually said what you mean.)

Well, here are some  to consider. When  we'll let . When  we'll let . And when  we'll let . Here,  is some fixed number; we can choose it to be anything we like.

This array of numbers satisfies our first property: . Indeed, once  we have , and the limit of an eventually-constant sequence is the thing it's eventually constant at.

What about the second property? Well, as you'll readily see I've arranged that for each  we have . So the sequence of sums converges to .

In other words, this is a "pattern" that makes the sum equal to . For any value of  we choose.

I believe there are more stringent notions of "pattern" -- stronger requirements on how the  approach  for large  -- for which it is true that every "pattern" that yields a finite sum yields . But does this actually end up lower-tech than analytic continuation and the like? I'm not sure it does.

(One version of the relevant theory is described at https://terrytao.wordpress.com/2010/04/10/the-euler-maclaurin-formula-bernoulli-numbers-the-zeta-function-and-real-variable-analytic-continuation.)

Comment by gjm on Social media use probably induces excessive mediocrity · 2024-02-18T03:06:15.264Z · LW · GW

Once again you are making a ton of confident statements and offering no actual evidence. "is a high priority", "they want", "they don't want", "what they're aiming for is", etc. So far as I can see you don't in fact know any of this, and I don't think you should state things as fact that you don't have solid evidence for.

Comment by gjm on Social media use probably induces excessive mediocrity · 2024-02-18T00:55:45.995Z · LW · GW

Let us suppose that social media apps and sites are, as you imply, in the business of trying to build sophisticated models of their users' mental structures. (I am not convinced they are -- I think what they're after is much simpler -- but I could be wrong, they might be doing that in the future even if not now, and I'm happy to stipulate it for the moment.)

If so, I suggest that they're not doing that just in order to predict what the users will do while they're in the app / on the site. They want to be able to tell advertisers "_this_ user is likely to end up buying your product", or (in a more paranoid version of things) to be able to tell intelligence agencies "_this_ user is likely to engage in terrorism in the next six months".

So inducing "mediocrity" is of limited value if they can only make their users more mediocre while they are in the app / on the site. In fact, it may be actively counterproductive. If you want to observe someone while they're on TikTok and use those observations to predict what they will do when they're not on TikTok, then putting them into an atypical-for-them mental state that makes them less different from other people while on TikTok seems like the exact opposite of what you want to do.

I don't know of any good reason to think it at all likely that social media apps/sites have the ability to render people substantially more "mediocre" permanently, so as to make their actions when not in the app / on the site more predictable.

If the above is correct, then perhaps we should expect social media apps and sites to be actively trying not to induce mediocrity in their users.

Of course it might not be correct. I don't actually know what changes in users' mental states are most helpful to social media providers' attempts to model said users, in terms of maximizing profit or whatever other things they actually care about. Are you claiming that you do? Because this seems like a difficult and subtle question involving highly nontrivial questions of psychology, of what can actually be done by social media apps and sites, of the details of their goals, etc., and I see no reason for either of us to be confident that you know those things. And yet you are happy to declare with what seems like utter confidence that of course social media apps and sites will be trying to induce mediocrity in order to make users more predictable. How do you know?

Comment by gjm on Social media use probably induces excessive mediocrity · 2024-02-17T23:16:15.379Z · LW · GW

"Regression to the mean" is clearly an important notion in this post, what with being in the title and all, but you never actually say what you mean by it. Clearly not the statistical phenomenon of that name, as such.

(My commenting only on this should not be taken to imply that I find the rest of the post reasonable; I think it's grossly over-alarmist and like many of Trevor's posts treats wild speculation about the capabilities and intentions of intelligence agencies etc. as if it were established fact. But I don't think it likely that arguing about that will be productive.)

Comment by gjm on Opinions survey (with rationalism score at the end) · 2024-02-17T02:54:49.512Z · LW · GW

What's going on is that tailcalled's factor model doesn't in fact do a good job of identifying rationalists by their sociopolitical opinions. Or something like that.

[EDITED to add:] Here's one particular variety of "something like that" that I think may be going on: an opinion may be highly characteristic of a group even if it is very uncommon within the group. For instance, suppose you're classifying folks in the US on a left/right axis. If someone agrees with "We should abolish the police and close all the prisons" then you know with great confidence which team they're on, but I'm pretty sure the great majority of leftish people in the US disagree with it. If someone agrees with "We should bring back slavery because black people aren't fit to run their own lives" then you know with great confidence which team they're on, but I'm pretty sure the great majority of rightish people in the US disagree with it.

Tailcalled's model isn't exactly doing this sort of thing to rationalists -- if someone says "stories about ghosts are zero evidence of ghosts" then they have just proved they aren't a rationalist, not done something extreme but highly characteristic of (LW-style) rationalists -- but it's arguably doing something of the sort to a broader fuzzier class of people that are maybe as near as the model can get to "rationalists". Roughly the people some would characterize as "Silicon Valley techbros".

Comment by gjm on Opinions survey (with rationalism score at the end) · 2024-02-17T02:54:01.996Z · LW · GW

There are definitely answers that your model wants rationalists to give but that I think are incompatible with LW-style rationalism. For instance:

  • "People's anecdotes about seeing ghosts aren't real evidence for ghosts" (your model wants "agree strongly"): of course people's anecdotes about seeing ghosts are evidence for ghosts; they are more probable if ghosts are real than if they aren't. They're just really weak evidence for ghosts and there are plenty of other reasons to think there aren't ghosts.
  • "We need more evidence that we would benefit before we charge ahead with futuristic technology that might irreversibly backfire" (your model wants "disagree" or "disagree strongly"): there's this thing called the AI alignment problem that a few rationalists are slightly concerned about, you might have heard of it.

And several others where I wouldn't go so far as to say "incompatible" but where I confidently expect most LWers' positions not to match your model's predictions. For instance:

  • "It is morally important to avoid making people suffer emotionally": your model wants not-agreement, but I think most LWers would agree with this.
  • "Workplaces should be dull to reflect the oppressiveness of work": your model wants not-disagreement, but I think most LWers would disagree (though probably most would think "hmm, interesting idea" first).
  • "Religious people are very stupid"; your model wants agreement, but I think most LWers are aware that there are plenty of not-very-stupid religious people (indeed, plenty of very-not-stupid religious people) and I suspect "disagree strongly" might be the most common response from LWers.

I don't claim that the above lists are complete. I got 11/24 and I am pretty sure I am nearer the median rationalist than that might suggest.

Comment by gjm on the gears to ascenscion's Shortform · 2024-02-14T16:08:06.615Z · LW · GW

I don't have particularly strong opinions and think you should do whatever you like with your name, but just as a datapoint I (1) didn't think "the gears to ascension" was either so cool a name as to demand respect or so stupid a name as to preclude it, and (2) don't think the "often wrong" in your name will make much difference to how I read your comments.

I don't think it ever occurred to me to think that calling yourself "the gears to ascension" amounted to claiming to be a key part of some transhumanist project or anything like that. The impression it gave me was "transhumanist picking a name that sounds cool to them".

The "often wrong" provokes the following thoughts: (1) this person is aware of often being wrong, which is more than most people are, so maybe take them more seriously? (2) this person is, by their own account, often wrong, so maybe take them less seriously? (3) this person is maybe doing a sort of defensive self-deprecatory fishing-for-compliments thing, so maybe take them less seriously? but all of these are pretty weak effects, and I think 2+3 more or less exactly cancel out 1.

"Lauren (often wrong)" is probably about equally memorable to "the gears to ascension". if your goal is to have all your comments stand on their own, then aside from the one-off effect of reducing the association between things said by "Lauren" and things said by "gears" I don't think the change will do much one way or the other. "Lauren" on its own is probably less memorable and your comments might be treated as more independent of one another if you just called yourself that. (But there appear already to be two users called just Lauren, so something slightly more specific might be better.)

Comment by gjm on I played the AI box game as the Gatekeeper — and lost · 2024-02-12T21:05:44.836Z · LW · GW

The trouble with these rules is that they mean that someone saying "I played the AI-box game and I let the AI out" gives rather little evidence that that actually happened. For all we know, maybe all the stories of successful AI-box escapes are really stories where the gatekeeper was persuaded to pretend that they let the AI out of the box (maybe they were bribed to do that; maybe they decided that any hit to their reputation for strong-mindedness was outweighed by the benefits of encouraging others to believe that an AI could get out of the box; etc.). Or maybe they're all really stories where the AI-player's ability to get out of the box depends on something importantly different between their situation and that of a hypothetical real boxed AI (again, maybe they bribed the gatekeeper and the gatekeeper was willing to accept a smaller bribe when the outcome was "everyone is told I let the AI out" rather than whatever an actual AI might do once out of the box; etc.).

Of course, even without those rules it would still be possible for gatekeepers to lie about the results. But if e.g. a transcript were released then there'd be ways to try to notice those failure modes. If the gatekeeper-player lets the AI-player out of the box and a naysayer says "bah, I wouldn't have been convinced", that could be self-delusion on the naysayer's part (or unawareness that someone playing against them might have adopted a different method that would have worked better on them) but it could also be that the gatekeeper-player really did let the AI-player out "too easily" in a way that wouldn't transfer to the situations the game is meant to build intuitions about.

Comment by gjm on [deleted post] 2024-02-10T13:59:39.447Z

It seems to me that "are against immigration" means multiple things that are worth separating. If "old Republicans are against immigration", that might mean

  • that when asked their opinions, that's what they say
  • that when given the choice of voting for anti-immigration Republican candidates or pro-immigration Democratic candidates they choose the former
  • that when given the choice of anti-immigration Republican candidates or pro-immigration Republican candidates they choose the former
  • that when they have the opportunity to influence Republican party policy they try to push it in an anti-immigration direction

and these are all different things.

Voting mostly for reasons other than affecting the outcome certainly explains the second of those, but that one wasn't particularly in need of explanation anyway if Republican candidates generally suit the self-interest of old Republicans better than Democratic candidates.

(Perhaps your argument is that actually voting Republican at all is against the self-interest of old Republican voters? If so, it seems to me like more support for that claim is needed.)

As for the others:

  • Stating "immigration is bad", like voting for anti-immigration candidates, doesn't actually have much likelihood of changing immigration policy, so maybe your explanation also covers that (and, more generally, pretty much any expression of political opinions). What it doesn't explain is why those old Republicans would want to say that immigration is bad; an obvious explanation (which again generalizes to wealthy Democrats advocating tax increases, etc.) is that they consider their interests advanced on the whole if their party does well, and so they advocate for all their party's positions.
  • Do we actually know anything about what old Republicans do when they have a (real) choice between two Republican candidates who differ mostly in immigration policy? (To whatever extent e.g. Donald Trump is a particularly anti-immigration candidate, his success in the Republican primaries might be evidence, but it feels like there are way too many other confounding factors there.) My impression is that there just aren't very many pro-immigration Republicans around to vote for.
  • Do we actually know anything about what old Republicans do if and when they have a chance to influence Republican immigration policy? (My impression is that most old Republicans have no such chance, and those who do might be highly atypical in various relevant ways.)
Comment by gjm on Dreams of AI alignment: The danger of suggestive names · 2024-02-10T12:14:54.193Z · LW · GW

Do you have some concrete examples where you've explained how some substantial piece of the case for AI accident risk is a matter of word games?

Comment by gjm on Most experts believe COVID-19 was probably not a lab leak · 2024-02-05T00:52:05.955Z · LW · GW

The prior for any given newly-emerged virus being a natural zoonosis rather than a lab leak is higher when there are fewer labs to leak.

I agree that the prior for a leak happening from any given lab at any given time doesn't depend on how many labs there are, of course.

Comment by gjm on Most experts believe COVID-19 was probably not a lab leak · 2024-02-04T02:47:43.253Z · LW · GW

The DEFUSE proposal that you linked to doesn't (so far as I can tell) say anything about where the furin cleavage site work would be done. OP here includes an image that seems to show a Word document or something with a comment about that, but it isn't obvious to me where it's from or how it's known to be genuine or anything.

The "uniformly spaced recognition sites and BsmBI" thing (at least, the instance of it that I've found) looks rather sketchy and unconvincing to me, though I'm not really competent to evaluate it (are you?). It's possible that what I'm looking at isn't what you're referring to; I'm talking about a post on Alex Washburne's Substack where he draws attention to mention in the DEFUSE proposal of "reverse genetic systems" and "infectious clone technology" (though so far as I can see neither of these is actually mentioned in the proposal itself), claims (I do not know with what evidence) that using these methods would produce unusually regularly spaced instances of certain genome subsequences that are targeted by BsmBI or a similar enzyme, and claims that the SARS-CoV-2 shows such unusually regularly spaced instances.

But (1) unless there's something I'm missing this "specifically links WIV to" those methods only in a very weak sense (e.g., the DEFUSE proposal doesn't in fact say that they are going to use those methods, or that it would be done at the WIV), (2) Washburne doesn't provide any support for his claim that researchers using this technique would in fact make the relevant segments unusually uniform in length, and (3) nor does he seem to give any details of the analysis that supposeedly shows that SARS-CoV-2 has such unusually uniform segments.  He makes some claims about earlier drafts of the DEFUSE proposal supposedly obtained by FOIA requests, which if correct go some way to filling these gaps a bit, but if he actually shows those or gives evidence that they're real then I haven't seen it.

(Note: I find the style of Washburne's writing very offputting; it pattern-matches to e.g. creationists crowing about their nonsensical statistical arguments. Lots of expressions of triumph, pointlessly flowery writing, that sort of thing. Of course that isn't strong evidence that Washburne is wrong in the same sort of way as the creationists are, but I find it does strongly incline me to skepticism. It's odd that, when arguing that something previously dismissed as a conspiracy theory is probably true, it doesn't occur to him to try not writing like a conspiracy theorist.)

Comment by gjm on Most experts believe COVID-19 was probably not a lab leak · 2024-02-03T23:41:54.508Z · LW · GW

You're saying "this virus" again when what's actually known is that WIV was studying coronaviruses, not specifically that it was studying SARS-COV-2.

(If it turns out that WIV was studying SARS-COV-2 specifically before it started infecting humans then yes, that would be very strong evidence in favour of lab leak theories.)

Anyway: yes, I do agree that the fact that SARS-COV-2 first got into humans somewhere rather near a lab that was studying similar viruses is substantial Bayesian evidence that it got into humans via that lab. But the exact same thing that makes it substantial evidence (there aren't many such labs, whereas there are many opportunities for natural zoonosis which could happen in a wider variety of places) also means that the prior is low.

So the question is roughly "how dangerous do you think, on priors, a single WIV-like lab is, compared to a large wet market?". ("Dangerous" meaning probability of releasing coronaviruses into the human population.) If, before hearing about SARS-COV-2, you would have thought WIV was about as likely to release coronaviruses into the human population as the Wuhan wet market, then after hearing about SARS-COV-2 you should think the probability it was a lab leak is about 50%. (And then potentially modify that substantially depending on all the details that we're ignoring in this discussion, which might give more concrete evidence to distinguish the hypotheses.) Etc.

[EDITED to add:] I've now seen your post about this; I agree that the DEFUSE thing seems like highly relevant evidence (but haven't looked into the DEFUSE proposal myself to check whether I agree with what you say about it). If it's correct that WIV is known to have been working on something much more specifically matched to SARS-COV-2 then that dangerousness ratio looks quite different (because something so much more specific is correspondingly less likely to occur as a natural zoonosis).

Comment by gjm on Most experts believe COVID-19 was probably not a lab leak · 2024-02-03T15:43:51.601Z · LW · GW

Indeed, but my impression is that low response rates are the default outcome when you mail around a survey like this.

(It's natural to wonder about bias. My feeling is that people who think COVID-19 was a lab leak are probably more, not less, likely to want to answer an anonymous survey that gives them a chance to say so.)

Comment by gjm on Most experts believe COVID-19 was probably not a lab leak · 2024-02-03T15:41:25.549Z · LW · GW

Aside from the fact that you're answering a different question from Brendan's, this argument seems like it involves some assumptions that are not known to be correct.

Isn't the right version of your question "Create a virus in a natural spillover event; what is the chance that that spillover happens within a few miles of a lab that is studying similar viruses?"?

The answer to that might be "fairly large", if e.g. it happens that there are virus labs and likely spots for natural zoonotic spillover located near to one another. Which is in fact the case in Wuhan, no?

(I don't know how well the details of the case fit the "escape from the Wuhan virus lab" and "zoonosis at the Wuhan wet market" hypotheses. Maybe they're a better fit for the former than for the latter. But that's a very different sort of argument from "there's a virus lab that was studying coronaviruses near to where COVID-19 was first seen in humans".)

Comment by gjm on Most experts believe COVID-19 was probably not a lab leak · 2024-02-03T15:35:59.563Z · LW · GW

I think you're answering a different question from the one Brendan asked.

You're answering "Why do you think COVID-19 escaped from a lab?".

Brendan was asking "Conditional on COVID-19 having escaped from a lab, why do you think it was created there rather than being a natural virus they were studying in that lab?".

Comment by gjm on [deleted post] 2024-02-02T15:50:31.180Z

I think using the phrase "level 6 lies" when referring to Scott's taxonomy is itself at least a "level 6 lie".

Here, by way of reminder, is Scott's list. (I've abbreviated it a bit; you can find the full version at the far end of trevor's link if you want to check my honesty.) 1. Reasoning well and getting things right. 2. Reasoning well and getting things wrong by bad luck. 3. Reasoning badly through honest incompetence. 4. Reasoning badly because of unconscious bias. 5. Presenting true facts in a way that misleads, "subconsciously and unthinkingly". 6. Presenting true facts in a way that misleads, with deliberate intent to deceive. 7. Saying false things on purpose, in order to deceive.

I claim that "levels" 1-4 in this list are simply, straightforwardly, uncontroversially not lying.

It seems highly dubious to me to call 5 and 6 (5 especially) "lying", but even if we decide to do so Scott's 6 is not in any useful sense the sixth of seven levels of lying, it's at most the second of three levels.

Further, Scott's own position, stated very clearly in the article you linked to, is that things at level 5 are not lies, and that most things at level 6 are not lies either. "I prefer to reserve lying for 7 and the most egregious cases of 6, and to have a pretty high standard for accusing people of this rather than 2/3/4/5."

If you call things at level 6 (and, it seems to me, not in fact "the most egregious cases of 6" in this case) "level 6 lies" then you are not, contrary to your own description, applying "Scott's criteria for media lies".

I suggest that "brazenly lied" also strongly implies something more than "lying by omission".

(Others have already made essentially the same point I am making here, but I think it needs making with more force and details.)

Comment by gjm on Adam Smith Meets AI Doomers · 2024-02-01T02:24:36.599Z · LW · GW

Meta: If you're going to say "Written with the assistance of GPT-4" then I, at least, want to know roughly what GPT-4 assisted with. Did you use it to clean up little mistakes in spelling and grammar? (In that case, I don't actually see why it's necessary to mention GPT-4 at all. "Written with the assistance of the Chambers English Dictionary.") Did you write a first version of the article and use GPT-4 to make the writing style more, well, GPT-4-like? Did you say "Please write me an article about economics and AI risk suitable for posting on Less Wrong"? Or what? The inferences I draw both about the article and about GPT-4 are going to be quite different in those different cases.

Comment by gjm on Notes on Innocence · 2024-01-31T02:24:35.365Z · LW · GW

I actually considered writing "cardiologists" instead of "pianists" :-).

I think what's needed is some sort of context in which the thought "group X are all criminals" is salient (even if only as a deliberate exaggeration). For someone who has strongly anti-black attitudes, that thought may be salient all the time when X = black people. For someone who's just heard about some statistics saying that pianists commit a bit more crime, it's probably salient enough because they've specifically been thinking about pianists and crime. But e.g. a few years after that criminologists' conference, when everyone's aware that pianists commit a bit more crime but no one particularly hates pianists as a result or anything, I don't think they'd find the joke funny.

Comment by gjm on Notes on Innocence · 2024-01-31T02:21:16.338Z · LW · GW

I'm not sure whether the ambiguity you're referring to is (1) the black paint / black skin one (which I mention in my other comment) or (2) something else.

If (1), I flatly disagree that that ambiguity is essential to (or even relevant to) the joke. I think Alice is expecting Bob and Carol to understand "black" as meaning "for black people" right from the outset.

If (2), then I'm not sure whether you mean (a) something to do with the alleged criminality of black people, or (b) something else.

If (a), then I think you misunderstand what I'm doing with the comparison. (Also, I don't think "there is no ambiguity here" is a good way of describing the difference between the two jokes.)

If (b), then perhaps you could do me the favour of explaining more clearly what you have in mind, because in case (2b) I have clearly failed to grasp it.

I should justify "I think you misunderstand what I'm doing with the comparison". (Here I'm assuming we're in case 2a.) I'm not saying "the black-people version of the joke isn't funny, because the pianist version of the joke isn't funny and there are no relevant differences between them". I'm saying "since the pianist version of the joke is uncontroversially not-funny, any funniness in the black-people version of the joke must depend on what's different about the two versions of the joke" -- more specifically, I think it depends mostly on the idea that black people are criminals -- which is relevant because you claimed that (apparently as a matter of objective fact) the black-person version of the joke "is funny", and I don't think that's an accurate way to describe something whose funniness is completely dependent on particular ideas or attitudes that many people don't hold.

People who think that pianists are criminals would (I think) find the pianist version of the joke about as funny as people who think that black people are criminals find the black-people version. The difference isn't in (what I at least would call) the structure of the joke, it's in the context that makes certain aspects of the structure salient.

Comment by gjm on Notes on Innocence · 2024-01-31T02:07:28.693Z · LW · GW

By "couldn't" I didn't mean "would be physically incapable of", I meant "certainly wouldn't", which is what I took your "would not be ..." to be saying. "Usually wouldn't", I have no disagreement with.

Comment by gjm on Processor clock speeds are not how fast AIs think · 2024-01-31T01:07:59.357Z · LW · GW

I think that when you say

My understanding is that it is not true that if you ran computers for a long time that they would beat the human also running for a long time

(which I don't disagree with, btw) you are misunderstanding what Ege was claiming, which was not that in 1997 chess engines on stock hardware would beat humans provided the time controls were long enough, but only that in 1997 chess engines on stock hardware would beat humans if you gave the chess engines a huge amount of time and somehow stopped the humans having anything like as much time.

In other words, he's saying that in 1997 chess engines had "superhuman but slower-than-human performance": that whatever a human could do, a chess engine could also do if given dramatically more time to do it than the human had.

And yes, this means that in some sense we had superhuman-but-slow chess as soon as someone wrote down a theoretically-valid tree search algorithm. Just as in some sense we have superhuman-but-slow intelligence[1] since someone wrote down the AIXI algorithm.

[1] In some sense of "intelligence" which may or may not be close enough to how the term is usually used.

I feel like there's an interesting question here but can't figure out a version of it that doesn't end up being basically trivial.

  • Is there any case where we've figured out how to make machines do something at human level or better if we don't care about speed, where they haven't subsequently become able to do it at human level and much faster than humans?
    • Kinda-trivially yes, because anything we can write down an impracticably-slow algorithm for and haven't yet figured out how to do better than that will count.
  • Is there any case where we've figured out how to make humans do something at human level or better if we don't mind them being a few orders of magnitude slower than humans, where they haven't subsequently become able to do it at human level and much faster than humans?
    • Kinda-trivially yes, because there are things we've only just very recently worked out how to make machines do well.
  • Is there any case where we've figured out how to make humans do something at human level or better if we don't mind them being a few orders of magnitude slower than humans, and then despite a couple of decades of further work haven't made them able to do it at human level and much faster than humans?
    • Kinda-trivially no, because until fairly recently Moore's law was still delivering multiple-orders-of-magnitude speed improvements just by waiting, so anything we got to human level >=20 years ago has then got hugely faster that way.
Comment by gjm on Notes on Innocence · 2024-01-31T00:49:20.592Z · LW · GW

I agree that someone who behaves like Bob is almost certainly being performatively-fake-innocent, but I think you're wrong to say that someone unfamiliar with the stereotypes couldn't behave that way. For one thing, Bob-as-portrayed isn't impervious to explanation or flatly uninterested in understanding. He asks for explanations and doesn't really get them, and if he ends up not understanding it's mostly because Alice hasn't really tried to help him understand (perhaps because Alice thinks, as you do, that he can't be sincere).

If someone told the "abortion clinic for pianists" version of the joke in my other comment in this thread, I can imagine responding very much like Bob. (Aside from the black paint / black people misunderstanding, which wouldn't have a parallel in that case.) I'd be assuming that there was some relevant thing about pianists that I didn't know, or some pun I was failing to detect, and I'm not sure I could do much better than "I still don't get it".

Comment by gjm on Notes on Innocence · 2024-01-31T00:42:58.955Z · LW · GW

Having the form of a joke is not sufficient to make something funny. I think you're right that David goes too far when he says it "isn't really a joke" -- it is really a joke -- but to whatever extent it's even meaningful to say "this is/isn't funny" without appending "to me" or "to the average 21st-century San Franciscan" or whatever, you can't refute "it isn't funny" just by saying that the thing is joke-shaped.

Suppose it had been "What do you call an abortion clinic for pianists?" with the same punchline. There would be the exact same structure, the exact same "subverting by means of semantic ambiguity" at the end. But I am fairly sure that essentially no one in the world would find it funny. And the only difference between this version and the one in the OP is that some people think black people are very often criminals and no one thinks that about pianists.

Maybe that's enough to make the joke funny for people who think black people are very often criminals. (I'm inclined to think not.) But I don't think you can claim that "of course it's funny" if its funniness depends on a belief that not everyone shares.

("But black people are more likely to be criminals than white people, I've seen the statistics!" Maybe so, but I don't think that's enough. Suppose it turns out that pianists are a bit more likely to be criminals than the general population; would that make the pianist version funny? Nope. I think the joke depends on equating "black people" and "criminals"; of course that doesn't mean that to find it funny you have to think all criminals are black and all black people are criminals, but I think you do need opinions that can round off to that; part of the humour, such as it is, comes from the exaggeration involved in doing so.)

Comment by gjm on Newton's law of cooling from first principles · 2024-01-17T12:13:20.816Z · LW · GW

Hmm. You're definitely right that my analysis (if it deserves so dignified a name) assumes all collisions are head-on, which is wrong. If "the x-axis" (i.e., the normal vector in the collision) is oriented randomly then everything still works out proportional to the kinetic energies, but as you say that might not be the case. I think this is basically the same issue as the possible bogosity of the "possibly-bogus assumption" in my original analysis.

Dealing with this all properly feels like more work than I want to do right now, though :-).

Comment by gjm on The case for training frontier AIs on Sumerian-only corpus · 2024-01-16T15:57:36.704Z · LW · GW

This is a terrible idea because as we all know Sumerian is uniquely suited for hacking humans' language-interpretation facilities and taking control of their brains.

Comment by gjm on Newton's law of cooling from first principles · 2024-01-16T14:56:21.174Z · LW · GW

(Disclaimer: I am a mathematician, not a physicist, and I hated the one thermodynamics course I took at university several decades ago. If anything I write here looks wrong, it probably is.)

Let's try to do this from lower-level first principles.

The temperature is proportional to the average kinetic energy of the molecules.

Suppose you have particles with (mass,velocity)  and  and they collide perfectly elastically (which I believe is basically what happens for collisions between individual molecules). The centre-of-mass frame is moving with velocity  relative to the rest frame, and there the velocities are  and . In this frame, the collision simply negates both velocities (since this preserves the net momentum of zero and the net kinetic energy) so now they are  and , so back in the rest frame the velocities are  and . So the first particle's kinetic energy has changed by (scribble scribble)  .

(I should say explicitly that these velocities are vectors and when I multiply two of them I mean the scalar product.)

Now, the particles with which any given particle of our cooling object comes into contact will have randomly varying velocities. I think the following may be bogus but let's suppose that the direction they're moving in is uniformly random, so the distribution of v2 is spherically symmetrical. (It may be bogus because it seems like two particles are more likely to collide when their velocities are opposed than when they are in the same direction, and if that's right then it will induce a bias in the distribution of v2.) In this case, the v1 v2 term averages out to zero and for any choice of |v2| we are left with constant . (KE2 - KE1). So as long as (1) our possibly-bogus assumption holds and (2) the rate at which object-particles and environment-particles interact isn't changing, the rate of kinetic energy change should be proportional to the average value of KE2-KE1, which is to say proportional to the difference in termperatures.

This is the (unmodified) Newton cooling law.

So if my calculations are right then any wrongness in the Newton law is a consequence of assumptions 1 and 2 above failing in whatever ways they fail.

I think assumption 2 is OK provided we keep the environment at constant temperature. (Imagine an environment-molecule somewhere near the surface of our object. If it's heading towards our object, then how long it takes before it collides with an object-molecule depends on how many object-molecules there are around, but not on how they're moving.)

I am suspicious of this "Lambert's law". Suppose the environment is at absolute zero -- nothing is moving at all. Then "Lambert's law" says that the rate of cooling should be infinite: our object should itself instantly drop to absolute zero once placed in an absolute-zero environment. Can that be right?

Comment by gjm on If Clarity Seems Like Death to Them · 2024-01-12T01:17:04.911Z · LW · GW

I am not (deliberately or knowingly) strawmanning anything, and what you call "doubling down" I call "not having been convinced by your arguments". If you think tailcalled was doing something more heinous than (1) having purposes other than advancing the discussion here and (2) not going out of his way to say so, then maybe you should actually indicate what that was; your accounts of his alleged dishonesty, so far, look to me like (1) + (2) + your disapproval, rather than (1) + (2) + something actually worse than 1+2.

If "the problem is at the level of his character" then I do not think there is any realistic chance that complaining about his character will do anything to solve the problem.

Have you ever seen any case where a substantial improvement to someone's character came about as a result of someone telling them on an internet forum what a bad person they were? I don't think I have.

At this point I shall take habryka's advice and drop this discussion. (Not only because of habryka's advice but because I agree with him that this conversation seems unlikely to be very productive, and because the LW user interface -- deliberately -- makes it painful to take part in discussions downthread of highly-downvoted comments.) I will not be offended if you choose to get in the last word.

Comment by gjm on If Clarity Seems Like Death to Them · 2024-01-09T02:38:19.753Z · LW · GW

I am deeply unconvinced by the argument "Some time after writing X, tailcalled said he said it partly to do Y; it's very unclear how X could possibly do Y; therefore when tailcalled wrote X he did it under false pretenses". It certainly does seem to follow from those premises that tailcalled's account of why he did X isn't quite right. But that doesn't mean that when he wrote X there was anything dishonest going on. I actually think the most likely thing is that he didn't in fact write X in order to do Y, he just had a vague notion in his mind that maybe the discussion would have effect Y, and forgot that he hadn't so far got round to saying anything that was likely to do it. Never attribute to malice what is adequately explained by incompetence.

(Not very much incompetence. This sort of discussion is easy to lose track of.)

And, again, it is not "false pretenses" to engage in a discussion with more than one goal in mind and not explicitly lay out all one's goals in advance.

an evasiveness that he does not typically display

Oh. I'd thought you were mostly alleging persistent character flaws rather than one-off things. Anyway: I won't say it's impossible that what you say is true, but I am so far unconvinced.

I cannot effectively criticise a community without criticising its members

Perhaps I have been unclear about what it is I think you have been doing in this thread that it would be better not to do. I am not objecting to criticizing people's behaviour. (I think I disagree with many of your criticisms, but that's a separate matter.) What I think is both rude and counterproductive is focusing on what sort of person the other person is, as opposed to what they have done and are doing. In this particular thread the rot begins with "thus flattering your narcissism" -- I don't agree with all your previous criticism of tailcalled but it all has the form "you did X, which was bad because Y", which I think is fine; but at this point you switch to "and you are a bad person". And then we get "you've added one more way to feel above it all and congratulate yourself on it" and "your few genuine displays of good faith" and "goal-oriented towards making you appear as the sensible moderate" and "you have a profound proclivity for bullshitting" and so forth.

I think this sort of comment is basically never helpful. If what you are trying to do here is something that can't be done without this sort of comment, then I think it would be better not to do it . (More precisely: if you think that what you are trying to do here is something that can't be done without such comments, then I think you are probably wrong unless what you are trying to do is mostly "make tailcalled feel bad" or something.)

Comment by gjm on How to find translations of a book? · 2024-01-08T22:16:07.816Z · LW · GW

Probably-obvious question: you mention finding one of them in a library, but have you tried asking the librarians there for help? They might have more relevant expertise than the readership of Less Wrong...

Comment by gjm on Bayesians Commit the Gambler's Fallacy · 2024-01-08T17:39:45.983Z · LW · GW

If I'm understanding the paper correctly -- and I've only looked at it very briefly so there's an excellent chance I haven't -- there's an important asymmetry here which is worth drawing attention to.

The paper is concerned with two quite specific "Sticky" and "Switchy" models. They look, from glancing at the transition probability matrices, as if there's a symmetry between sticking and switching that interchanges the models -- but there isn't.

The state spaces of the two models are defined by "length of recent streak", and this notion is not invariant under e.g. the prefix-XOR operation mentioned by James Camacho.

What does coincidental evidence for Sticky look like? Well, e.g., 1/8 of the time we will begin with HHHH or TTTT. The Sticky:Switchy likelihood ratio for this, in the "5-step 90%" model whose matrices are given in the OP, is 1 (first coin) times . 58/.42 (second coin) times .66/.34 (third coin) times .74/.26 (fourth coin), as the streak builds up.

What does coincidental evidence for Switchy look like? Well, the switchiest behaviour we can see would be HTHT or THTH. In this case we get a likelihood ratio of 1 (first coin) times .58/.42 (second coin) times .58/.42 (third coin) times .58/.42 (third coin), which is much smaller.

It's hard to get strong evidence for Switchy over Sticky, because a particular result can only be really strong if it was preceded by a long run of sticking, which itself will have been evidence for Sticky.

This seems like a pretty satisfactory explanation for why this set of models produces the results described in the paper. (I see that Mlxa says they tried to reproduce those results and failed, but I'll assume for the moment that the correct results are as described. I wonder whether perhaps Mlxa made a model that doesn't have the multi-step streak-dependence of the model in the paper.)

What's not so obvious to me is whether this explanation makes it any less true, or any less interesting if true, that "Bayesians commit the gambler's fallacy". It seems like the bias reported here is a consequence of choosing these particular "sticky" and "switchy" hypotheses, and I can't quite figure out whether a better moral would be "Bayesians who are only going to consider one hypothesis of each type should pick different ones from these", or whether actually this sort of "depends on length of latest streak, but not e.g. on length of latest alternating run" hypothesis is an entirely natural one.

Comment by gjm on If Clarity Seems Like Death to Them · 2024-01-08T17:07:37.091Z · LW · GW

Well, maybe I'm confused about what tailcalled's "original comment" that you're complaining about was, because looking at what I thought it was I can't see anything in it that anyone could possibly expect to convince anyone that Blanchardians are abusive. Nor much that anyone could expect to convince anyone that Blanchardians are wrong, which makes me suspect even more that I've failed to identify what comment we're talking about. But the only other plausible candidate I see for the "original comment" is this one, which has even less of that sort. Or maybe this one, which again doesn't have anything like that. What comment do you think we are talking about here?

I am fairly sure my opinions of tailcalled's responses here is very similar to my opinion of his comments elsewhere which haven't (so far as I've noticed) involved you at all, so I don't find it very plausible that those opinions are greatly affected by the fact that on this occasion he is arguing with someone I'm finding disagreeable.

"Pointing out character flaws". "Insults". Po-TAY-to. Po-TAH-to. My complaint isn't that the way in which you are pointing out tailcalled's alleged character flaws is needlessly unpleasant, it's that you're doing it at all. (And I would say the same if tailcalled were spending all his time pointing out your alleged character flaws, whatever those might be, but he isn't.) As far as I am concerned, when an LW discussion becomes mostly about the character of one of its participants, it is very unlikely that it is doing any good to anyone. And if what you mostly want to do here is point out people's character flaws, then even if those character flaws are real I think it's probably not very helpful.

It doesn't look to me as if LW is the hotbed of "constant abuse" you are trying to portray it as (and no, I'm not trying to insist that "constant" has to mean "literally nonstop" or anything). It looks to me -- and here I'm going off my own impression, not e.g. anything tailcalled may have said about the situation -- as if Zack gets plenty of disagreement on LW but very little abuse. So to whatever extent your "accusations of injustice" are of the form "tailcalled denies that Zack is constantly being abused, but he is", I find myself agreeing with tailcalled more than with you. Again, this was already my impression, so it can't be a halo/horns thing from this conversation.

(Of course, you may have me pigeonholed as one of the "crowd of enablers". Maybe you're right, though from my perspective I'm pretty sure I'm not abusing anyone and have no intention or awareness of engaging in the specific catch-22 you describe. I have disagreed with Zack from time to time, though.)

Comment by gjm on If Clarity Seems Like Death to Them · 2024-01-08T04:36:06.269Z · LW · GW

I am not persuaded by any part of your analysis of the situation.

Saying something relevant to an ongoing discussion (which it seems clear to me tailcalled's original comment was) while also hoping it will be persuasive to someone who has disagreed with you about something else is not "false pretenses".

It is certainly true that I am put off by your disagreeable manner. I do not think this is the halo effect. Finding unpleasantness unpleasant isn't the halo/horns effect, it's just what unpleasantness is; as for any opinions I may form, that's a matter of reasoning "if Cornelius had good arguments I would expect him to use them; since he evidently prefers to insult people, it is likely that he doesn't have good arguments". Of course you might just enjoy being unpleasant for its own sake, in which case indeed I might underestimate the quality of the arguments or evidence you have at your disposal; if you want me (or others who think as I do) not to do that, I suggest that you try actually presenting said arguments and evidence rather than throwing insults around.

It doesn't look to me as if tailcalled is being evasive; if anything he[1] seems to me to be engaging with the issues rather more than you are. (Maybe he's being evasive in whatever other venues your Drama is spilling over from; I have no way of knowing about that.) In any case, evasiveness doesn't compel insults. There is no valid inference from "tailcalled is being evasive" to "I must insult devote a large fraction of what I say to tailcalled to insulting him".

[1] I actually have no idea of tailcalled's gender; I'm going along with your choice of pronoun. In the unlikely (but maybe less unlikely in this particular sort of context) event that this is leading my astray, my apologies to tailcalled.

It does not look to me as if your repeated insultingness towards tailcalled is a necessary consequence (or in fact any sort of consequence) of having to keep pulling the conversation back to something he is avoiding talking about. (I'm not sure what it is that you think he is avoiding talking about. Maybe it's How Terrible Tailcalled Is, but in that case I don't think you get to say "I'm only being insulting to tailcalled because he keeps trying to make the conversation be about something other than how awful he is".)

Comment by gjm on If Clarity Seems Like Death to Them · 2024-01-08T00:33:09.132Z · LW · GW

I haven't followed whatever Drama may be going on between you and tailcalled elsewhere, but I don't see anything manipulative or under-false-pretenses about what you're complaining about here.

(And, for what it's worth, reading this thread I get a much stronger impression of "importing grudges from elsewhere" from you than from tailcalled.)

Comment by gjm on If Clarity Seems Like Death to Them · 2024-01-07T03:06:04.803Z · LW · GW

I would find this discussion more enlightening and more pleasant to read if you would focus on the issues rather than devoting so much of what you write to saying what a bad person you think tailcalled is.

Of course there's no particular reason why you should care what I find enlightening or pleasant, so let me add that one strong effect of the large proportion of insults in what you write is that it makes me think it more likely that you're wrong. (Cf. this old lawyers' saying.)

Comment by gjm on The Act Itself: Exceptionless Moral Norms · 2024-01-02T19:05:11.579Z · LW · GW

Perhaps your flashlight-clobbering story is permissible according to Aquinas (though I find his account of things insufficiently precise to tell), but it is definitely not permissible according to the accounts described in the SEP entry you linked to. You are protecting your house by means of injuring the intruder, and those accounts say explicitly that the DDE says that if you're forbidden to make bad thing X happen then you are likewise forbidden to make bad thing X happen in order to achieve good consequence Y of X. What you're allowed to do is to do something that produces both Y and X for the sake of Y, but Y has to happen in a way that isn't a consequence of X.

Perhaps you can claim that actually the protection of your house isn't happening by means of injuring the intruder. (You're just trying to scare him off and cause him some pain, and it's just too bad that in the process you fractured his skull[1].) And maybe in that case your action is in fact permissible according to the DDE. But, once again, I say: if so, so much the worse for the DDE. The permissibility of clobbering an intruder with a flashlight and predictably injuring him should not depend on whether there's some other mechanism by which your action might have protected your house, when you know that in fact that's not how it's going to play out.

[1] I'm tweaking the scenario a little in order to make the harm done something that's more plausibly Something Not Allowed.

If your intention was just to scare him off and cause him some pain, and you weren't at all expecting to end up fracturing his skull, then I don't think you need the fancy double-effect stuff: you can just say that you're not morally liable for unexpected, unpredicted consequences of your actions, provided your expectations and predictions were arrived at in good faith.

It is very possible that I'm missing something here. I have the impression that you have studied this stuff quite a lot more than I have, so if I think you're completely wrong about something fairly basic then I should take very seriously the possibility that the error is actually mine. But I don't see what my error might be here. Do you disagree that in your scenario you're getting the good effect (safe home) by means of the bad one (injured intruder)? Or do you disagree that the analyses of the DDE in the SEP say you're not allowed to do that? Or what?

I think it's worth distinguishing "has an explicit discussion of double-effect" from "has a special way of handling double-effect cases". As you say, standard-issue Benthamite utilitarianism has a way of dealing with such cases (i.e., just put them through the same machinery as all other cases and turn the handle) even though it doesn't do anything special about them. Isn't that likewise true for Aristotle, Kant, et al?

I am not an expert on either Aristotelian or Kantian ethics. But my crude model of Aristotle says: ethics is about being a person of good character, and what you should do about this sort of puzzle is to become a person of good character and then act in whatever way that leads you to act. And my crude model of Kant says: ethics is about obeying moral laws whose universal application you endorse, and what you should do about this sort of puzzle is to find moral laws whose universal application you endorse and then do what they say. Neither of these in itself resolves the puzzle, but then the same is true for Bentham's principle of utility (you should take whichever action leads to the greatest net excess of pleasure over pain). All of them provide tools you can use to pick an answer. None of them says that you need some sort of special-case rule just for these situations. Benthamite utilitarianism arguably has the merit of giving (in principle) a more specific and well-defined answer.

As for what exactly should be considered part of a person's "intent", I think I prefer not to think of things in those terms; I am not convinced that the difference between "I intended to knock the intruder out and thereby protect my home" and, say, "I intended to hit the intruder in a way that would stop him further invading my home, and correctly anticipated that the way this would happen is that I would knock him out, but I didn't intend that specific consequence" is very clear-cut, nor that it should be morally significant. I would prefer a different analysis: as you contemplate what you might do about the intruder, you anticipate various consequences and have whatever attitudes toward them you have, and I think those attitudes can be morally significant. Did you know you were going to knock him out or not? Were you hoping to hurt him, or hoping to cause him as little pain as possible? How did these predictions and attitudes influence your choice? Etc. If, say, you predicted that your action would knock him out and possibly cause lasting brain damage, and you tried to strike him in a way that would maximize the chance of unconsciousness and subject to that minimize the chance of permanent damage, then all of that is relevant, but whether we classify your action as "intended to knock him out" or "intended to protect your home" is not.

Comment by gjm on AI Is Not Software · 2024-01-02T13:14:58.556Z · LW · GW

An AI system is software in something like the same way a human being is chemistry: our bodies do operate by means of a vast network of cooperating chemical processes, but the effects of those processes are mostly best understood at higher levels of abstraction, and those don't behave in particularly chemistry-like ways.

Sometimes our bodies go wrong in chemistry-based ways and we can apply chemistry-based fixes that kinda work. We may tell ourselves stories about how they operate ("your brain doesn't have enough free serotonin, so we're giving you a drug that reduces its uptake") but they're commonly somewhere between "way oversimplified" and "completely wrong".

Sometimes an AI system goes wrong in code-level ways and we can apply code-based fixes that kinda work. We may tell ourselves stories about how they operate ("the training goes unstable because of exploding or vanishing gradients, so we're introducing batch normalization to make that not happen") but they're commonly somewhere between "way oversimplified" and "completely wrong".

But most of the time, if you want to understand why a person does certain things, thinking in terms of chemistry won't help you much; and most of the time, if you want to understand why an AI does certain things, thinking in terms of code won't help you much.

Comment by gjm on The Act Itself: Exceptionless Moral Norms · 2024-01-02T02:16:33.736Z · LW · GW

It seems like you're defining "double-effect" broadly enough to say that "greater-good Benthamite utilitarianism" gives an account of it (albeit one you find unsatisfactory), but also saying that Thomistic ethics is "the one which has discussion of double-effect" (emphasis mine). This seems odd to me.

It seems to me that what Thomistic ethics distinctly has is a particular way of handling cases where an action has both positive and negative consequences. If what you're demanding from your starting point is that it has specifically the Thomistic understanding of such cases then indeed you'd better start from Thomism, but only because you're presupposing it. If what you're demanding is that it is able to reason about such cases then it seems to me that literally any account of ethics will quality. I'm guessing that you're looking for something intermediate, but I'm not sure what. E.g., maybe you want a system that has some special-cased way of dealing with such cases but don't mind much what it is, or something. I don't feel that I have a good grasp of what your desideratum actually is here.

If you believe I am misunderstanding what you mean by "intent", or what Thomistic ethics means by "intent", or something of the sort, perhaps you could clarify what you consider to be the right meaning[1] and/or be more explicit about what you think I am getting wrong and why you think that?

[1] If it's specifically your meaning of "intent" that you think I'm getting wrong, then of course what you consider to be the right meaning is by definition the right meaning.

(I don't think it means "a statement one makes to oneself", and I'm not sure why you'd think I think it means that. But I do think that one can too easily fool oneself about what one intends, in part by making statements to oneself.)

Comment by gjm on The Act Itself: Exceptionless Moral Norms · 2024-01-01T21:22:13.337Z · LW · GW

Talk of "the standard Thomistic framework" is something of a red flag for me: for whom is the Thomistic framework standard? For some subset of the Roman Catholic Church, I guess, though my understanding is that the RCC itself explicitly doesn't endorse Thomism as such. Does anyone outside the RCC have good reason to think of Thomism as a natural starting point?

The whole "double effect" thing has always seemed to me to be a big pile of confusion. I'm not sure it's necessarily confused itself, but it encourages confused thinking.

For instance, consider your example of clobbering an intruder with your flashlight. You say you don't intend to knock him out, but you do intend to make your home safe, and your action only makes your home safe by means of knocking him out. This is in fact a nice clean example of something that is not permitted by traditional formulations of the doctrine of double effect, at least not if knocking the intruder out is considered bad. (E.g., in the first two formulations of the doctrine the SEP page gives, the first says "the good effect must be produced directly by the action, not by the bad effect" -- but if clobbering the intruder with the flashlight makes your home safe, it does so by means of knocking him out. The second lists four conditions of which the third is "that the good effect not be produced by means of the evil effect"; same again.)

My understanding is that if you hit the intruder with the intention that you'll knock him out and thereby render your home safe, this is forbidden according to the DDE; but if you are able to convince yourself that what you're doing is hitting him in order to encourage him to run away, then even if in fact you know that you will knock him out and that this is how the blow will actually keep your home safe then you are OK. And this seems to me like (1) mere sophistry and (2) a positive encouragement to self-deception.

(I'm not sure whether traditional RC thought would consider knocking the intruder out to be the sort of harm that you need to do this kind of reasoning about. But if not, we can just tweak our hypothetical a bit and suppose that a blow hard enough to knock him out has a non-negligible probability of actually killing him.)

The inclusion of this sort of thing in "the standard Thomistic framework" seems to me a strong reason for not starting from that framework, unless you have some sort of prior commitment to it.

I think it's entirely plausible that we'd do well to adopt some exceptionless moral norms for essentially the reasons you describe, but I don't see any good reason to think that if we do so we should try to fit them into the Thomistic framework, "double effect" and all.

I might, of course, be wrong about "double effect". Maybe it makes more sense than I think it does. Maybe it's less an incentive to self-deception and other bad thinking than I think it is. But I suggest that (1) if there isn't a particular reason to think that that specific idea is key to thinking straight about the question of exceptionless moral norms, then you'd do better to separate them, so that people who don't embrace the "double effect" idea may still be able to offer some insight, and (2) if there is a particular reason to think that, then you should say what it is, so that the rest of us can respond appropriately.

Comment by gjm on The proper response to mistakes that have harmed others? · 2024-01-01T12:45:36.781Z · LW · GW

Forgiveness = forgiven-ness = having been forgiven by someone.

Usually, if you harm someone, you care about being forgiven by them. And surely that isn't something they can just automatically be deemed to have done.

If you have wronged someone, and then fixed all that can be fixed and learned all that can be learned and sworn not to do the like again, then perhaps you're entitled to claim forgiveness (just as after the initial wrong your victim might be entitled to claim compensation from you), but if in fact they don't forgive you then you aren't forgiven even if you are entitled to be forgiven (just as if you'd refused to compensate them then they wouldn't in fact have been compensated despite their entitlement) and this seems like an important distinction.

There might be other entitles that could be known to have forgiven you "automatically" if you've done those things. Maybe every harm done hurts society-as-a-whole, and society-as-a-whole has somehow decided that anyone who fixes and learns and swears appropriately is forgiven. Maybe every harm done is an insult to the gods, and the gods have revealed an unchangeable divine commitment to forgive anyone who fixes and learns and swears appropriately. But that would be on top of, not instead of, whatever forgiving your actual victim might do or not do.

As Said observes in response to a related objection from Ben, arguably this is mostly a disagreement about words. If you say "they have forgiven me but don't acknowledge it" and I say "they are obliged to forgive you but haven't actually done it", maybe there isn't an actual difference in consequences. But I think "X has forgiven me" and "X is obliged to have forgiven me" suggest quite different states of affairs and one is nearer the truth than the other.

Comment by gjm on NYT is suing OpenAI&Microsoft for alleged copyright infringement; some quick thoughts · 2023-12-30T03:10:43.920Z · LW · GW

I think "humans are people and AIs aren't" could be a perfectly good reason for treating them differently, and didn't intend to say otherwise. So, e.g., if Mikhail had said "Humans should be allowed to learn from anything they can read because doing so is a basic human right and it would be unjust to forbid that; today's AIs aren't the sort of things that have rights, so that doesn't apply to them at all" then that would have been a perfectly cromulent answer. (With, e.g., the implication that to whatever extent that's the whole reason for treating them differently in this case, the appropriate rules might change dramatically if and when there are AIs that we find it appropriate to think of as persons having rights.)

Comment by gjm on Will 2024 be very hot? Should we be worried? · 2023-12-29T17:12:49.583Z · LW · GW

Well, one possibility is (1) that the article got it terribly wrong. But to my ignorant eye there are at least two others. (2) Perhaps water vapour in the stratosphere stays around for much longer than water vapour in the troposphere. (And most water vapour is in the troposphere, so any sort of average figure will be dominated by that.) (3) Your link says that an average molecule of water stays in the atmosphere for 9 days, but that isn't the same as saying that a change in the amount of water will only persist for that long; maybe there is a constant exchange of water molecules that leaves amounts roughly unchanged, so that if you put 2.3 metric fucktons of extra water into the atmosphere then a month later there will still be 2.3 metric fucktons of excess water but the specific water molecules will be different.

Perhaps someone who knows some actual climatology can tell us how plausible 1,2,3 are.

Here's the paper I think everyone claiming years is referencing: https://agupubs.onlinelibrary.wiley.com/doi/full/10.1029/2022GL099381. That in turn references https://agupubs.onlinelibrary.wiley.com/doi/10.1029/97GL01289 for which I can see only the abstract, which says "tau=1.3 years". If tau has the usual meaning (time to decay by a factor of e) then 5 years would be roughly the time for a 20x decay; but there may be more details that make the "5-10 years" figure less misleading than that makes it sound (e.g., upper versus lower stratosphere -- the water vapour from the recent eruption went a long way up).

Comment by gjm on NYT is suing OpenAI&Microsoft for alleged copyright infringement; some quick thoughts · 2023-12-28T17:35:11.157Z · LW · GW

I don't say that the same policies must necessarily apply to AIs and humans. But I do say that if they don't then there should be a reason why they treat AIs and humans differently.

Comment by gjm on NYT is suing OpenAI&Microsoft for alleged copyright infringement; some quick thoughts · 2023-12-28T17:27:37.535Z · LW · GW

If in the specific case of NYT articles the articles in question aren't intended to be publicly accessible, then this isn't just a copyright matter. But the OP doesn't just say "there should be regulations to make it illegal to sneak around access restrictions in order to train AIs on material you don't have access to", it says there should be regulations to prohibit training AIs on copyrighted material. Which is to say, on pretty much any product of human creativity. And that's a much broader claim.

Your description at the start of the second paragraph seems kinda tendentious. What does it have to do with anything that the process involves "arrays of numbers"? In what sense do these numbers "represent the work process behind the copyrighted material"? (And in what sense if any is that truer of AI systems than of human brains that learn from the same copyrighted material? My guess is that it's much truer of the humans.) The bit about "increase the likelihood of ... producing the copyrighted material" isn't wrong exactly, but it's misleading and I think you must know it: it's the likelihood of producing the next token of that material given the context of all the previous tokens, and actually reproducing the input in bulk is very much not a goal.

It may well be true that all progress on AI is progress toward our doom, but it's not obviously appropriate  to go from that to "so we should pass laws that make it illegal to train AIs on copyrighted text". That seems a bit like going from "Elon Musk's politics are too right-wing for my taste and making him richer is bad" to "so we should ban electric vehicles" or from "the owner of this business is gay and I personally disapprove of same-sex relationships" to "so I should encourage people to boycott the business". In each case, doing the thing may have the consequences you want, but it's not an appropriate way to pursue those consequences.

Comment by gjm on In Defense of Epistemic Empathy · 2023-12-27T21:17:09.633Z · LW · GW

I think you're (maybe deliberately) being unfair to David Lewis here.

I'm pretty sure that if you'd asked him "Are there, in the actual world, talking donkeys?" he would have said no. Which is roughly what most people would take "Are there talking donkeys?" to mean. And if someone says "there are talking donkeys", at least part of why we think there's something unreasonable about that is that if there were talking donkeys then we would expect to have seen them, and we haven't, which is not true of talking donkeys that inhabit other possible-but-not-actual worlds.

David Lewis believed (or at least professed to believe; I think he meant it) that the things we call "possible worlds" are as real as the world we inhabit. But he didn't think that they are the actual world and he didn't think that things existing in them need exist in the actual world. And I think (what I conjecture to have been) his actual opinion on this question is less absurd and more reasonable than the opinion you say, or at least imply, or at least insinuate, that he held.