Pretty much everything here seems wrong to me. Some comments, in rough order of appearance:

You call the EHT a multi-billion-dollar project. I don't think I believe you. Can you provide some actual figures?

You say that LIGO-VIRGO "filters their noisy data with a template of what they want to see". _Every_ kind of filtering can, with a sufficient lack of charity, be described that way. (E.g., even the most simple-minded moving-average filter amounts to saying that you're looking for signals with relatively little very high-frequency content, but you expect there to he high frequencies present in the noise.) There is nothing wrong with doing it, either; what matters is how you then analyse the results. If you think LIGO's analysis is wrong, you need to explain how it's wrong; making a complaint that amounts to "they filter their data" is no good; that's what everyone does and there's nothing wrong with it.

You say that it's circular reasoning if you say you've confirmed GR by using GR to construct theories and then checking that your observations match the theories. It's not circular reasoning at all, it's how science works. You take a theory, you put some effort into working out what the theory says you should see, and you look at whether you see that or not. Again, it's very possible to do that wrongly -- confirmation bias is a thing -- but a complaint that amounts to "they claimed to have confirmed a theory by doing experiments based on that theory" is no good; that's what everyone does and there's nothing wrong with it.

You say the gravitational wave community has exhibited a "lack of attention to earlier measurements", on the basis that earlier measurements claimed to have found black holes and turned out to be wrong, and LIGO/VIRGO isn't doing the _exact same thing_ that made it possible to check that the earlier claims were wrong, namely combining large numbers of independent verifications. But (1) your description of those earlier measurements doesn't match what's in the article you link to (you say hundreds of independent groups all thought they'd found GWs and they only discovered they were wrong when they combined their results; the article says _one_ researcher claimed to have found GWs, everyone else disagreed, and when they looked they found errors in his analysis), and (2) it is not always the case that when something goes wrong and gets fixed, next time around you should apply the exact same fix in advance; sometimes there are better ways. Repeating an experiment N times reduces the noise by a factor of sqrt(N) (at least for certain common kinds of noise) and there may be ways to reduce it more effectively per dollar spent.

You say LIGO fails to use "control variables". This is nonsense. Anything they don't vary is a control variable, and "using control variables" is not a virtue. What you're actually describing in the paragraph beginning "In a well-designed experiment" is a control _group_ or simply a _control_. Some experiments use controls, some don't; it's not clear to me what it would _mean_ to use a control in the case of LIGO, and it seems to me that you could consider _all the times it doesn't detect anything_ to constitute control measurements.

You say LIGO "had announced 50 detections" as of 2019-12 but as of 2020-12 "are only standing by 10 of those". But you don't quote what they actually said, or provide any links. The "Open data from the first and second observing runs of Advanced LIGO and Advanced Virgo" paper published on 2020-12-25 says that those runs produced "11 confident detections and 14 marginal triggers". That doesn't look to me like a claim of 50 detections. Could you please be more specific about what they claimed in 2019-12 and what they said in 2020-05? I am betting that if there is anything resembling a "50 detections" claim, it was something like "50 candidate events" and confirming only 10 of them is in no way evidence of anything wrong.

You say "They expected to see at least a few black hole collisions and at least one neutron star collision per month since August 2019. It has been 9 months". (Obvious implication: they aren't seeing what they said they would see.) According to https://www.ligo.caltech.edu/news/ligo20200326, when they suspended their third observing run near the end of 2020-03 (because of COVID-19) they had seen 56 detections (I don't know whether this means candidates, fully confirmed detections, or what) in the ~400 days of run 3. That's about four per month. Seems to fit their prediction just fine.

You complain, again, about LIGO's use of "templates" and compare a couple of graphs to show how different less-filtered data look compared with their published plots. But your plot purporting to be "the same data ... with only a whitening filter and a Fourier transform" is no such thing. Look at the y-axis label: "Cross-correlation". This is the cross-correlation between the Hanford and Livingston signals. It is nothing remotely like the raw data, nor should it be.

So far as I can see, in the plot you show ("This was what LIGO published and used to win a Nobel Prize")the data in the top frames is _not_ the result of any sort of template-filtering at all. (I think there's some bandpass filtering, which is absolutely routine, and that's it.)

You quote some accusations of "misconduct" on the basis that "pieces of that figure were illustrative and the detection claim is not based on that plot". The thing that's "illustrative" is the _second_ row in the "Nobel Prize" plot, and the point of the remark about how it's only "illustrative" is that the properly-done fit (which _was_ the basis for the detection claim) matches the observed signals _better_. The point of the "illustrative" plots is to let you see by eye that the actually-observed signals have the right sort of shape.

You object to the LIGO researchers' response to the Copenhagen objectors because they said (in your paraphrase) "you forgot to use an FFT windowing function even though that is a mistake no physicist would ever make". Well, sometimes physicists make mistakes you wouldn't think they would. The relevant question here is: _Did_ Cresswell et al fail to do it, or didn't they? Green and Moffatt say their results look like they did fail. I haven't seen any rebuttal to that.

You say that the LIGO researchers have no way to distinguish distant gravitational waves from other possible sources nearer-by. Well, obviously one can never prove that an observation doesn't come from some currently unknown source producing signals by currently unknown means, but that criticism applies equally to _all_ observational science. We think we see a supernova a long way off, but maybe it's actually some thing much nearer to us that _just happens_ to have happened exactly in between us and the star we think went supernova. Sure, it's possible, but we have a simpler explanation! And so it is for LIGO. If you have a specific alternative theory for what LIGO has been detecting, let's hear it. (You kinda-sorta, I assume mostly frivolously, mention a possible string of events. "After all, if something dark and mysterious happens in deep space and causes the sun to burp and that causes the Earth's core to gurgle and that causes a lake to heat up which causes a thunderstorm which causes a lightning strike which hits a Schumann resonance and LIGO detects that ...". But obviously that's not relevant because (1) it has no details that would enable us to tell what sort of observations such a sequence of events might produce and (2) something _that_ local would not produce results that would fool LIGO except by extreme coincidence; that's why they have multiple detectors thousands of miles apart.)

You complain that the black-hole events LIGO claims to have found "are absurdly larger than what they had expected to see based on other measurements and based on the theory of black holes". I would like some details substantiating that complaint. I remark that LIGO would _always_ tend to detect larger-mass black-hole events for the obvious reason that they produce stronger gravitational waves and LIGO needs to be staggeringly sensitive to detect anything at all.

Your theoretical objections to the whole mode of operation of LIGO looks all wrong to me, in multiple ways, but I am not a general-relativist and won't get into that particular argument (but I remark that if you were right, that seems like the sort of error that I would expect The Scientific Establishment to pounce on instantly, so the fact that LIGO is generally a respectable high-prestige operation is evidence against).

And, while we're talking about the actual physics, the following sentence seems to me like evidence of hopeless confusion (on your part, I'm afraid, not that of the scientific establishment): "there will be discontinuities between transverse and longitudinal waves which might be measured as a sort of friction coefficient [...] In olden times, one would call these discontinuities magnetic monopoles, but today, we call them all sorts of things -- displacement currents, positrons, electrons, matter, antimatter, ... black holes". I don't think this makes contact with reality at any point.

Somewhere around here, I lost the will to live, so I've paid less detailed attention to the end than to the beginning.

(I am very fallible and the chances are that there is at least one mistake in what I've written above. But there would need to be one hell of a lot of mistakes for your complaints about gravitational wave detection to be convincing to me.)

Thanks for presenting your thesis. However, one of your figures doesn't support your argument on closer inspection. The figure that you point to as being the 'unfiltered' data is measuring cross-correlation between the Hanford and Livingston datasets, so we should expect it to look completely different than the datasets themselves.

I also want to push back on a particular point - there's nothing wrong in principle with using a black-hole shaped filter to find black holes. You just have to adjust the prior based on the complexity of your filter.

A lot of this reads like you are trying to apply the structure of an experiment to a thing that is, um, not an experiment. Like, we all learn the steps of an experiment in school (where they often incorrectly call the experimental method "the scientific method"). But there are whole sciences, like astronomy, and cosmology, and geology, that don't do experiments, they just make observations and analyze them in the context of what we already know from experiments in other areas of science. That is what LIGO does. We can't do experiments on gravitational ways, because we don't have the capacity to produce gravitational waves. All we can do is observe them. And that is still a perfectly valid scientific endeavor. And in particular, it is a scientific endeavor in which the notion of a "control" doesn't seem to make a whole lot of sense. Now, I don't have the technical competence to evaluate these kinds of high level physics things for myself, I don't know the math of general relativity, so I'm not going to try. But I generally trust the scientific community, and I'm not going to update much on a blog post that seems to misunderstand what these things are trying to do.

So, you did mean the Bouman talk I found. As I say, she wasn't "the leader of that project" and she did not say what you say she did.

The particular things that you claim there are "absurd" are not absurd, it's just that you don't understand the procedures they describe and are taking them in the most uncharitable way possible.

(I haven't listened to the CalTech talk so can't comment with any authority on what Bouman meant by all the things you quote her as having said there, but it is absolutely not true that "any single one of the statements [] would disqualify an experiment", and amusingly the single statement you choose to attack there at greatest length is the most obviously not-disqualifying. You say, and I quote, "Most sensible researchers would agree that if the resolution of your experiment is equivalent to taking a picture of an orange on the moon, this means that you cannot do your experiment.". You appear to be arguing that if something sounds impossibly hard, then you should just assume that it is, literally, impossibly hard and that it can never be done. Once upon a time, "equivalent to speaking in New York and being heard in Berlin" would have sounded like it meant impossibly hard. Once upon a time, "equivalent to adding up a thousand six-digit numbers correctly in a millisecond" would have sounded like it meant impossibly hard. Some things that sound impossibly hard turn out to be possible. The EHT folks claim that taking a picture with orange-on-the-moon resolution turns out to be possible. Of course they could be wrong but they aren't obviously wrong; what they're claiming breaks no known laws of physics, for instance. And obviously they aren't unaware that getting a picture of an orange on the moon is very difficult. So I think it's downright ridiculous to say that their project is unreasonable because they're trying to do something that sounds impossibly hard.)

Nope, not playing any more of that game. If you want to make a point, make it. If you want to hint vaguely that you're smarter than me by posing as Socrates, go ahead if you wish but don't expect my cooperation.

Since you seemingly can't defend nor withdraw your claim that EHT cost billions of dollars, a reasonable person can only assume that the rest of the factual content of your post is suspect.

Dustin's point, as I understand it, is not that you overestimated or that you underestimated, nor that you didn't give a detailed accounting of all the facilities involved, it's that you're confusing two completely different questions. (1) How much did the EHT project cost? (2) How much did the telescopes used by the EHT project cost to build and run? You made a claim about #1 and when challenged on it offered some numbers relating to #2.

You do say one thing that purports to link them: "... if EHT kept the telescopes in operation when they would've otherwise lost funding ...". But that's one heck of a big if and I know of no reason to think that EHT kept any telescopes in operation that would otherwise have lost funding. And even if it did, that wouldn't justify including the cost of building the telescopes in your estimate of the cost of EHT, unless the telescopes in question were never used for anything other than EHT.

(One journalistic outlet has given a concrete estimate for the cost of the EHT project. They say 50 to 60 million dollars. I don't know where they got that estimate or how much to trust it, but it sounds much much more believable to me than your "billions of dollars".)

There is a TED talk by (actually, an interview with, as part of TED2019) Sheperd Doeleman, head of the EHT collaboration, whose transcript you can read on the TED website. It doesn't say anything even slightly like that. Is there some other TED talk by her that you're referring to? (I can't find any evidence that there is another.)

The only other thing I can find that you conceivably might be referring to is a TEDx talk by Katie Bouman, from 2017 (before the EHT picture was produced). Her title is "How to take a picture of a black hole" and it includes a prediction of roughly what the picture might be expected to look like, and includes the words "my role in helping to take the first image of a black hole is to design algorithms that find the most reasonable image that also fits the telescope measurements". Maybe that's what you mean?

She doesn't say "exactly", or even approximately, that applying the same pipeline to random input would generate a similar result. Quite the reverse; let me quote her again. "What would happen if Einstein's theories didn't hold? We'd still want to reconstruct an accurate picture of what was going on. If we bake Einstein's equations too much into our algorithms, we'll just end up seeing what we expect to see. In other words, we want to leave the option open for there being a giant elephant at the centre of our galaxy." She says, in other words, that a key consideration in their work was not doing exactly what you say she said they did.

(Shortly after that bit there is a slide that, if wilfully misunderstood, might seem to fit your description. Its actual meaning is pretty much the reverse. I won't go into details right now because I don't know whether you saw that slide and misunderstood it; I don't know whether this is the TED talk you're referring to at all. But I guess this is it.)

Incidentally: Katie Bouman was a PhD student, was not an astronomer, and was certainly not the leader of the EHT project. The project was already happening and already funded, but I suppose you could call her talk "selling the project to the public" in the sense in which any attempt to describe anything neat one's doing is "selling the project". Bah.

Your descriptions of what I said in the comments on "The New Scientific Method" are not accurate. They are like your purported quotations from Katie Bouman's talk (though at least you didn't put them in quotation marks this time): in condensing what I actually said into a brief and quotable form, you have apparently attempted to make it sound as silly as possible rather than summarizing as accurately as possible. I think you shouldn't do that.

(My description in terms of "weirdness" was meant to help to clarify what is going on in an algorithm that you criticized but apparently hadn't understood well. It turns out that it was a mistake to try to be as clear and helpful as possible, rather than writing defensively so as to make it as difficult as possible for someone malicious to pick things that sound silly.)

I already told you (in comments on that other post) what motivates me: bad science, and especially proselytizing bad science, makes me sad. It makes me especially sad when it happens on Less Wrong, which aims to be a home for good clear thinking. Having seen the previous iteration of Less Wrong badly harmed by political cranks who exploited the (very praiseworthy) local culture of taking ideas seriously even when they are nonstandard or appear bad at a first glance, I am not keen to leave uncriticized a post that is confidently wrong about so many things.

I don't know what anyone else may have done, but I at least have not downvoted all your comments and posts. I have downvoted some specific things that seem to me badly wrong; that's what downvoting is meant for. (As it happens, it looks to me as if you have downvoted all my comments on your posts.)

I'm not arguing that the telescopes are useless

It did not seem like you were making such an argument, nor was I asserting that you were making such an argument.

The telescope could have cost umpteen trillions of dollars and that fact alone would not support your claim that EHT cost billions of dollars.

I'm not sure how to understand the fact that the previous statement is obvious and yet you still made your comments. I feel like the most charitable interpretation that I can come up with still does not leave a good impression of your overall argument.

I'm not harping on this apparent mistake for no reason. It's just that of all the things described by gjm this seems like it might be the easiest to explicate.

I didn't ask how much was spent on LIGO. I asked how much was spent on EHT. Those are very different projects.

(So I'm afraid everything you wrote above was irrelevant to the question I asked. I regret not making it clearer, though I confess I'm not quite sure how I could have made it clearer since what I wrote was "You call the EHT a multi-billion-dollar project. I don't think I believe you. Can you provide some actual figures?".)

Also, in case it wasn't obvious, the question about the cost of EHT was very much the least important part of what I wrote; obviously it doesn't make much difference to the rightness or wrongness of your claims about LIGO whether you got the size of the EHT project right or not.

So, you seem to continue to use a rhetorical device wherein you do not directly address the points that your interlocuters are bringing up and just answer the question you wish was asked.

For example, this comment I'm replying to here has almost zero bearing on what I said. Saying EHT is bad is not a way to address the argument that EHT did not cost billions of dollars. EHT may very well be bad, but that has no bearing on the subject at hand.

In your previous comment to me in this thread you did the same thing.

Whereas I think you should try harder to explain it, because it's not making any sense to me as a justification for your (plainly incorrect) claim about that figure and right now my leading hypothesis is that you just don't understand the mathematics and/or the physics involved well enough to see what's going on and are trying to obfuscate, and there is a (not very high) limit to how much trouble I am willing to go to to understand something that seems likely not to be worth understanding.

I could, of course, be wrong about this. As I already mentioned, I am very fallible. Feel free to convince me.

I might as well answer your question about chirped signals. If you have a signal that looks like $f\left(t\right)\sin (t+k{t}^2)$ where f is a slowly varying function (compared with the chirpy factor) then subtracting a slightly time-shifted copy of it gives you roughly the derivative, which when f varies slowly is roughly $f\left(t\right)(1+kt)\cos (t+k{t}^2)$, which is indeed a chirped signal that resembles the initial chirp albeit with some extra variation in amplitude. If you have a phase-shifted version available instead of a time-shifted one, the resemblance is closer because the $1+kt$ factor goes away. So yes, subtracting chirpy signals with a small shift gives you similar-ish chirpy signals. Now, how does this give any reason to think that that plot is a less-processed version of "the prize-winning figure"?

(I wish you wouldn't keep calling it "the prize-winning figure". Obviously Nobel Prizes are not in fact awarded for figures, and I do not believe you have any evidence for the implied claim that if the figure had looked different then the LIGO team wouldn't have won the Nobel Prize.)

I'm not sure what point you're now making; it looks to me as if it has nothing to do with what we were talking about before. Are you saying that the LIGO team should have used a different technique to identify gravitational wave events? If so, that claim requires much more evidence than "I thought of another way to do it". Or are you saying that some plot they made is in fact the result of subtracting two related signals with a phase shift and that this is some sort of sign of incompetence or fraud or something? Or what?

In any case, it seems like you've given up defending your claim that the plot from Ian Harry's article is some sort of "original" less-cleaned-up version of the plot you keep calling "the prize-winning figure". Which is just as well, because that claim is indefensible.

You may find it quite clear, but you are interpreting it quite wrong if you think it's some sort of less-processed version of "the prize-winning figure". It's not: it's something completely different.

I read the article. samshap is 100% right and you are 100% wrong.

[EDITED to add:] "What is asserted without evidence can be dismissed without evidence" -- but I might as well give some justification for my claim. Here is what the article says:

First I begin by cross-correlating the Hanford and Livingston data, after whitening and band-passing, in a very narrow 0.02s window around GW150914. This produces the following:

(followed by the graph you provide here). Note two things.

First: "I begin by cross-correlating the Hanford and Livingston data" -- just as samshap says.

Second: "in a very narrow 0.02s window". That's about 1/10 of the time period represented by the main plots, which go from 0.25s to 0.45s "relative to September 14, 2015 at 09:50:45 UTC" (not that we can tell from your presentation, because you clipped off the bottom part of the figure which includes the time axes). So this could not possibly be an alternative to the other plots; the horizontal axes aren't in any way compatible.

The context for this is that the (LIGO-skeptical) Cresswell et al paper is looking at the time lags between LIGO observations, and claiming to cast doubt on the idea that seeing two very similar signals at the two detectors at a certain time-lag is evidence of anything. So, in particular, Cresswell et al try to show that you can get the same 7ms lag by looking at other things without the actual signal in it. (One of the things they look at is the residual noise from the LIGO data, after subtracting off the black-hole-merger model. This is why it's relevant that the actual best-fit model is better than the "illustrative" one -- because if you subtract off a crude model, what remains will have some real signal in it, so it's unsurprising if it shows some of the same temporal correlations as the actual signal does.) So now Ian Harry shows the cross-correlation graph for the LIGO data before subtracting off the fitted model, and after subtracting the (best) fitted model. The graph you reproduce here is the cross-correlation before subtracting the model; the next one (not reproduced here) is the cross-correlation after subtracting the model, which shows no 7ms spike.

Note that the context makes excellent sense of having a cross-correlation graph at this point in the article, and would make no sense at all of having a raw-LIGO-observation-data graph instead.

Calculating opportunity costs is great, but that isn't what you did.

That is a way to make a rough estimate in the same way that providing the construction costs for a whole shopping mall is a way of providing a rough estimate of how much it costs for me to walk in the door of said mall.

In other words, there are too many unknowns and counterfactuals for that to even begin to be a useful way of calculating how much EHT cost.

In a way it's almost besides the point. You made the positive claim, seemingly without any solid facts, that it cost billions of dollars. When you were called on it, a way to increase the confidence of others in your arguments and presented facts would be to say something like "you know, I shouldn't have left that in there, I withdraw that statement".

By not doing so and sticking to your guns you increase the weight others give to the idea that you're not being intellectually honest.

Your current tack might be useful in political rhetoric in some quarters, but it doesn't seem like it will be effective with your current audience.

Figuring out past detections were false, seems like cases of trying to replicate earlier findings, i.e. doing things right.

LIGO has no way to distinguish its signature from some more mundane, local occurence.

Why not? This?:

If you detect a wave at two locations and you know its propagation speed, you can determine the direction from which the wave came, but not the distance to the thing that caused the wave.

APS journals charge two thousand dollars

Wow.

After putting effort into debunking this stuff, I do ask myself why I bother and I think that I want to demystify the pop-sci nonsense used to lure young people into physics servitude. I think they might find better things to do with their time and I’d like to help them avoid the mistakes I made.

An admirable goal.

If you would like to hear this post read aloud, try this video.

Meta: the video didn’t make it through the cross-posting, it seems. (I am not sure if Less Wrong supports video embedding; I think it may not. You might want to just link the video.)

I see. With regard to your post and the origin of the black hole "pic", do you believe that applying the same pipeline to random images or even noise would generate a similar result?

It's unclear if you're claiming that you have actual figures that show the EHT actually cost billions of dollars or if you're claiming that you think it's likely, but just a guess, that it kept all those radio telescopes "in business", or if you're taking back your claim that it cost billions of dollars.

Thanks for the interesting read. I absolutely lack the background to comment on your conclusions, but your post made me remember some questions I had on Black Holes that no physicist I talked to could answer, I never would have guessed the field had detractors.

If you don't mind me asking, are you also a climate change skeptic?