Teach People to Recognize the Sound of Covid?

post by jefftk (jkaufman) · 2020-11-06T14:30:17.177Z · LW · GW · 8 comments

Contents

8 comments

Yesterday I read Zvi's 11/5 Covid update which had:

Researchers claim to have a machine learning system that can diagnose Covid-19 over the phone by analyzing recordings of a forced cough (pdf). They claim sensitivity of 98.5% and specificity of 94.2%, and for asymptomatic cases a sensitivity of 100% (?!?) and specificity of 83.2%. I'm curious to what extent errors in the test are correlated from day to day. This can all be offered at unlimited scale for essentially zero additional cost. So, of course, it will presumably be illegal indefinitely because it's not as good as accurate as a PCR test and/or hasn't gone through the proper approval process, and no one will ever use it. Then again, if one were to somehow download or recreate such a program and run it, who would know?

You can see how they collected data, and contribute your own cough at opensigma.mit.edu. It doesn't seem to me like they actually have great data on which recordings correspond to people who have it? For example, I submitted mine this morning, and said that I don't think I have Covid. If, however, I later learned that I did have Covid at the time I submitted the sample, there doesn't seem to be any way for me to tell them.

You could get better data, however, by collecting alongside regular Covid tests. Have everyone record a sample of a forced cough when you test them, label the cough with their results once you have it, and you end up with high-quality labeled samples. They trained their AI on 5,320 samples, but at current testing rates we could get 80k samples in a single day in just Massachusetts.

It might turn out that even with higher quality data you still end up with a test that is less accurate than the standard of care, and so are unable to convince the FDA to allow it it. (This is an unreasonable threshold, since even a less accurate test can be very useful as a screening tool, but my understanding is the FDA is very set on this point.) Is there another way we could scale out auditory diagnosis?

Very roughly, their system is one where you take lots of samples of coughs, labeled with whether you think they were produced by someone with coronavirus, and train a neural network to predict the label from the sample. What if instead of artificial neural networks, we used brains?

People are really smart, and I suspect that if you spent some time with a program that played you a sample and you guessed which one it was, and then were told whether you were right, you could learn to be quite good at telling them apart. You could speed up the process by starting with prototypical standard and Covid coughs, as identified by the AI, and then showing progressively borderline ones as people get better at it. In fact, I suspect many medical professionals who have worked on a Covid ward already have a good sense of what the cough sounds like.

I don't know the regulations around what needs a license, but I think there's a good chance that hosting a tool like this does not require one, or that it requires one that is relatively practical to get? If so, we could train medical professionals (or even the general public?) to identify these coughs.

Automated screening would be much better, since the cost is so much lower and it could be rolled out extremely widely. But teaching humans to discriminate would be substantially cheaper than what we have today, and with much weaker supply restrictions.

(I looked to see whether the researchers made their samples available, but they don't seem too. Listening to some would've been a great way to check how practical this seems.)

8 comments

Comments sorted by top scores.