MetaMed: Evidence-Based Healthcare
post by Eliezer Yudkowsky (Eliezer_Yudkowsky) · 2013-03-05T13:16:45.807Z · LW · GW · Legacy · 193 commentsContents
193 comments
In a world where 85% of doctors can't solve simple Bayesian word problems...
In a world where only 20.9% of reported results that a pharmaceutical company tries to investigate for development purposes, fully replicate...
In a world where "p-values" are anything the author wants them to be...
...and where there are all sorts of amazing technologies and techniques which nobody at your hospital has ever heard of...
...there's also MetaMed. Instead of just having “evidence-based medicine” in journals that doctors don't actually read, MetaMed will provide you with actual evidence-based healthcare. Their Chairman and CTO is Jaan Tallinn (cofounder of Skype, major funder of xrisk-related endeavors), one of their major VCs is Peter Thiel (major funder of MIRI), their management includes some names LWers will find familiar, and their researchers know math and stats and in many cases have also read LessWrong. If you have a sufficiently serious problem and can afford their service, MetaMed will (a) put someone on reading the relevant research literature who understands real statistics and can tell whether the paper is trustworthy; and (b) refer you to a cooperative doctor in their network who can carry out the therapies they find.
MetaMed was partially inspired by the case of a woman who had her fingertip chopped off, was told by the hospital that she was screwed, and then read through an awful lot of literature on her own until she found someone working on an advanced regenerative therapy that let her actually grow the fingertip back. The idea behind MetaMed isn't just that they will scour the literature to find how the best experimentally supported treatment differs from the average wisdom - people who regularly read LW will be aware that this is often a pretty large divergence - but that they will also look for this sort of very recent technology that most hospitals won't have heard about.
This is a new service and it has to interact with the existing medical system, so they are currently expensive, starting at $5,000 for a research report. (Keeping in mind that a basic report involves a lot of work by people who must be good at math.) If you have a sick friend who can afford it - especially if the regular system is failing them, and they want (or you want) their next step to be more science instead of "alternative medicine" or whatever - please do refer them to MetaMed immediately. We can’t all have nice things like this someday unless somebody pays for it while it’s still new and expensive. And the regular healthcare system really is bad enough at science (especially in the US, but science is difficult everywhere) that there's no point in condemning anyone to it when they can afford better.
I also got my hands on a copy of MetaMed's standard list of citations that they use to support points to reporters. What follows isn't nearly everything on MetaMed's list, just the items I found most interesting.
90% of preclinical cancer studies could not be replicated:
http://www.nature.com/nature/journal/v483/n7391/full/483531a.html
193 comments
Comments sorted by top scores.
comment by Qiaochu_Yuan · 2013-03-05T00:49:32.184Z · LW(p) · GW(p)
It is fairly terrifying that the term "evidence-based medicine" exists because that implies that there are other kinds.
Replies from: ChristianKl, incogn, TraderJoe, buybuydandavis↑ comment by ChristianKl · 2013-03-05T23:29:47.383Z · LW(p) · GW(p)
LessWrong is a non-evidence-based method of teaching rationality. We don't have good evidence that someone will get more rational after reading the sequences.
You can make a reasonable theoretic argument that people will get more rational. You don't have the kind of evidence that you need for a EBM-treatment. In most domains where we make choices in our lives you don't follow pratices that are supported by evidence from peer-reviewed trials.
You don't get a haircut from a barber who practices evidence-based barbering. Even the people who pay a lot of money for their haircuts don't. Reading scientific papers just isn't the only way to gather useful knowledge.
The term evidence-based medicine comes from a http://jama.jamanetwork.com/article.aspx?articleid=400956 published in 1992.
It says:
Evidence-based medicine de-emphasizes intuition, unsystematic clinical experience, and pathophysiologic rationale as sufficient grounds for clinical decision making and stresses the examination of evidence from clinical research.
I wouldn't want someone to practice open-heart surgery on me based on his intuition but I don't see a problem with taking a massage from someone who read no scientific papers but who let's themselves be guided by his intuition and who has a positive track record with other patients.
Replies from: Qiaochu_Yuan, army1987↑ comment by Qiaochu_Yuan · 2013-03-06T00:59:53.627Z · LW(p) · GW(p)
Sure, but the idea that one should explicitly go about trying to teach rationality because there are these things called biases is much younger than the idea of medicine. Doctors have had a much longer time than LessWrong to get their act together.
Replies from: ChristianKl↑ comment by ChristianKl · 2013-03-06T15:52:26.132Z · LW(p) · GW(p)
Doctors have had a much longer time than LessWrong to get their act together.
The idea of teaching people to think better isn't new. Aristoteles also tried to teach a form of rationality. But even if the idea would be radically new, why would that matter?
Why should newer ideas be subject to a lower standard of evidence? Fairness? If you want to know the truth fairness has no place.
Let's look at another example: Romantic courtship. Do you practice evidence-based courtship, when you seek a fulfilling relationship with a woman? Would you say there no other form of courtship besides evidence-based courtship?
Most humans don't practice evidence-based courtship. Sometimes courtship doesn't work out. You can blame it on couple not being familiar with the scientific papers that are published on the subject of human courtship.
Nobody has shown that given the couple those scientific papers improves their relationship changes. Nobody has shown with EBM like evidence standards that doctors who are in touch with the scientific literature archieve better health outcomes for their patients.
That doesn't mean to me that EBM has no place, but I don't see a reason to reject any approach to increase my health that isn't backed by EBM.
There's some scientific evidence that suggests that vitamin D is good. In the blogosphere there are people who found that taking vitamin D supplements first thing in the morning is better than taking them in the evening. There's no trial for the timing of vitamin D supplements. I still take them first thing in the morning.
Replies from: Strange7↑ comment by A1987dM (army1987) · 2013-03-09T12:11:17.945Z · LW(p) · GW(p)
You don't get a haircut from a barber who practices evidence-based barbering.
That's not exactly 100.00% true -- I once overheard a barber priding himself with the fact that someone once got laid the night after getting a haircut from him.
Jokes aside, barbering is evidence-based -- given that it works at all, then barbers either have knowledge of how to do that hard-coded in their DNA (unlikely) or have learned to do that -- using evidence (even though not in a systematized way). You can immediately see that if you use this cutting technique then your client's hair will look this way. OTOH, a practitioner of non-evidence-based medicine cannot immediately see that giving a patient this substance diluted in 10^20 times as much water or sticking a needle in this particular spot or whatever will help cure the patient. (Likewise, musicians are normally evidence-based musicians to some extent, but astrologists are not evidence-based astrologists; can you find more examples?)
Replies from: incogn, ChristianKl↑ comment by incogn · 2013-03-09T13:03:01.754Z · LW(p) · GW(p)
If you interpret evidence-based in the widest sense possible, the phrase sort of loses its meaning. Note that the very post you quote explains the intended contrast between systematic and statistical use of evidence versus intuition and traditional experience based human learning.
Besides, would you not say that astrologers figure out both how to be optimally vague, avoiding being wrong while exciting their readers, much the same way musicians figure out what sounds good?
Replies from: army1987↑ comment by A1987dM (army1987) · 2013-03-09T13:09:20.332Z · LW(p) · GW(p)
If you interpret evidence-based in the widest sense possible, the phrase sort of loses its meaning. Note that the very post you quote explains the intended contrast between systematic and statistical use of evidence versus intuition and traditional experience based human learning.
Yes, but “intuition and traditional experience based human learning” is probably much less reliable in medicine than it is in barbering, so the latter isn't a good example in a discussion about the former.
Besides, would you not say that astrologers figure out both how to be optimally vague, avoiding being wrong while exciting their readers, much the same way musicians figure out what sounds good.
:-)
Something similar could be said about practitioners of alternative medicine, though.
Replies from: ChristianKl, incogn↑ comment by ChristianKl · 2013-03-09T16:33:02.245Z · LW(p) · GW(p)
Yes, but “intuition and traditional experience based human learning” is probably much less reliable in medicine than it is in barbering, so the latter isn't a good example in a discussion about the former.
The goal of barbering is to create haircuts that increase the attractiveness of the client to people besides the barber and the client.
A barber might think: "All my clients look really great", when in reality his haircuts reduce the attractiveness of the clients.
↑ comment by A1987dM (army1987) · 2013-03-09T16:43:55.272Z · LW(p) · GW(p)
Surely, judging someone's attractiveness using your System 1 alone is less hard than judging someone's health using your System 1 alone, for most people in most situations?
Replies from: ChristianKl↑ comment by ChristianKl · 2013-03-09T17:36:23.306Z · LW(p) · GW(p)
A professional barber is likely to notice a lot of things about a haircut that the average person doesn't see. It could be that he creates haircuts that look impressive to other barbers but don't look good to the average person of the opposing sex who isn't a barber.
I do think that you can get a decent assessment of someone's backpain by asking them whether it has gotten better. Actually that's even how most scientific studies who measure pain do it. They let the person rate their pain subjectively and when the subjective rating gets better through the drug they see it as a win.
For a lot of serious health issues it's easy to see when a person gets better.
Most homeopathists spend more time interviewing their patients and getting a good understanding of their condition than the average mainstream doctor who takes 5 minutes per patient.
↑ comment by incogn · 2013-03-09T15:06:18.893Z · LW(p) · GW(p)
I think the barbering example is excellent - it illustrates that, while controlled experiments more or less is physics, and while physics is great, it is probably not going to bring a paradigm shift to barbering any time soon. One should not expect all domains to be equally well suited to a cut and dried scientific approach.
Where medicine lies on this continuum of suitedness is an open question - it is probably even a misleading question, with medicine being a collection of vastly different problems. However, it is not at all obvious that simply turning up the scientificness dial is going to make things better. It is for instance conceivable that there are already people treating medicine as a hard science, and that the current balance of intuition and evidence in medicine reflects how effective these two approaches are.
I am not trying to argue whether astrology is evidence-based or not. I am saying that the very inclusive definition of evidence-based which encompasses barbering is, (a) nearly useless because it includes every possible way of doing medicine and (b) probably not the one intended by the others using the term.
Replies from: army1987↑ comment by A1987dM (army1987) · 2013-03-09T15:28:42.740Z · LW(p) · GW(p)
nearly useless because it includes every possible way of doing medicine
Huh? What evidence are homoeopathy and crystal healing and similar (assuming that's what Qiaochu_Yuan meant by “other kinds”) based on?
EDIT: Apparently not.
Replies from: Qiaochu_Yuan, ChristianKl↑ comment by Qiaochu_Yuan · 2013-03-09T18:58:05.236Z · LW(p) · GW(p)
"Other kinds" meant "whatever mainstream medicine does that doesn't fall under the evidence-based label," not alternative medicine. I should've been clearer.
Replies from: army1987, ChristianKl↑ comment by A1987dM (army1987) · 2013-03-10T14:55:22.663Z · LW(p) · GW(p)
Yes, I realized that later, while reading another branch of the thread (see my edit).
↑ comment by ChristianKl · 2013-03-09T19:01:27.885Z · LW(p) · GW(p)
What do you mean with "mainstream medicine" in that context?
Replies from: Qiaochu_Yuan↑ comment by Qiaochu_Yuan · 2013-03-09T19:37:13.124Z · LW(p) · GW(p)
What ambiguity is there in what I mean by "mainstream medicine" here?
Replies from: ChristianKl↑ comment by ChristianKl · 2013-03-09T19:56:24.961Z · LW(p) · GW(p)
A lot of people use the term evidence-based medicine interchangeable with mainstream medicine. What's in your opinion medicine that counts as mainstream medicine but that doesn't count as evidence-based medicine?
Replies from: Qiaochu_Yuan↑ comment by Qiaochu_Yuan · 2013-03-09T20:04:47.693Z · LW(p) · GW(p)
That doesn't agree with my experience. Evidence-based medicine refers to a specific and recent movement within mainstream medicine, which is much older.
Replies from: ChristianKl↑ comment by ChristianKl · 2013-03-09T21:28:12.473Z · LW(p) · GW(p)
If a doctor would today practice medicine the exact way it was practiced in 1950 I don't think you would say that the doctor practices mainstream medicine.
If you define "mainstream" by the amount of people who use it than homeopathy is probably "mainstream medicine". Even if you go by the status of the people, when the Queen uses homeopathy it's no low status treatment.
There are two reasons why homeopathy gets classified as "alternative medicine".
(1) It's uses a ideological framework that goes against the reductionist world view.
(2) There's are not enough high quality double blind studies to allow an institution such as cochrane to recommend homeopathy as a treatment.
The term Evidence-Based medicine got made up by a bunch of university professors in 1992 to describe the style of medicine that they were teaching. At the beginning the term intentionally downplayed clinicial experience. Today most medicial schools say that they teach Evidence-Based medicine but they weakened the definition in a way that allows clinicians using their clinical experience but that still focuses on peer reviewed trials.
If you don't pracitice medicine the way the university teach it in their normal programs than you are practicing in my opinion "alternative medicine".
↑ comment by ChristianKl · 2013-03-09T17:48:19.482Z · LW(p) · GW(p)
There are even dozens of scientific studies that support homeopathy. According to a report titled "Effectiveness, Safety and Cost-Effectiveness of Homeopathy in General Practice – Summarized Health Technology Assessment" commissioned by the Swiss government:
Many high-quality investigations of pre-clinical basic research proved homeopathic high-potencies inducing regulative and specific changes in cells or living organisms. 20 of 22 systematic reviews detected at least a trend in favor of homeopathy. In our estimation 5 studies yielded results indicating clear evidence for homeopathic therapy.
There are plenty people out there who can explain you why all those homeopathy studies are flawed, but on the other hand how many double blind controlled trials do you know that show that barbers can create haircuts that increase someone's chances with the opposing sex?
But in general people do buy homeopathic medicine not because they read the report of the Swiss government and belief it. They buy it based on anecdotal evidence. They hear that some friend had success with homeopathy and then the go out and buy it.
The fact that you are ideologically opposed to homeopathy and crystal healing working, doesn't mean that it fails to produce anecdotal evidence.
Replies from: army1987↑ comment by A1987dM (army1987) · 2013-03-10T15:03:21.310Z · LW(p) · GW(p)
If that was the only point of barbers, then already-married people, prepubescent children, homosexuals, etc. would never go to the barber's.increase someone's chances with the opposing sex
↑ comment by ChristianKl · 2013-03-09T16:21:44.183Z · LW(p) · GW(p)
OTOH, a practitioner of non-evidence-based medicine cannot immediately see that giving a patient this substance diluted in 10^20 times as much water or sticking a needle in this particular spot or whatever will help cure the patient.
That's wrong. If a acupuncturist puts needles in 10 people and 5 of them lose their back pain than he has "unsystematic clinical experience" that provides evidence for his treatment.
The core of evidence-based medicine is the belief that you shouldn't use that kind of evidence for clinical decision making but that doctors should read medicial journals that report clinical trials that show whether or not a treatment works.
Likewise, musicians are normally evidence-based musicians to some extent, but astrologists are not evidence-based astrologists; can you find more examples?
Actually musicians and astrologists are very similar. Both make money with providing entertaining performances for their clients. Members of those professions who ignore evidence about what entertains their clients go out of business.
Replies from: army1987, army1987↑ comment by A1987dM (army1987) · 2013-03-09T16:35:19.656Z · LW(p) · GW(p)
If a acupuncturist puts needles in 10 people and 5 of them lose their back pain than he has "unsystematic clinical experience" that provides evidence for his treatment.
Maybe some of those 5 would have lost their pain even without needles. Whereas the barber knows what his client would have looked like without the hair cut.
Replies from: Yosarian2↑ comment by Yosarian2 · 2013-05-12T18:05:57.066Z · LW(p) · GW(p)
Maybe some of those 5 would have lost their pain even without needles
Right, that's why it's unsystematic.
In the Bayesian sense of the word, "I stuck a needle in this person and the amount of pain he reported went down" would have to be considered to be evidence that would increase the Bayesian possibility that your hypothesis that acupuncture helps back pain is correct. However, it's not systematic, scientific evidence. To get that kind of evidence, you would have to do systematic studies of a large number of people, give some of them acupuncture and give some of them asprin, and see what the statistical result is.
I think that's what bogging this discussion down here, is that the word "evidence" is being used in two different ways. If we were perfectly rational beings, we would be able to use either kind of evidence, but the problem is that the first kind of evidence (individual unsystematic personal experiences) tends to be warped by all kinds of biases (selection bias, especially) making it hard to use in any kind of reliable way. You use it if it's all you have, but systematic evidence is very much preferable.
↑ comment by A1987dM (army1987) · 2013-03-10T15:09:10.745Z · LW(p) · GW(p)
Actually musicians and astrologists are very similar. Both make money with providing entertaining performances for their clients. Members of those professions who ignore evidence about what entertains their clients go out of business.
OK, if you consider the point of astrology to be “making money”, as opposed to “predicting people's personalities and future events”, then it is evidence-based -- but then again, if you consider the point of alternative medicine to be “making money”, as opposed to “improving people's health”, then it is evidence-based as well. (But now that Qiaochu_Yuan has made clear that it's not alternative medicine that he was talking about, this is kind of moot, so I'll tap out now.)
Replies from: ChristianKl↑ comment by ChristianKl · 2013-03-10T15:56:09.294Z · LW(p) · GW(p)
OK, if you consider the point of astrology to be “making money”, as opposed to “predicting people's personalities and future events”
I didn't. I advocated another goal, entertainment. I don't know that much about astrology but I think a fair percentage of the people who do pay a astrologists do it for entertainement purposes.
Letting someone stick needles inside you, when you go to a acupuncturist is less about getting entertainement.
The kind of people who like astrology often also like other personality tests that they find in magazines. People enjoy going through those tests.
If an astrologer would tell people something about their personality that's accurate but that those people aren't willing to accept, I doubt he would stay long in business.
A bit like the musician who only plays music that he himself considers to be good, but that's "too advanced" for his audience. If the musician only sees his own opinion of his work he's not different than an astrologer who only sees whether his horoscope is good. If you call that musician "evidence-based" than the astrologer who goes after his own judgement of his work is also "evidence-based".
But now that Qiaochu_Yuan has made clear that it's not alternative medicine that he was talking about, this is kind of moot, so I'll tap out now.
Why does that matter to the question whether barbers can be meaningfully to be said to practice evidence-based barbering?
Replies from: army1987↑ comment by A1987dM (army1987) · 2013-03-10T16:10:14.606Z · LW(p) · GW(p)
Why does that matter to the question whether barbers can be meaningfully to be said to practice evidence-based barbering?
I was claiming that barbering is more evidence-based than alternative medicine, but if alternative medicine is not what's being discussed, then even if I turned out to be right it still wouldn't be relevant.
↑ comment by incogn · 2013-03-05T01:53:40.288Z · LW(p) · GW(p)
Only in the sense that the term "pro-life" implies than there exist people opposed to life.
Replies from: MugaSofer, RomeoStevens, CCC↑ comment by RomeoStevens · 2013-03-05T02:32:18.499Z · LW(p) · GW(p)
pro-life is an intentional misuse of ontology.
↑ comment by CCC · 2013-03-05T12:06:14.408Z · LW(p) · GW(p)
A perusal of murder and suicide statistics - even the fact that such statistics exist - suggests the conclusion that there may, in fact, exist some people opposed to life; sometimes their own, sometimes that of others.
Replies from: Qiaochu_Yuan↑ comment by Qiaochu_Yuan · 2013-03-05T17:53:39.047Z · LW(p) · GW(p)
That's irrelevant to the point that incogn is making, though, which is that you can't make that inference from the fact that a label called "pro-life" exists because it's rhetoric. I'm willing to believe that the label "evidence-based medicine" is also rhetoric, but I don't actually know that yet; I would first have to know what doctors were doing before EBM became a thing.
Replies from: Eugine_Nier, handoflixue↑ comment by Eugine_Nier · 2013-03-06T05:29:47.004Z · LW(p) · GW(p)
I would first have to know what doctors were doing before EBM became a thing.
And how good the followers of EBM are at actually being evidence based as opposed Straw Vulcan.
↑ comment by handoflixue · 2013-03-05T20:20:50.057Z · LW(p) · GW(p)
Homeopathy? Crystal Therapy? Color Therapy? A quick Google search for "alternate medicine" should produce all sorts of non-evidence-based medical philosophies.
↑ comment by TraderJoe · 2013-03-05T10:49:04.615Z · LW(p) · GW(p)
You mean like acupuncture?
Replies from: Qiaochu_Yuan, MugaSofer↑ comment by Qiaochu_Yuan · 2013-03-05T17:39:20.065Z · LW(p) · GW(p)
Wikipedia informs me that evidence-based medicine is a movement in the health care community that really only got underways in the 90s. I am not sure I want to know what the health care community was doing before the 90s. I'm not talking about alternative medicine, I'm talking about whatever mainstream medicine was and is doing that doesn't fall under this label.
Replies from: pinyaka, satt, handoflixue, Yosarian2↑ comment by pinyaka · 2013-03-06T15:04:16.770Z · LW(p) · GW(p)
Well, until the mid-80's doctors believed that infants either a) didn't feel pain or b) wouldn't remember it anyway (mostly because of this study from the 40's), so they didn't use anesthesia for infants when performing heart surgery until someone collected evidence that babies were more likely to live through the surgery if given something to knock them out.
EDIT: Removed extraneous word
Replies from: satt↑ comment by satt · 2013-05-19T22:09:08.549Z · LW(p) · GW(p)
they didn't use anesthesia for infants when performing heart surgery until someone collected evidence that babies were more likely to live through the surgery if given something to knock them out
Really? I notice (with some relief) that the control babies in the linked study still got anaesthesia; it's just that they got nitrous oxide instead of nitrous oxide and fentanyl.
↑ comment by satt · 2013-03-09T18:51:40.140Z · LW(p) · GW(p)
I am not sure I want to know what the health care community was doing before the 90s.
On the bright side, some of it was just evidence-based medicine without the branding.
For example, the UK Medical Research Council put randomized trials on the map in 1948 with its randomized trial of streptomycin, which had been discovered only a few years before. The massive 1954 trials of the famous Salk polio vaccine also included a randomized trial comprising over 700,000 children. (That said, the non-randomized trial was even larger; the origin of this odd, hybrid study design is an interesting bit of history.)
↑ comment by handoflixue · 2013-03-05T20:30:58.347Z · LW(p) · GW(p)
Quoth said Wikipedia article, in the "criticisms":
"EBM applies to groups of people but this does not preclude clinicians from using their personal experience in deciding how to treat each patient. One author advises that "the knowledge gained from clinical research does not directly answer the primary clinical question of what is best for the patient at hand" and suggests that evidence-based medicine should not discount the value of clinical experience.[26] Another author stated that "the practice of evidence-based medicine means integrating individual clinical expertise with the best available external clinical evidence from systematic research".[1]"
Which suggests that the precursor to EBM is a combination of Education and Intuition. Sorry if I'm not framing it terribly well - there's an intuitive category in my head for this method, but I've never really had to refer to it explicitly. It's the same technique I use to troubleshoot computer problems - I get a hunch as to what is causing it, and then proceed through a mixture of "safe, generalized advice" (try rebooting!) and "advice specific to the problem I think it is" (aha, you must not have your DNS configured correctly). If both of those fail, THEN I'll resort to actually collecting data, analyzing it, and seeing where that leads me - "have you had other problems?", "hmm, let me look up this error code..."
I've generally observed this path as the default human behavior, with "call someone else" occurring when they hit the limit of their abilities.
Replies from: pinyaka↑ comment by pinyaka · 2013-03-06T15:06:39.348Z · LW(p) · GW(p)
I've generally observed this path as the default human behavior, with "call someone else" occurring when they hit the limit of their abilities.
Not a bad plan if you know the limits of your abilities and aren't trained to act confident even when you're not.
↑ comment by Yosarian2 · 2013-05-12T18:11:20.946Z · LW(p) · GW(p)
For a lot of the medical advances we had earlier in the 20th century, you didn't really need to do large-scale clinical studies to see if it was working. You gave someone an antibiotic, and they suddenly got much better. You gave people a polio vaccine, and they didn't get polio. You took someone's appendix out, and they didn't die.
It was really later in the 20th century, when medicine got more and more focused on treating and preventing long-term degenerative illnesses like cancer or heart attacks or high blood pressure, that it became more vital to measure the difference in a large-scale statistical way between how effective different types of treatment were over a long period of time.
Replies from: satt↑ comment by satt · 2013-05-19T22:05:06.431Z · LW(p) · GW(p)
For a lot of the medical advances we had earlier in the 20th century, you didn't really need to do large-scale clinical studies to see if it was working. You gave someone an antibiotic, and they suddenly got much better. You gave people a polio vaccine, and they didn't get polio. You took someone's appendix out, and they didn't die.
Not the best examples, although you're right about appendectomies! I nonetheless agree with the broader point that decades ago there was less need for fine-grained, systematic medical studies (you were just unlucky in your choice of examples).
↑ comment by buybuydandavis · 2013-03-06T13:50:06.678Z · LW(p) · GW(p)
On the positive side, evidence-based medicine promotes greater measurement of patient outcomes and sharing of that information to weigh treatment options.
On the negative side, it is largely about denying patients coverage and access to treatments based on officially approved disease models and preference trade offs that ignores evidence that was not used in the model and overrules patient preferences. It's evidence ignoring and patient controlling medicine.
Replies from: KnaveOfAllTrades↑ comment by KnaveOfAllTrades · 2013-03-06T18:31:24.219Z · LW(p) · GW(p)
I think you unknowingly {submitted this comment prematurely}? :)
Replies from: buybuydandavis↑ comment by buybuydandavis · 2013-03-06T18:48:16.154Z · LW(p) · GW(p)
Thanks. I edited around and left that last line when I should have deleted it.
All tidy now.
comment by DataPacRat · 2013-03-05T18:45:26.297Z · LW(p) · GW(p)
Once MetaMed has been paid for and done a literature search on a given item, will that information only be communicated to the individual who hired them, or will it be made more widely available?
Replies from: wedrifid↑ comment by wedrifid · 2013-03-05T20:46:20.090Z · LW(p) · GW(p)
Once MetaMed has been paid for and done a literature search on a given item, will that information only be communicated to the individual who hired them, or will it be made more widely available?
A related question: Assuming that the information remains private (as seems to be the most viable business model) will the company attempt to place restrictions on what the clients may do with the information? That is, is the client free to publish it?
Replies from: alyssavance, MichaelVassar↑ comment by alyssavance · 2013-03-05T20:59:33.547Z · LW(p) · GW(p)
Clients are free to publish whatever they like, but we are very strict about patient confidentiality, and do not release any patient information without express written consent.
Replies from: Viliam_Bur, pinyaka, wedrifid↑ comment by Viliam_Bur · 2013-03-06T10:21:05.349Z · LW(p) · GW(p)
I like the idea of clients being free to publish anything... but what will you do if they misrepresent what you said, and claim they got the information from you? If could be a honest mistake (omiting part of information that did not seem important to them, but which in fact changes the results critically), oversimplification for sake of popularity ("5 things you should do if you have cancer" for a popular blog), or outright fraud or mental illness. For example someone could use your services and in addition try some homeopatic treatment, and at the end they would publish your advice edited to include the recommendation for homeopathy.
So there should be a rule like: "either publish everything verbatim... or don't mention our name". (I guess you probably already have it, but I say this for the case you don't.)
↑ comment by wedrifid · 2013-03-05T21:03:16.675Z · LW(p) · GW(p)
Clients are free to publish whatever they like, but we are very strict about patient confidentiality, and do not release any patient information without express written consent.
Oh, Tom is involved too!
Thankyou for responding to our questions. I was curious.
↑ comment by MichaelVassar · 2013-03-07T07:08:06.330Z · LW(p) · GW(p)
We won't publish anything, but clients are free to publish whatever they wish to in any manner that they wish.
comment by freyley · 2013-03-05T18:40:10.778Z · LW(p) · GW(p)
caveats: they're new; it's hard to do what they're doing; they have to look serious; this is valuable the more it's taken seriously.
They have really wonderful site design/marketing...except that it doesn't give me the impression that they will ever be making the world better for anyone other than their clients. Here's what I'd see as ideal:
- They've either paid the $5k themselves, a drop in the bucket of their funding apparently, and put up one report as both a sample and proof of their intent to publish reports for everyone, or (better) gotten a client who's had a report to agree to allow them to release it.
- This report, above, is linked to from their news section and there's a prominent search field on the news section (ok), or there's a separate reports section (better)
- The news section has RSS (or the reports section has RSS, or both, best)
On a more profiteering viewpoint, they could offer a report for either $5k for a private report, or $3k for a public report, with a promise to charge $50 for the public report until they reach $5k (or $6k, or an internal number that isn't unreasonable) and then release it.
Most people who are seriously sick tend to get into a pretty idealistic mode, is my experience, and would actually be further convinced by putting their $5k both to help themselves and to help others, and while sure, they could release the report themselves, metamed has a central, more trustable platform. If they want me to believe that they're interested in doing that kind of thing, it'd be nice if they had something up there to show me that they hope to.
On preview, I realize that the easy objection is that these are personalized reports, and data confidentiality is important. They obviously will only be able to publish pieces of reports that are not personal, and this is obviously a more costly thing than just tossing a pdf up on a website. Hm.
All of that said, they look like a really exciting company, I really hope they do well (and then take my advice =).
Replies from: ChristianKl, NancyLebovitz, Grif↑ comment by ChristianKl · 2013-03-06T03:34:23.452Z · LW(p) · GW(p)
A patient might profit from open publishing of the report. If MetaMed starts getting a reputation for good reports it will get read by medical experts. If an experts reads something that's wrong in the report it would be great if there a way for that expert write a comment under the report. That comment could be very helpful to the patient.
↑ comment by NancyLebovitz · 2013-03-05T21:05:31.521Z · LW(p) · GW(p)
I'm not sure that's the best scheme, but I'm hoping MetaMed finds some way of taking their findings public.
Replies from: ShannonFriedman↑ comment by ShannonFriedman · 2013-03-06T17:20:59.377Z · LW(p) · GW(p)
They're all hardcore x-risk reductionists and good people. I would be very surprised if they didn't do everything they could to help people in any way they could as soon as it made sense, and they weren't sacrificing long term goals for shorter term ones.
↑ comment by Grif · 2013-03-06T17:54:22.226Z · LW(p) · GW(p)
I suspect that later, when they have more presence in the public and expert view, they will open up new payment options to increase visibility of their reports, but only after they have employed significantly more researchers and run them through rigorous epistemic ethics training. Otherwise, there's little stopping a Big Pharma company from hiring Metamed for a $3,000 report, and then posting a biased summary of the report on their news page, along with an "APPROVED BY METAMED" sticker. Even worse if Metamed considers the "approval sticker" to be useful to spreading awareness of evidence-based medicine. The potential for corruption is just too high.
comment by tgb · 2013-03-05T16:03:11.738Z · LW(p) · GW(p)
Shouldn't there be a disclosure of some sorts that MetaMed shares some sponsors with MIRI?
Simple diagnostic tools which may even ignore some data give measurably better outcomes in areas such as deciding whether to put a new admission in a coronary care bed (Green and Mehr 1997).
Better outcomes than what? Typical doctors' diagnostics, I assume?
Replies from: Eliezer_Yudkowsky, DanielLC↑ comment by Eliezer Yudkowsky (Eliezer_Yudkowsky) · 2013-03-05T16:20:46.607Z · LW(p) · GW(p)
Shouldn't there be a disclosure of some sorts that MetaMed shares some sponsors with MIRI?
I thought that was obvious by listing Jaan Tallinn as an x-risk funder and Peter Thiel, but yes, you're very correct that this should be explicitly stated on general principles. Will edit.
Replies from: tgb↑ comment by DanielLC · 2013-03-06T02:00:55.075Z · LW(p) · GW(p)
Shouldn't there be a disclosure of some sorts that MetaMed shares some sponsors with MIRI?
Why? That doesn't sound like much of a conflict of interest. Am I missing something?
If Metamed sponsored MIRI, that would definitely be an issue.
Replies from: Kaj_Sotala↑ comment by Kaj_Sotala · 2013-03-06T12:32:48.905Z · LW(p) · GW(p)
"Hey, MIRI folks, we're giving you a lot of money, how about you said a couple of nice words about this other company of ours?"
"Hey, our sponsors are funding another company, maybe if we helped promote that company and it ended up doing well, our sponsors would have more money to give us."
""Hey, our sponsors are funding another company, maybe if we helped promote that company our sponsors would be grateful and give us some extra money."
comment by MichaelHoward · 2013-03-05T02:30:25.524Z · LW(p) · GW(p)
For the sake of humanity, cute kittens, whatever it takes to get past your qualms about this being advertising...
Please promote this immediately to the front page so it can get as much attention as possible.
comment by EHeller · 2013-03-07T04:14:28.217Z · LW(p) · GW(p)
I'm overall not impressed- looking at their reports, what do they offer that up-to-date (uptodate.com) doesn't? Sure, they advertise at patients, and up to date is aimed at institutions- but most hospitals I'm familiar with (and hence almost any specialist physician) are going to have access already to up-to-date. Also, in general, I'm willing to bet most doctors are in a better position to digest a research report than the average patient.
Sure, its a good idea, but its already being done in a very comprehensive fashion by a company that already has something like 90+% market penetration for academic hospitals. What is metamed's comparative advantage?
Replies from: Zian, Error↑ comment by Zian · 2013-03-10T05:08:16.507Z · LW(p) · GW(p)
I get the impression that Metamed also figures out likely diagnoses, which would be a pre-requisite for using Up to Date.
Replies from: EHeller↑ comment by EHeller · 2013-03-11T17:54:50.516Z · LW(p) · GW(p)
That seems a very tricky proposition- for $5k you get a team of medical students and phd students doing a literature search for 24 hours. Without a diagnosis to start with and without an ability to order and receive test results (even if you suggest a test, will the results be back in 24 hours?) my prior would be that diagnosis would be extremely unlikely. WIthout a diagnosis, I'm not even sure how informative such a short literature search can be.
In the case of symptoms-just-started/no diagnosis, doesn't an experienced doctor at a hospital (with all the support staff a hospital implies-. labs,etc) have a pretty high competitive advantage? Apriori, an experienced physician with diagnostic equipment and several days should outperform some medical students with journal access and 24 hours.
Also, this whole thread I find myself shilling for the status-quo, but I should make it clear- hospitals scare the hell out of me. I've done statistical work for internal performance reviews for a large carrier in Southern California and found tons of alarming medical mistakes. I just don't see how Metamed solves any of the actual problems. Most mistakes are of the form transfer orders go through, but patient isn't moved (thus being on a floor under no one's care for X hours), or pharmacy doesn't deliver a necessary medication to a patient in a timely fashion, pharmacy compounded medication in the wrong fluid, ICU doctor refuses to admit ICU level patient because he wants to go home early (leaving a critical patient in a DOU). Total misdiagnosis/mismanagement DOES happen,but its not a leading-order type of mistake, and its usually not because of a lack of access to relevant evidence-based-medication, but rather despite a lot of access to info. Also, at least at one large hospital group in Southern California- this mistreatment tends to be Bayesian in nature- most patients are things like heart attack/stroke and so if you present with symptoms that fit even loosely into one of those large categories, you get treated for them. Such a system does a lot of good for the typical patient ,but if you have a rare disease, it can send you totally down the wrong path. Trying to 'fix' this problem can do more harm than good (save the occasional rare illness patient at the expense of dozens of more typical patients)
The best solution is to find a competent doctor (or even a competent ICU nurse) and pay them to be the point of contact all the hospital doctors have to go through before they are allowed to treat you, but hardly anyone can afford a concierge doctor.
Replies from: NancyLebovitz↑ comment by NancyLebovitz · 2013-03-13T15:58:59.890Z · LW(p) · GW(p)
This matches my feeling that a lot of what's wrong with (American?) medicine is the result of patients being viewed as low status.
What you've been seeing is what can go wrong at the hospital. I've heard a fair amount of anecdotes about sloppy diagnosis-- patients' symptoms being ignored for months or years of doctor visits. My impression is that doctors who listen and think are not terribly common.
Replies from: shminux↑ comment by Shmi (shminux) · 2013-03-13T18:40:45.945Z · LW(p) · GW(p)
A typical family doctor's appointment is scheduled every 15 min where I am (except for annual checkups). This includes the time between patients for any necessary paperwork. So, not much you can do for people with rare symptoms in that setting. This is where MetaMed can help, since they spend 100 to 1000 times more time than that on each case and are looking specifically for edge cases and individualized treatment.
Replies from: NancyLebovitz, EHeller↑ comment by NancyLebovitz · 2013-03-13T20:01:09.119Z · LW(p) · GW(p)
I agree that fifteen minutes minus paperwork is shockingly short. Still, there are doctors who do reasonably well at paying attention.
Most of my information is from the fat acceptance community, where there are a great many stories about doctors who just tell fat patients to lose weight*, regardless of symptoms. The typical stories seem to be either "I had to go to three or four doctors to find one who would listen" or "I must be lucky, I have a great doctor". I can't derive a strong opinion about the proportion of attentive doctors from this, though I wouldn't be surprised to find that it's under half.
*I've also seen a few stories from unusually thin people who were simply told to gain weight, and one from a man who (as far as I could tell) was lean and muscular, but was told to lose weight by a doctor who literally only looked at his BMI.
Replies from: NancyLebovitz↑ comment by NancyLebovitz · 2013-06-03T11:37:40.829Z · LW(p) · GW(p)
International list of fat-friendly medical professionals
A fat friendly professional does not necessarily avoid mentioning a client's weight, but he or she avoids making an issue of it, avoids lectures and humiliation, and respects the client's wishes with regard to weight discussions.
If a client asks not to be weighed, the request is acknowledged without complaint and taken into account automatically on future visits. (Note: there are a few cases where weighing is necessary, for example, when administering certain medications, chemotherapy, or anesthesia.)
If weight sometimes contributes to a problem, the professional may mention this, but also considers other diagnoses and recommends tests to determine the actual diagnosis if appropriate. If weight loss is a recommended treatment for a problem, the fat friendly professional may mention this, but at minimum will also recommend and prescribe other treatments. A fat friendly professional accepts a client's wish not to use weight loss as a treatment.
Ideally, the professional's office has available armless chairs, large blood pressure cuffs, large examination gowns, and other equipment suitable for fat people. If not, the office acknowledges the importance of such items when told.
Some fat friendly professionals believe that fat is not unhealthy. Others may believe that fat is unhealthy, but may acknowledge that weight loss doesn't work or is dangerous and/or that the client has a right to direct his or her own treatment.
↑ comment by EHeller · 2013-03-18T06:05:43.029Z · LW(p) · GW(p)
Sure- but if you have some rare symptom, any decent family doctor should say "go see a specialist" and refer you. You certainly aren't going to contact metamed every time you get sick, and for chronic conditions, a specialist (with journal and up-to-date access) is going to be the managing physician. Anything other than routine sniffles, vaccinations and check-ups and you probably have exceeded your family doctor's expertise.
The big problem for misdiagnosis at the family-med level are the hordes of relatively rare diseases with common symptoms, but this is a very hard problem to solve. Having spent some time dealing with this as a statistical problem, even if you have a rare cluster of common symptoms, its usually the case that you are more likely to have a rare presentation of a common disease than it is that you have a rare disease.
Replies from: Kenny↑ comment by Error · 2013-03-07T16:07:46.377Z · LW(p) · GW(p)
Presumably what you're paying for is for someone smarter than you to do the literature research for you; if I'm reading uptodate's product page correctly, they make the information available but it's up to you (or your physician) to sort through it and figure out what applies to you.
Replies from: EHeller↑ comment by EHeller · 2013-03-07T23:49:12.924Z · LW(p) · GW(p)
UpToDate provides a summary of research based on disease. i.e. for this disease, these treatments are recommended because of study A,B and C and physiological facts D, E and F. There are counter-indications and risks from these treatments because of x,y, and z. Unfortunately, I can't reproduce one of their reports here, but its not just a huge literature dump, its summarized and treatment options are graded.
Looking at the metamed concierge report on Gout (linked to elsewhere in this thread) its formatting appears to be very much like an UpToDate report- the most recent literature is digested and summarized at a decently high level, but it doesn't strike me as better than (or even different from!) the UpToDate recommendations. Given that 90% of academic hospitals already have paid for UpToDate, and honestly in most cases it will be better for the physician to interpret the report, I can't see very much for metamed to bring to the table.
Also worth pointing out- the people who write the summaries for UpToDate are most often researchers in the field of the illness. Near as I can tell from their webpage metamed's researchers are often medical students, or non-medical phd students (the point being that with metamed you are paying for something general called "expertise" and in many cases not actual field-relevant medical expertise).
Replies from: drethelin↑ comment by drethelin · 2013-03-09T21:35:26.675Z · LW(p) · GW(p)
What's the price difference between metamed and going out of your way to go to an academic hospitals? Am I wrong in thinking most hospitals are not academic hospitals?
Replies from: EHeller↑ comment by EHeller · 2013-03-09T23:24:46.271Z · LW(p) · GW(p)
Probably depends on your insurance (i.e. if you are with an HMO, you'll be locked in to the network unless you get a referral). Outside of HMOs, I'm not aware of an insurance that has a copay difference between going to an academic or community hospital. If you go to a community hospital with something complicated, you'll almost certainly end up transferred to an academic referral center anyway.
comment by xv15 · 2013-03-05T18:10:09.908Z · LW(p) · GW(p)
Only one out of 21 obstetricians could estimate the probability that an unborn child had Down syndrome given a positive test
Say the doctor knows false positive/negative rates of the test, and also the overall probability of Down syndrome, but doesn't know how to combine these into the probability of Down syndrome given a positive test result.
Okay, so to the extent that it's possible, why doesn't someone just tell them the results of the Bayesian updating in advance? I assume a doctor is told the false positive and negative rates of a test. But what matters to the doctor is the probability that the patient has the disorder. So instead of telling a doctor, "Here is the probability that a patient with Down syndrome will have a negative test result," why not just directly say, "When the test is positive, here is the probability of the patient actually having Down syndrome. When the test is negative, here is the probability that the patient has Down syndrome."
Bayes theorem is a general tool that would let doctors manipulate the information they're given into the probabilities that they care about. But am I crazy to think that we could circumvent much of their need for Bayes theorem by simply giving them different (not necessarily much more) information?
There are counterpoints to consider. But it seems to me that many examples of Bayesian failure in medicine are analogously simple to the above, and could be as simply fixed. The statistical illiteracy of doctors can be offset so long as there are statistically literate people upstream.
Replies from: CCC, xv15, buybuydandavis, prase↑ comment by CCC · 2013-03-05T18:32:07.989Z · LW(p) · GW(p)
This stops working in the case where some of the people upstream can't be trusted. Consider the following statement:
"The previous test, if you have a positive result, means that the baby has a 25% chance of having Down syndrome, according to the manufacturer. But my patented test will return a positive result in 99% of cases in which the baby has Down syndrome."
Replies from: xv15↑ comment by xv15 · 2013-03-05T19:15:56.612Z · LW(p) · GW(p)
"False positive rate" and "False negative rate" have strict definitions and presumably it is standard to report these numbers as an outcome of clinical trials. Could we similarly define a rigid term to describe the probability of having a disorder given a positive test result, and require that to be reported right along with false positive rates?
Seems worth an honest try, though it might be too hard to define it in such a way as to forestall weaseling.
Replies from: None↑ comment by [deleted] · 2013-03-05T19:29:14.721Z · LW(p) · GW(p)
If I understand the following Wikipedia page correctly:
http://en.wikipedia.org/wiki/Positive_predictive_value
The term you are requesting is Positive predictive value and Negative predictive value is the term for not having a disorder given a negative test result.
It also points out that these are not solely dependent on the test, and also require a prevalence percentage.
But that being said, you could require each test to be reported with multiple different prevalence percentages:
For instance, using the above example of Downs Syndrome, you could report the results by using the prevalence of Downs Syndrome at several different given maternal ages. (Since prevalence of Down's Syndrome is significantly related to maternal age.)
Replies from: xv15↑ comment by xv15 · 2013-03-06T02:54:34.387Z · LW(p) · GW(p)
thanks, PPV is exactly what I'm after.
The alternative to giving a doctor positive & negative predictive values for each maternal age is to give false positive & negative rates for the test plus the prevalence rate for each maternal age. Not much difference in terms of the information load.
One concern I didn't consider before is that many doctors would probably resist reporting PPV's to their patients because they are currently recommending tests that, if they actually admitted the PPV's, would look ridiculous! (e.g. breast cancer screening).
↑ comment by xv15 · 2013-03-06T19:22:51.638Z · LW(p) · GW(p)
Another alternative is to provide doctors with a simple, easy-to-use program called Dr. Bayes. The program would take as input: the doctor's initial estimate of the chance the patient has the disorder (taking into account whatever the doctor knows about various risk factors) the false positive and false negative rates of a test.
The program would spit out the probability of having the disorder given positive and negative test results.
Obviously there are already tools on the internet that will implement Bayes theorem for you. But maybe it could be sold to doctors if the interface were designed specifically for them. I could see a smart person in charge of a hospital telling all the doctors at the hospital to incorporate such a program into their diagnostic procedure.
Failing this, another possibility is to solicit the relevant information from the doctor and then do the math yourself. (Being sure to get the doctor's prior before any test results are in). Not every doctor would be cooperative...but come to think of it, refusal to give you a number is a good sign that maybe you shouldn't trust that particular doctor anyway.
↑ comment by buybuydandavis · 2013-03-06T13:06:09.017Z · LW(p) · GW(p)
Okay, so to the extent that it's possible, why doesn't someone just tell them the results of the Bayesian updating in advance?
Because then they would be assuming they had all relevant prior information for that particular patient. They don't.
For example, age of mother, age of father, their genes, when they've lived where, what chemicals they've been exposed to, etc., are many factors the manufacturer has no knowledge of, but the doctor might. Naturally, it would be helpful for the company to make an online diagnostic model of all known relevant factors available online, updated as new information comes in, but given the regulatory and legal climate (at least here in the US), something so sensible is likely completely infeasible.
↑ comment by prase · 2013-03-09T21:12:54.747Z · LW(p) · GW(p)
The incidence of the disease may be different for different populations while the test manufacturer may not know where and on which patients the test is going to be used.
Also, serious diseases are often tested multiple times by different tests. What would a Bayes-ignorant doctor do with positives from tests A and B which are accompanied with information: "when test A is positive, the patient has 90% chance of having the syndrome" and "when test B is positive, the patient has 75% chance of having the syndrome"? I'd guess most statistically illiterate doctors would go with the estimate of the test done last.
comment by Larks · 2015-05-25T18:11:11.558Z · LW(p) · GW(p)
It appears that MetaMed has since gone out of business.
Replies from: beberly37comment by AlexSchell · 2013-03-05T01:05:49.025Z · LW(p) · GW(p)
you shouldn't feed the patient rat poison
Are you referring to warfarin here or am I imagining things?
Replies from: Eliezer_Yudkowsky↑ comment by Eliezer Yudkowsky (Eliezer_Yudkowsky) · 2013-03-05T13:16:08.171Z · LW(p) · GW(p)
(Blinks.)
Hadn't thought of that. Actually, from what I understand, the status of warfarin is mostly okay now because they test for unusual sensitivity to it before they administer it?
Replies from: rhollerith_dot_com, AlexSchell↑ comment by RHollerith (rhollerith_dot_com) · 2013-03-05T14:28:45.969Z · LW(p) · GW(p)
Prescription warfarin -- actually they might use related molecules these days with the same basic mechanism of action -- kills 10s of thousands per year: they die from loss of blood because the warfarin-like molecules have inhibited the clotting mechanism more than intended. So for example someone I was friends with died (in his sleep) this way.
Nevertheless, warfarin-like molecules have positive expected global utility because clots cause so much negative utility. So for example I was on it for about 6 years. You're supposed to get a blood test every 2 weeks for as long as you're on it.
Since Alex was a grad student in pharmacy, he'll probably correct any untruths in the above in the unlikely event there are any.
ADDED. "Unusual sensitivity" is the wrong way to describe it.
Replies from: Kawoomba↑ comment by Kawoomba · 2013-03-05T18:01:23.178Z · LW(p) · GW(p)
Of note, 23andme tests for genetically determined warfarin tolerance. I can copypasta their references on demand.
Replies from: J_C↑ comment by J_C · 2013-03-06T12:43:08.404Z · LW(p) · GW(p)
This is actually not relevant as warfarin dosage is determined by regular testing and dose adjustment. Your inborn metabolic rate is a very small effect compared to, for example, dietary preferences. (for those who are unfamiliar with the agent, warfarin antagonises the effects of vitamin K and so must be adjusted against dietary intake)
Unfortunately there are many people in the health sector offering tests that, whilst factually correct, are irrelevant to a patient's care.
Replies from: Kawoomba↑ comment by Kawoomba · 2013-03-06T14:03:45.899Z · LW(p) · GW(p)
Someone tell the NHS, which is sponsoring a large trial to explore just that question.
The influence of the genotype varies from "typical sensitivity" to "may require greatly decreased warfarin dose". A range that is all but irrelevant, regular testing or no (think for example of the initial dosage).
↑ comment by AlexSchell · 2013-03-05T19:24:56.898Z · LW(p) · GW(p)
There is the option of being tested for polymorphisms of 1-2 of the most relevant metabolic enzymes, which account for some of the bleeding risk. My impression is that genotyping is not routinely done. Also, warfarin is risky in normal metabolizers (many many drug/food/disease interactions). I agree with Richard on the overall cost-benefit. (ETA: Though there are new expensive drugs approved for some of the same indications -- rivaroxaban and dabigatran -- that show some promise of being safer.)
comment by Shmi (shminux) · 2013-03-05T17:13:06.904Z · LW(p) · GW(p)
This is a new service and it has to interact with the existing medical system, so they are currently expensive, starting at $5,000 for a research report. (Keeping in mind that a basic report involves a lot of work by people who must be good at math.) If you have a sick friend who can afford it - especially if the regular system is failing them, and they want (or you want) their next step to be more science instead of "alternative medicine" or whatever - please do refer them to MetaMed immediately.
A friend of mine suffers from debilitating effects of fibromyalgia, to the degree that she had to quit her job. She has tried all possible conventional and alternative medicine, with little success. She would certainly be prepared to pay $5000 or more for a near-certain relief, but not for yet another literature search of undetermined benefit. I'm guessing she is not the target audience for MetaMed?
Replies from: None, khafra, Wakarimahen, buybuydandavis, John_Maxwell_IV, AndrewH↑ comment by [deleted] · 2013-03-06T15:03:13.307Z · LW(p) · GW(p)
Well, according to their FAQ, they offer a trial service. So your friend would not have to continue to a larger report if the trial seemed to be indicating low benefits of further research.
Can I try MetaMed before committing to a large purchase? Yes. If your case has a larger budget, we can start with a smaller, trial report to ensure quality of service, and confirm that MetaMed is the right choice for you.
And they also offer Financial aid - There is almost no information about this posted, other than that it exists. I guess you would have to call to determine more about how it worked. If your friend did qualify, that would be a substantial boon:
http://metamed.com/financial-aid
Once you have consulted with our medical team, if you need financial aid to help with the cost of your MetaMed service, we will email you an application right away.
And it looks like overall there are at least three tiers of potential research:
http://metamed.com/personalized-research-for-individuals
And here are examples of reports at each tier:
http://metamed.com/static/Meta_Sleep.pdf (Standard)
http://metamed.com/static/Meta_H_Pylori.pdf (Plus)
http://metamed.com/static/Meta_Gout.pdf (Concierge)
Hopefully that information will help with deciding whether or not to contact them for more information.
Replies from: EHeller, Scottbert↑ comment by EHeller · 2013-03-07T04:10:43.132Z · LW(p) · GW(p)
Those results do not impress me as to the value of their research. There is nothing there that isn't covered by Up-to-Date (http://www.uptodate.com/home) and every hospital I've done stats work for (several, all over the country both community and academic) has provided their physicians with up to date access.
Your best bet (apparently) would be simply asking your physician for the up-to-date report for your diagnosis. If your physician does not have up-to-date access, get a referral to the nearest academic center.
Replies from: EHeller↑ comment by EHeller · 2013-03-07T08:07:58.249Z · LW(p) · GW(p)
I'm unsure why I've been voted down here- if shminux's friend wants a version of a report similar in quality to what metamed can provide she can ask her physician if the physician has up-to-date (or an other research aggregator) access. If the physician doesn't, its potentially a measure of quality (which can otherwise be hard to judge), and she should get a referral to an academic medical center, which will definitely have something like up-to-date available. This seems to me like decently practical advice for those who have insurance, but don't have the money for metamed. I'm still relatively new so I'm requesting explicit feedback to improve the quality of my posts.
Edited to Add: According to the company, > 90% of American academic hospitals already have subscribed to Up-to-Date, so getting a referral to an academic center has a great chance of getting you to someone with already-paid-for-access to this sort of report.
Replies from: None↑ comment by [deleted] · 2013-03-07T14:17:34.868Z · LW(p) · GW(p)
I'm glad you linked competition, so we can compare similar industries for reference (I asked a very similar thing about Watson.) One item that stood out on their site:
Dr. Gordon Guyatt from McMaster University, who coined the term "evidence-based medicine," makes 6-8 visits to UpToDate per year to work with UpToDate editors. Watch him in action with our physician editors as they review and analyze medical evidence that informs UpToDate graded recommendations.
http://www.uptodate.com/home/editorial
And I was also able to find their methods for that grading as well, in case anyone wants to compare Meta Research Methods across Meta Research Organizations:
↑ comment by Scottbert · 2013-03-07T16:00:25.048Z · LW(p) · GW(p)
Sleep apnea seems like something regular doctors should be able to figure out, and I know gout has at least been known for a long time. Are these meant to be examples of what the reports look like more than examples of how Metamed can find obscure treatments? $5000 seems a bit much for someone to be told 'get checked for sleep apnea and lose weight'.
↑ comment by khafra · 2013-03-05T20:39:21.968Z · LW(p) · GW(p)
If she's tried all possible conventional and alternative medicine, MetaMed will not help. If she missed something (1) obscure but promising, (2) cutting edge and promising, or (3) unique about her particular body that makes an unusual treatment promising; MetaMed might be able to help.
So, if $5000 is what certain relief is worth to her, MetaMed isn't for her. If certain relief is worth $10000 to her, she should estimate how likely it is that paid, reasonably savvy researchers can find something she's missed; and go for it if she feels it's over 50% likely.
Replies from: shminux↑ comment by Shmi (shminux) · 2013-03-05T20:52:15.964Z · LW(p) · GW(p)
she should estimate how likely it is that paid, reasonably savvy researchers can find something she's missed; and go for it if she feels it's over 50% likely.
No, it's much worse than that: "how likely it is that paid, reasonably savvy researchers can find something she's missed" AND it has a near certainty of helping. Current prior: nothing has helped so far, so the odds of something she missed ended up being useful is pretty low. If the estimate of helpfulness is, say, 1% (that's pretty optimistic), and the odds that MetaMed will find something new is 50%, then certain relief has to be worth $100k.
Replies from: Elithrion, Wakarimahen, Decius, wedrifid, pinyaka↑ comment by Elithrion · 2013-03-06T00:52:23.212Z · LW(p) · GW(p)
You meant $1 mil, right?
Replies from: shminux↑ comment by Shmi (shminux) · 2013-03-06T02:01:22.585Z · LW(p) · GW(p)
Right, sorry.
Replies from: MichaelVassar↑ comment by MichaelVassar · 2013-03-07T09:57:24.033Z · LW(p) · GW(p)
So about what do you think it IS worth? FYI, I think, based on experience with people whom have tried everything, that a 1% chance of finding something is unrealistically low. 20% with the first $5K and a further 30% with the next 35K would fit my past experience.
Replies from: EHeller↑ comment by EHeller · 2013-03-08T06:37:19.057Z · LW(p) · GW(p)
Define tried everything? Your prior is that there is a 1/5 chance a handful of researchers can find something helpful in 24 hours that isn't listed in something like an Up-To-Date report on the diagnosis (a decent definition of 'everything')?
Does Metamed do patient tracking to see if their recommendations lead to relief? Or do they deliver a report and move on?
Replies from: shokwave↑ comment by shokwave · 2013-03-08T07:20:40.159Z · LW(p) · GW(p)
Your prior is that there is a 1/5 chance a handful of researchers can find something helpful in 24 hours that isn't listed in something like an Up-To-Date report on the diagnosis (a decent definition of 'everything')?
From the body of the main post (source):
"It is frequently stated that it takes an average of 17 years for research evidence to reach clinical practice. Balas and Bohen, Grant, and Wratschko all estimated a time lag of 17 years measuring different points of the process."
Granted, I know very little about Up-To-Date, but I would be surprised if they completely eliminated that 17-year lag, especially in the more obscure conditions. They do, after all, have to cover all the conditions, and their return on investment is obviously going to be much higher on common conditions than on obscure ones. In fact, if they put out a fantastically detailed report on Stage III Boneitis (fictional) and nobody suffers a case that year, they've wasted their money. I strongly suspect Up-To-Date is aware of this, though I obviously have no way of knowing whether it affects their decisions.
MetaMed's offer is, as far as I understand it, "pay us 5k and we'll eliminate the 17-year lag for your particular case". This lets them plausibly offer value that Up-To-Date can't, in some cases.
Disclaimer: I am not associated with MetaMed, but I do think they're cool.
Replies from: EHeller↑ comment by EHeller · 2013-03-08T23:53:23.712Z · LW(p) · GW(p)
Have you actually read the metamed sample reports/what do you think metamed actually does? As far as I can tell, their core product is to have a team of medical and phd students do a literature search for about 1 working day (compare to Up-to-date, where actual researchers in various fields write the reports and clinicians edit the treatment plans). This seems highly unlikely to move that 17 year lag even a little bit.
I have no horse in this race, but I have worked as a statistician for hospital researchers and for health insurance companies. I just happen to think metamed's boosters here are dramatically underestimating the availability of evidence-based-medicine literature surveys in the clinical hospital setting.
↑ comment by Wakarimahen · 2013-03-06T06:41:26.064Z · LW(p) · GW(p)
Current prior: nothing has helped so far, so the odds of something she missed ended up being useful is pretty low.
This assumes she's good at sifting through the massive expanse of information available, and good at implementing the suggestions therein. These are two extremely questionable assumptions. Knowing nothing about her except that she has severe fibromyalgia and that she's the friend of a frequent poster on LW--two factors that hardly seem very relevant, and I'd put the likelihood of those two assumptions holding up to be very low. Quite bluntly, most people have no idea what's really out there. The Internet is a vast space.
↑ comment by pinyaka · 2013-03-06T14:38:30.876Z · LW(p) · GW(p)
That assumes they find only one thing that she hasn't tried. I have a sister with fibro and some cursory googling on my part suggests that there are so many theories out about what's wrong and what can bring relief that it's difficult to believe that a systematic search by good researchers will only turn up a single thing that she hasn't tried. That said, you're right that 1% helpfulness is probably pretty optimistic.
↑ comment by Wakarimahen · 2013-03-06T06:28:16.191Z · LW(p) · GW(p)
All possible conventional and alternative medicine? I doubt it. This is a mind-destroying sentence if I ever saw one. I'd suggest re-wording it to "she's tried a ton of different approaches both from conventional and alternative medicine".
First thing to be said: Fibromyalgia is one of those health issues where there are no widely adopted hypotheses for the base mechanism at work. This means, quite simply, that there is little hope for targeting it specifically. It's not a case where e.g. your lips are chapped and your knuckles are splitting, and one of the first places you look is hydration--more water, more trace minerals, etc. Instead it's a health issue where you have nothing to target, and your only real hope is to do whatever you can to improve your general health, and hope whatever the yet-to-be-discovered underlying cause is taken out by fortunate accident.
Look to the other symptoms. What other symptoms does she have? It doesn't matter whether they're considered to be related. Constipation, headaches, splitting nails, PMS, dry skin, cold extremities, dandruff, frequent colds, dizziness upon standing too quickly, acne... anything at all. Note it, target it, fix it. Keep doing this for years. Make a checklist. Anything to be considered a symptom. Notice it, treat it, move on. Do this for a long enough time, and either the fibromyalgia will go away or get better, or it won't. But at least you tried, and believe me: Her life will be better either way. Well, unless she doesn't like hard work.
Potential leads I found through a few brief Google searches:
http://www.jonbarron.org/article/fibromyalgia-goes-pharmaceutical
http://evolutionarypsychiatry.blogspot.com/2012/02/magnesium-deficiency-and-fibromyalgia.html
http://www.westonaprice.org/miscellaneous/fibromyalgia
http://paleohacks.com/questions/133865/what-are-the-causes-of-fibromyalgia#axzz2Mjjvmzq6
http://paleohacks.com/questions/1990/paleo-and-fibromyalgia#axzz2Mjjvmzq6
Good luck.
Replies from: shminux↑ comment by Shmi (shminux) · 2013-03-06T16:55:36.645Z · LW(p) · GW(p)
Not sure why the parent is upvoted so much. Trivial and rather useless advice, some platitudes, a few rather suspect google hits (paleohacks? really?), and a veiled insult "unless she doesn't like hard work".
Replies from: Wakarimahen↑ comment by Wakarimahen · 2013-03-06T19:37:24.548Z · LW(p) · GW(p)
I'm surprised as well. I expected to be downvoted to -2 or so pretty quickly, and stay around there.
As for your disagreements, I should stress that what I said is perhaps the absolute most important thing for the average person with a health issue like that to hear. All too many people get hung up on trying to target the problem specifically, when they're dealing with an issue where doing so is not practical. Day after day, they ask, "What causes fibromyalgia? What are the new treatments suggested for it?" They remain fixated on these questions, while they sweep all sorts of other symptoms under the rug--random symptoms like headaches or splitting nails, which may be coming from the same source.
As for the Google hits, I'm not sure why you're calling them suspect. Jon Barron is one of the best alternative health writers out there, the Weston A. Price Foundation has a huge following, PaleoHacks is perhaps the best forum on paleo (which is a diet and lifestyle with a massive following), and the other link is a blog that I've seen cited a bunch of times in paleo circles as being someone who is less likely than average to fall for various forms of silliness.
Is this enough evidence to suggest you should read the links and take them seriously? No idea. They have a lot of links within them though. My goal was to as quickly as possible find some articles that put the conditions for 'tab explosion' in place in a way I thought would be beneficial. Generally when conventional medicine doesn't have the answer, the best place to look is where people are talking about paleo. Even stereotypically non-paleo things like raw vegan juicing, such as the Gerson Diet, will come up in paleo circles--quite simply because it seems to work.
↑ comment by buybuydandavis · 2013-03-06T13:14:56.171Z · LW(p) · GW(p)
I recommend looking into low dose naltrexone. Cheap, safe, and with reported success for fibromyalgia (I haven't looked into that use in particular). Generally appropriate for pain issues, as it is an opioid receptor antagonist with over night use modeled (and perhaps verified) as upregulating opioid receptors, and thereby pain relief, during the day. I believe D-Phenylalanine limits breakdown of opioids, and would be another cheap, safe, and effective addition to this treatment.
Also, the stimulation of gnrh release is likely generally helpful in people past 30.
Recent small study out of Stanford on LDN for Fibromyalgia: http://www.ncbi.nlm.nih.gov/pubmed/23359310
If you're interested, message me and I'll send you where I get it cheap through a compounding pharmacy in NYC.
Replies from: shminux↑ comment by Shmi (shminux) · 2013-03-06T17:21:58.103Z · LW(p) · GW(p)
I have glanced at the abstract, and the study appears to be deeply flawed. They see significant self-reported pain reduction in about 30% of the patients vs 20% for placebo. Whoa, big deal. What are the odds that another study would replicate these results? Moreover, they did not compare it with the mainline painkillers like acetaminophen or ibuprofen, or anything else cheap currently on the market, or with the classic fibro drugs, like pregabalin. To sum up, there is zero reason to try it specifically, except maybe as one of the many random things to try in desperation.
Replies from: buybuydandavis↑ comment by buybuydandavis · 2013-03-06T18:41:07.571Z · LW(p) · GW(p)
They see significant self-reported pain reduction in about 30% of the patients vs 20% for placebo.
That's not how I read the abstract. It's not % of patients, but % of pain reduction, whatever that means. I assume they're referring to some self reported numeric pain scale.
The percentage of responders for "significant pain reduction" was 32% naltrexone vs 11% placebo , which strikes me as significant. If I had serious pain, and someone offered me a "widely available, inexpensive, safe, and well-tolerated" treatment with a 30% chance of "significant pain reduction" I'd be all over it. Your mileage may vary.
To sum up, there is zero reason to try it specifically, except maybe as one of the many random things to try in desperation.
Really? "Zero reason"? So you predict equal efficacy as a random treatment, such as spinning around and squawking like a chicken?
Plenty of reasons. You don't have to like them or know of them. I wasn't attempting or claiming to prove anything, just trying to point you to some information I thought would be helpful. I hope she finds something.
↑ comment by John_Maxwell (John_Maxwell_IV) · 2013-03-06T07:42:16.172Z · LW(p) · GW(p)
I've read that some of the pain in fibromyalgia typically comes from trigger points; has she researched those?
comment by [deleted] · 2013-03-05T13:51:43.807Z · LW(p) · GW(p)
I am under the impression that IBM's Watson is being tested in a few hospitals for something which seems at least somewhat similiar to Metamed, but I don't know enough about either to really judge well. Sample link to what I am referring to:
Is anyone familiar enough with both Metamed and Watson to help me compare and contrast the support provided by the two of them?
Replies from: atucker, Vaniver↑ comment by atucker · 2013-03-05T17:38:32.934Z · LW(p) · GW(p)
From what I understand, Watson is more supposed to do machine learning and question answering in order to do something like make medical diagnoses based on the literature.
MetaMed tries to evaluate the evidence itself, in order to come up with models for treatment for a patient that are based on good data and an understanding of their personal health.
They both involve reviewing literature, but MetaMed is actually trying to ignore and discard parts of the literature that aren't statistically/logically valid.
↑ comment by Vaniver · 2013-03-06T01:06:31.666Z · LW(p) · GW(p)
My view:
MetaMed is designed to extract the maximal amount of information out of the medical research community that exists today. Much of their value-add involves 'meta' evidence that is difficult for others to collect or interpret. (A doctor may be skilled at understanding how a part of the body works, but not how the medical research community actually works.) If you have a condition that is serious, rare, or strange enough that investing $5k in making medical attention more effective seems like a good idea, then you should talk to MetaMed. MetaMed is in no way a substitute for doctors; it's a way to find which doctors you should be talking to, and about what.
Watson can be a substitute for doctors. The key enabler for Watson is massive amounts of data on patients, and the statistical knowledge to make good use of that data. One of the things to remember here is that expert diagnosis systems have been around for a long time, but that if you're expert enough to prepare the relevant information for the computer you're probably expert enough to make an okay guess yourself, at which point using the computer doesn't seem very high priority. Eventually, Watson will enable patients and nurses to input most of the necessary information using natural language. It doesn't look like Watson is a substitute for medical research, but is rather a complement to it- if you have all the patient data together, you can build great models, and great models allow for superior discoveries. (Watson might eventually be able to automate parts of the hypothesis-generating and testing aspects of medical research, but I expect humans to have strong to moderate comparative advantage here for at least two decades.)
The short version: MetaMed makes better use of existing evidence that anyone else; Watson will generate a river of new evidence that will dramatically alter all parts of medicine.
Replies from: Kenny↑ comment by Kenny · 2013-03-31T23:09:02.753Z · LW(p) · GW(p)
[Emphasis mine]
Eventually, Watson will enable patients and nurses to input most of the necessary information using natural language.
Dear Bayes, I hope not! I'd hope there's much more precise info that could be input instead.
Replies from: Vanivercomment by ema · 2013-03-05T18:08:05.101Z · LW(p) · GW(p)
According to their site Jaan Tallinn is not the CEO but chairman of the board. Zvi Mowshowitz is the CEO.
Replies from: thescoundrel, Mycroft65536↑ comment by thescoundrel · 2013-03-06T15:57:20.217Z · LW(p) · GW(p)
Wow- that is former MTG Pro Zvi, one of the best innovators in the game during his time. Awesome to see him involved in something like this.
↑ comment by Mycroft65536 · 2013-03-05T22:52:08.519Z · LW(p) · GW(p)
Jaan is also the CTO, I'm not sure if that's on the website.
comment by KnaveOfAllTrades · 2013-03-06T19:34:30.343Z · LW(p) · GW(p)
Have enough people at MetaMed been influenced sufficiently by (meatspace) LessWrong/think 'similarly enough' to LW rationality that we should precommit to updating by prespecified amountson the effectiveness of LW rationality in response to its successes and failures?
Replies from: gwern, MichaelVassar, dreeves↑ comment by gwern · 2013-03-08T04:58:47.209Z · LW(p) · GW(p)
At a first glance, I'm not sure humans can update by prespecified amounts, much less prespecified amounts of the right quantity in this case: something like >95% of all startups fail for various reasons, so even if LW-think could double the standard odds (let's not dicker around with merely increasing effectiveness by 50% or something, let's go all the way to +100%!), you're trying to see the difference between... a 5% success rate and a 10% success rate. One observation just isn't going to count for much here.
↑ comment by MichaelVassar · 2013-03-07T10:02:05.582Z · LW(p) · GW(p)
Definitely, though others must decide the update size.
↑ comment by dreeves · 2013-03-07T21:09:34.717Z · LW(p) · GW(p)
Interesting question! Since it's an especially interesting question for those not fully in the in-crowd I thought it might be worth rephrasing in less technical language:
Is MetaMed comprised of LessWrong folks or significantly influenced by LessWrong folks, or that style of thinking? If so, this sounds like a great test of the real-world efficacy of LessWrong ideas. In other words, if MetaMed succeeds that's some powerful evidence that this rationality shit works! (And to be intellectually honest we have to also precommit to admitting that -- should MetaMed fail -- it's evidence that it doesn't.)
PS: Since Michael Vassar is involved it's safe to say the answer to the first part is yes!
Replies from: aliceycomment by Daniel Kokotajlo (daniel-kokotajlo) · 2021-02-10T16:15:41.974Z · LW(p) · GW(p)
Is there a postmortem somewhere of why this didn't work? Ah, I see there is.
Replies from: austin-chen↑ comment by Austin Chen (austin-chen) · 2021-02-10T16:36:51.838Z · LW(p) · GW(p)
Thank you for posting this; I found it very helpful in clarifying some things with my current startup. And thanks to Zvi (presumably) for writing this too!
Replies from: daniel-kokotajlo↑ comment by Daniel Kokotajlo (daniel-kokotajlo) · 2021-02-10T17:36:05.113Z · LW(p) · GW(p)
I think Sarah Constantin is the author not Zvi
comment by Dr_Manhattan · 2013-03-05T00:44:07.130Z · LW(p) · GW(p)
it's spam, but it's our spam. upvoted. (I don't mean I work for Meta; I just support the Cause and things/community supporting it)
Replies from: atucker↑ comment by atucker · 2013-03-05T17:31:03.584Z · LW(p) · GW(p)
Upvoted, but I'm a bit confused as to what we're trying to refer to with "spam".
If by spam we mean advertising, yes. Definitely.
If by spam we mean undesirable messaging that lowers the quality of the site, then I would think that this is very much not spam.
Replies from: Qiaochu_Yuan, Dr_Manhattan↑ comment by Qiaochu_Yuan · 2013-03-05T17:50:02.820Z · LW(p) · GW(p)
Some people (myself included) use "spam" to refer to any kind of advertising in a public setting, e.g. you might preface an email sent out to multiple mailing lists as "sorry for the spam, guys, but..." even if it's a valuable and high-quality email. The connotation, to me, is mildly self-deprecating rather than strictly negative.
↑ comment by Dr_Manhattan · 2013-03-05T18:38:02.499Z · LW(p) · GW(p)
If this startup was not associated with MIRI I would downvote it; there are lots of great startups but this is not the place to advertise them.
Replies from: MugaSofer↑ comment by MugaSofer · 2013-03-06T09:23:10.887Z · LW(p) · GW(p)
It's medicine, done rationally. This is a site about rationality. The relevance seems clear regardless of it's origin.
Replies from: Dr_Manhattan↑ comment by Dr_Manhattan · 2013-03-06T19:36:03.940Z · LW(p) · GW(p)
A lot of businesses could have "done rationally" appended to them. MetaMed is "medicine, done rationally" (using statistics). Google is "search done rationally"(with statistics). The only reason medicine stands out is due the the rather poor baseline.
Replies from: wedrifid↑ comment by wedrifid · 2013-03-07T12:10:07.373Z · LW(p) · GW(p)
The only reason medicine stands out is due the the rather poor baseline.
Alternative words to "only" include "valid" and "sufficient".
Google is "search done rationally"(with statistics).
Your example doesn't support your intended conclusion. In a world with irrational and often unhelpful search engines and an unknown, newly formed "Google", it would be entirely appropriate to make people aware of it, in a similar post to this one.
comment by Rukifellth · 2013-06-02T16:09:43.442Z · LW(p) · GW(p)
Seeing these statistics has got me thinking.
I've checked the undergraduate course requirements as my local university's medical faculty, and there's nothing listed for probability and statistics. I'm considering setting up an appointment with somebody about this, assuming doctors not being able to uncover test results properly is a serious problem.
Would this be worth it, or am I wasting time?
Replies from: Eliezer_Yudkowsky↑ comment by Eliezer Yudkowsky (Eliezer_Yudkowsky) · 2013-06-02T19:32:25.834Z · LW(p) · GW(p)
I hate to say it, but my guess is that you're wasting time unless your universe^H^H^H university has unusually good undergraduate statistics courses.
Replies from: gwern, Vaniver↑ comment by gwern · 2013-06-03T01:12:59.769Z · LW(p) · GW(p)
I don't think it's a waste of time. If you pay attention in your introductory courses, you'll learn a good chunk of how to abuse NHST and what the criticisms of it mean. I have learned very little Bayesian statistics, but for trying to understand the very large existing medical/psychological research corpus, I have never regretted focusing my reading on frequentist material.
Replies from: Eliezer_Yudkowsky, Rukifellth↑ comment by Eliezer Yudkowsky (Eliezer_Yudkowsky) · 2013-06-03T01:20:09.563Z · LW(p) · GW(p)
I defer to your superior domain knowledge of universities.
Replies from: gwern↑ comment by gwern · 2013-06-03T01:27:48.338Z · LW(p) · GW(p)
You don't have to; you can see CMU's "Probability & Statistics" for yourself, for example.
↑ comment by Rukifellth · 2013-06-04T02:52:11.788Z · LW(p) · GW(p)
Oh not for me- I'm doing CS, but it seems like we could get very large returns in hospital performance for the effort expended in teaching med students the proper stats training.
I'm not sure what to expect here, except that at best they'll flat out say that the program is difficult enough as it is, and at worst shrug with some kind of vague "corporate-representative-being-questioned" answer. In my wildest dreams they could come up with some new-fangled "Life Stats" course, streamlined so only the parts related to diagnostics and prognostics are taught.
comment by Shmi (shminux) · 2013-03-21T17:44:20.571Z · LW(p) · GW(p)
Hanson recently commented on MetaMed on OB, but not here, so might as well quote some of it:
I wrote this post because I know several of the folks involved, and they asked me to write a post endorsing MetaMed. And I can certainly endorse the general idea of second opinions; the high rate and cost of errors justifies a lot more checking and caution. But on what basis could I recommend MetaMed in particular? Many in the rationalist community think you should trust MetaMed more because they are inside the community, and therefore should be presumed to be more rational.
But any effect of this sort is likely to be pretty weak, I think. Whatever are the social pressures than tend to corrupt the usual medical authorities, I expect them to eventually corrupt successful new medical firms as well. I can’t see that being self-avowed rationalists offers much protection there. Even so, I would very much like to see a much stronger habit of getting second opinions, and a much larger industry to support that habit. I thus hope that MetaMed succeeds.
This is a rather lukewarm and highly qualified endorsement, if you can call it that. The rest of the post is also a worthwhile read, as a critical assessment of LW cultishness:
As with religion, the main problem comes when a self-described rationalist community starts to believe that they are in fact much more rational than outsiders, and thus should greatly prefer the beliefs of insiders.
[...]
Replies from: KawoombaI’ve noticed a substantial tendency of folks in this rationalist community to prefer beliefs by insiders, even when those claims are quite contrarian to most outsiders. Some say that since most outsiders are quite irrational, one should mostly ignore their beliefs. They also sometimes refer to the fact that high status insiders tend to have high IQ and math skills. Now I happen to share some of their contrarian beliefs, but disagree with many others, so overall I think they are too willing to believe their insiders, at least for the goal of belief accuracy.
↑ comment by Kawoomba · 2013-03-21T18:26:53.556Z · LW(p) · GW(p)
It is indeed a kind of poisoned 'endorsement' that MetaMed could have done without, a Nessus' tunic. It's telling that he obviously didn't run it by MetaMed before publishing it, or didn't straight out decline to give a recommendation, instead of wrapping it in "I expect it will be corrupted like all the others". It's surprising after reading his recent elogy on Yvain, who's involved with MetaMed.
comment by CCC · 2013-03-05T12:15:01.829Z · LW(p) · GW(p)
This seems like a really good idea. Especially given the impossibility of a single doctor keeping up with all the literature...
Moreover, I rather expect Metamed to be able, ater a while, to suggest profitable research opportunities to people looking to do medical research.
comment by NancyLebovitz · 2013-03-06T05:30:40.763Z · LW(p) · GW(p)
This is a new service and it has to interact with the existing medical system, so they are currently expensive, starting at $5,000 for a research report. (Keeping in mind that a basic report involves a lot of work by people who must be good at math.) If you have a sick friend who can afford it - especially if the regular system is failing them, and they want (or you want) their next step to be more science instead of "alternative medicine" or whatever - please do refer them to MetaMed immediately.
What might it be worth to people to find out that some or all of the usual procedures are so dangerous and/or ineffective as to be not worth doing?
Replies from: Mycroft65536↑ comment by Mycroft65536 · 2013-03-06T05:57:21.580Z · LW(p) · GW(p)
Likely more than the list price of those procedures. People who have expensive potentially harmful procedures being done on them would get great benefits having MetaMed review those procedures.
comment by Tyrrell_McAllister · 2013-03-05T05:48:52.878Z · LW(p) · GW(p)
Statistical and Health Illiteracy (Patients)
Is this a placeholder for more citations?
Replies from: Eliezer_Yudkowsky↑ comment by Eliezer Yudkowsky (Eliezer_Yudkowsky) · 2013-03-05T16:22:37.456Z · LW(p) · GW(p)
I accidentally left that in after deleting the section underneath it. Like I said, this was only a fraction of their total citations list.
comment by prase · 2013-03-09T21:17:40.445Z · LW(p) · GW(p)
I'd be interested in the linked Begg's paper but it's behind a paywall. Can someone please tell what exactly they had done and how did they obtain all those various p-values?
Replies from: gwern↑ comment by gwern · 2013-03-09T22:30:15.492Z · LW(p) · GW(p)
"On inferences from Wei's biased coin design for clinical trials". You can always request fulltexts here.
Replies from: prasecomment by Decius · 2013-03-07T09:02:38.821Z · LW(p) · GW(p)
What are the capital investments that need to be recouped in the early adopter period? Is the price tag based on "It is worth more than this to our target market.", or "It costs this much to do this research, with reasonable amortizing of capital costs."?
Replies from: Strange7, Kawoomba↑ comment by Strange7 · 2013-03-10T05:29:47.425Z · LW(p) · GW(p)
At a guess: computer hardware, office space, recruiting competent people, training them to work together effectively, and a well-organized library of the company's previous reports so that not every request requires them to reinvent the wheel.
Replies from: Decius↑ comment by Decius · 2013-03-10T05:40:50.329Z · LW(p) · GW(p)
Computer hardware and support adequate for the large-scale implementation is roughly 500k capital and 240K/yr; office space incl utilities should be in the realm of $500k/yr (again, for a large-scale operation) and one hundred competent people should be about $4m/year; with another $1m for management expenses. Total capital+first year's operating expenses is ~$6m; if they expect to sell a thousand reports per year (over one man-month each) at $5k each, they repay the investors almost in the first year.
I haven't tried to price the custom software involved, but for such a (in the large business sense) small investment I don't see why they didn't start full-scale.
Replies from: Larks↑ comment by Kawoomba · 2013-03-07T09:27:21.070Z · LW(p) · GW(p)
Is price ever based on anything other than "this will maximize our revenue over this period of time"?
Replies from: Decius↑ comment by Decius · 2013-03-07T17:30:20.511Z · LW(p) · GW(p)
The terminology you wanted was 'maximize our profit'. And yes, some pricing is based on a goal other than maximizing profit.
Replies from: Jayson_Virissimo, wedrifid↑ comment by Jayson_Virissimo · 2013-03-14T06:22:18.044Z · LW(p) · GW(p)
...some pricing is based on a goal other than maximizing profit.
I've worked at three jobs where the firm was not even in the ballpark of approximately maximizing profit. The first is now out of business. The second and third were in government.
Replies from: Decius↑ comment by wedrifid · 2013-03-11T18:14:54.571Z · LW(p) · GW(p)
And yes, some pricing is based on a goal other than maximizing profit.
For example, it is sometimes based on the goal "Maximize the bonus given to the CEO".
Replies from: Decius↑ comment by Decius · 2013-03-11T20:09:28.596Z · LW(p) · GW(p)
I was referring to corporations with core values that don't involve Laffer peaks of money.
Replies from: wedrifid↑ comment by wedrifid · 2013-03-12T02:45:27.117Z · LW(p) · GW(p)
I was referring to corporations with core values that don't involve Laffer peaks of money.
This position wasn't challenged, missed, nor even weakened by my reply. Rather, it was strengthened by the agreement with the actual claim you made.
Replies from: Decius↑ comment by Decius · 2013-03-14T02:13:56.573Z · LW(p) · GW(p)
Sorry; it seemed to me that you were agreeing with the claim I made by subverting the intent. As someone who intends to create and invest heavily into a corporation with goals other than maximizing the taxable investment income I receive, it is a sensitive subject to me.
Replies from: wedrifid↑ comment by wedrifid · 2013-03-14T02:24:49.022Z · LW(p) · GW(p)
Sorry; it seemed to me that you were agreeing with the claim I made by subverting the intent. As someone who intends to create and invest heavily into a corporation with goals other than maximizing the taxable investment income I receive, it is a sensitive subject to me.
Exciting (and brave, considering the failure rate). I'm curious... what industry/goal if you don't mind sharing?
Replies from: Decius↑ comment by Decius · 2013-03-14T02:48:18.364Z · LW(p) · GW(p)
Coffee shop/social justice. Its a planned attempt to deinforce class division by making the middle-class investor(s) a little richer while making the working-class employees each absolutely more richer.
I'm currently have someone with the philosophical background but not quite enough business training to serve as general manager. I've figured a low six-digit upfront investment along with a couple years to get a fully-qualified general manager, a couple more hundred $k in capital costs, and another in operating losses each year for four years, leading to a recoup starting eight years after beginning. I've got about half that now, and just need to get my day job adjusted and settled to a better location to oversee the operation and expect to cover the remaining investment out of income. (Before I actually start, I'm going to develop enough of a plan to know how much reality deviates from the plan at any given point)
comment by minusdash · 2015-03-03T01:02:04.417Z · LW(p) · GW(p)
What this company is doing might indirectly lead to very dangerous situations. They are promoting an anti-establishment, private, non-supervised spinoff version of medicine, where they don't give medical advice but give advice on how to try getting treated by medical professionals. By this they are creating distrust in mainstream medicine and this blog post is helping them in their dismissal of medicine as practiced in legally supervised hospitals and medical establishments, since this post lists studies which show that medicine is "bad" but shows no studies about whether MetaMed is any better. There is a reason why healthcare professionals have to go through certain legal procedures before they are allowed to give medical advice.
MetaMed is not a licensed healthcare provider. ALL INFORMATION AND CONTENT IS PROVIDED FOR GENERAL INFORMATIONAL AND EDUCATIONAL PURPOSES ONLY AND IS NOT MEANT TO BE AND IS NOT A SUBSTITUTE FOR PROFESSIONAL ADVICE, DIAGNOSIS OR TREATMENT PROVIDED BY A PHYSICIAN OR OTHER QUALIFIED AND LICENSED HEALTHCARE, CLINICAL OR MEDICAL PROFESSIONAL PURSUANT TO PERSONAL IN-PERSON EXAMINATION AND PROFESSIONAL CONSULTATION. The information and content is neither intended to dictate what constitutes reasonable, appropriate or best care for any given health issue, nor is it intended to be used as a substitute for the independent judgment of a physician or other healthcare professional for any given health issue.
I find this absolutely shocking and reading the endorsement of this company on this website makes me seriously reconsider what is going on around here.
Replies from: Lumifer↑ comment by Lumifer · 2015-03-03T01:10:30.124Z · LW(p) · GW(p)
They are promoting an anti-establishment, private, non-supervised spinoff version of medicine
That's an excellent idea and I endorse it :-P Particularly the "non-supervised" part. I am not quite sure what do you mean by "spinoff", though. What is spun off what?
as practiced in legally supervised hospitals and medical establishments
What exactly is a "legally supervised hospital" and who's doing the supervising?
There is a reason why healthcare professionals have to go through certain legal procedures before they are allowed to give medical advice.
Do tell. Is it to make lawyers rich?
Replies from: minusdash↑ comment by minusdash · 2015-03-03T01:55:28.228Z · LW(p) · GW(p)
By spinoff I mean that this company takes medicine's research results and then say they can handle and aggregate it better than the doctors who are licensed to do exactly that and then they start to do at least a part of it (giving certain limited kinds of advice or certain limited kinds of health reports that aren't legally considered medical advice).
Playing with medicine is a dangerous field. For example this post induces distrust in legally-professed medicine at a time when "alternative medicine" is getting more and more popular (homeopathy, Chinese traditional "medicine", etc.). Dismissing experts and starting your own better version of the field without actually formally, academically challenging the status quo and showing how your method would be better, seems very wrong to me. They should go and reform medicine to be the way it should be (if it really is broken) from the inside and by taking the legal burden of giving out actual medical advice. If their method works, they should publish their superior results in peer-reviewed medical journals and get their methods into regular practice. Have they done such a thing? I couldn't find such a thing on their website.
They essentially say that they will give some non-medical advice but the real weight of potential legal ramifications still has to lie on the (according to this post) incompetent licensed medical professional. If he disagrees with the company, the patient may get angry for having wasted the money, so they may look for a licensed doctor who is willing to do the stuff that the company's literature review might indicate.
The way this post dismisses medical experts reminds me of the way Scientology dismisses psychiatry.
Do tell. Is it to make lawyers rich?
No, for patient safety.
(Certainly, MetaMed doesn't endanger patient safety by giving out reports that have to be acknowledged by a licensed doctor, but they make it look as if the doctor couldn't decide himself and needed external help. This in turn decreases the trust in medicine and could potentially increase the trust in all of the alternative crap like homeopathy and faith healing, too.)
It's somewhat like hiring legally-not-responsible people who'd keep telling civil engineers who are experts at bridge design how they should change stuff here and there and that they don't really know what they are doing.
Replies from: Lumifer↑ comment by Lumifer · 2015-03-03T02:50:33.950Z · LW(p) · GW(p)
this company takes medicine's research results and then say they can handle and aggregate it better than the doctors who are licensed to do exactly that
Doctors are not licensed to "handle and aggregate" research results. They are licensed to treat people as best as they can and keeping up with the latest academic research is not a requirement for keeping their license. In fact, most doctors are too busy treating people to allocate enough time to read research.
this post induces distrust in legally-professed medicine
I consider this a good thing and I say this as someone who has doctors in the family and doctors among friends. Medics are not gods and not even engineers -- an attempt to make medicine "evidence-based" is proceeding over a lot of screaming and dragging heels. In some areas -- like e.g. surgery -- contemporary medicine is pretty good. In other areas -- like e.g. nutrition -- it's pretty bad.
And what's up with your obsession with "legally-professed"? Getting a license doesn't magically grant you smarts and competence. There are a lot of different doctors and some of them are really not good doctors.
...without actually formally, academically challenging the status quo
I believe their advice is based on published medical research which is precisely what you are asking for.
Replies from: EHeller, minusdash↑ comment by EHeller · 2015-03-03T02:56:35.131Z · LW(p) · GW(p)
I think the bigger issue with metamed is that they are trying to fill a role that already exists in the market. UpToDate is a bigger, more ambitious company (instead of generating per-patient reports they have a huge database of research summaries curated by actual experts for each disease) that almost all academic medical centers already pay for.
↑ comment by minusdash · 2015-03-03T02:59:23.416Z · LW(p) · GW(p)
I just found Robin Hanson's critique and it's almost the same as my opinion.
My obsession over legally-professed is that it's a lot easier to give actionable options if there is no legal risk for doing so. Why won't they take all the legal risks of giving actual medical advice instead of just actionable options?
Replies from: Lumifer↑ comment by Lumifer · 2015-03-03T04:27:10.756Z · LW(p) · GW(p)
it's almost the same as my opinion
Really? You consider MetaMed unethical and dangerous. Robin Hanson considers it a useful source of second opinions but thinks it may not be all that much better than second opinions from other sources (e.g. doctors).
You say "I find this absolutely shocking and reading the endorsement of this company on this website"
Robin Hanson says "Even so, I would very much like to see a much stronger habit of getting second opinions, and a much larger industry to support that habit. I thus hope that MetaMed succeeds."
if there is no legal risk for doing so
MetaMed has been granted immunity from lawsuits..?
The legal risks are lesser if you are a licensed MD.
Replies from: minusdash↑ comment by minusdash · 2015-03-03T15:07:22.413Z · LW(p) · GW(p)
Upon more reflection, I'm not able to defend my point and my thoughts are confused and therefore I'm gravitating towards the established and mainstream viewpoint that only licensed and authorized doctos should do doctor stuff. On uncertain territory it's better to stick to well-known landmarks. Since I'm not confident in my capability of a deep enough analysis of the pros and cons, I feel that the way to convince me would be to first convince people who are experts of the medical field and of the regulations, towards whom I already have an established chain of trust.
Replies from: Lumifer↑ comment by Lumifer · 2015-03-03T15:57:57.767Z · LW(p) · GW(p)
Yay for more reflection! :-)
I would recommend to continue with even more reflection, now about that chain of trust you say you have established. Ioannidis would probably be helpful and you can google up his actual papers.
Replies from: minusdash↑ comment by minusdash · 2015-03-03T16:36:13.272Z · LW(p) · GW(p)
Yes I'm familiar with his most famous paper and what he says about medical research findings. Has he ever endorsed MetaMed in particular? If peer reviewed research finding are often false, how can MetaMed tell the difference without trying to replicate them? Different research papers use different assumptions, differently calibrated measurements, different subjects, it seems very hard to aggregate this in practice, although I'm not a medical researcher. Why should I believe that a company started by futurists and entrepreneurs would be up to this task? Where is the evidence for the actual efficacy of their particular methodology, as evaluated by independent third-parties?
Replies from: Lumifer↑ comment by Lumifer · 2015-03-03T18:09:34.753Z · LW(p) · GW(p)
it seems very hard to aggregate this in practice
Yes, it is. However there is, for example, the Cochrane Collaboration which is dedicated to exactly that.
Why should I believe that a company started by futurists and entrepreneurs would be up to this task?
You should not. I am not arguing that MetaMed is better than everyone else or even that it is very good. I am arguing that it's not evil, not dangerous (relative to the usual baseline), and a useful thing to have around.
It's goal is not to provide you with THE TRUTH, it's goal is to give you a digestible summary of the current research on topics of particular interest to you. Often this summary functions as a second opinion, or it could provide the context for making medical decisions. It is as fallible as the rest of contemporary medicine.