Ensembling the greedy doctor problem

post by Ryan Kidd (ryankidd44) · 2022-04-18T19:16:00.916Z · LW · GW · 12 comments

Contents

13 comments

The greedy doctor problem [LW · GW] is defined as follows: suppose you want a possible illness to be diagnosed correctly, but your doctor is incentivized to diagnose you sick because they can then charge you more for treatment. How do you get an accurate diagnosis and avoid paying more than you need?

A naive solution is to get a second opinion. However, a second doctor is also incentivized to diagnose you ill! What if you leverage adversarial dynamics [LW · GW] and pay one doctor for treatment only if both doctors agree you are ill and the other only if both doctors agree you are hale? Unfortunately, this instantiates a form of the "battle of the sexes" game and the doctors randomise their diagnoses according to a mixed-strategy Nash equilibrium. Neither of these approaches reliably gets you diagnosed correctly.

Assuming the doctors cannot coordinate through signalling, I think ensembling might solve the problem in the absence of unforseen acausal coordination. My (totally impractical) strategy is as follows:

• I assemble a host of patients (ideally independent and identically distributed in symptoms) and submit them to both doctors for diagnosis.
• Each doctor is told that they will only be paid if they diagnose all patients identically to the other doctor, with the exception that neither will be paid if they diagnose all patients ill or hale (this assumes that at least one patient is not ill).
• If the doctors each try to diagnose only a few patients as hale, it is unlikely that they will choose the same subset of patients. The doctors are thus incentivized to coordinate on a Schelling point, ideally "telling the truth".

There are three obvious Schelling points: diagnosing all patients ill; diagnosing all patients hale; and diagnosing all patients truthfully (assuming that the doctors agree). Likely, at least one patient will be genuinely ill and at least one will be genuinely hale. The doctors can get more money if they diagnose some patients as ill, which biases them away from the all-hale Schelling point. The doctors will not get paid if they diagnose all patients as ill, which biases them away from the all-ill Schelling point. Thus, the only remaining obvious Schelling point is all-truthful.

It is possible that the doctors could acausally coordinate on an alternative Schelling point. For example, there might be mutually exploitable patterns in the data or precommited doctor strategies (e.g. from medical school, the scoundrels) that allow the doctors to diagnose more patients as ill and thus receive a higher pay. If any untruthful Schelling point is known to the game overseer, they can disincentivise coordination on it by refusing to pay the doctors if they coordinate on it. Thus, only unforseen Schelling points are dangerous in single-shot iterations of the greedy doctor game.

In iterated forms of this game, it is important to not tell the doctors why they were or were not paid to prevent signalling and coordination. Even a simple distance measure between the doctors' diagnoses, such as the Hamming distance, might allow the doctors to coordinate over time. If the doctors only receive the "paid/unpaid" signal, in the absence of acausal coordination, they will likely search in the vicinity of a Schelling point until the "paid" signal is received. If the number of game iterations is large and the patient number is small, the doctors will probably coordinate away from the all-truthful Schelling point. Thus, increasing the number of patients is critical to prevent trivial coordination over time. Increasing the number of doctors also can reduce the probability of trivial coordination.

If the game overseer is confident that some minimum percentage of patients is genuinely hale, they can refuse to pay the doctors if their diagnosis ensembles agree within a certain Hamming distance of the known untruthful Schelling points. This makes it harder for the doctors to trivially coordinate, as the set of diagnosis ensembles in the vicinity of the Hamming distance perimeter is large.

12 comments

Comments sorted by top scores.