A Small Negative Result on Debate

crossposting my comments from Slack thread:

Here are some debate trees from experiments I did on long-text QA  on this example short story:

Tree

Debater view 1

Debater view 2

Our conclusion was that we don’t expect debate to work robustly in these cases. In our case this was mostly because in cases where the debate is things like ’is there implied subtext A?’,  human debaters don’t really know why they believe some text does or doesn’t have a particular implication. They have some mix of priors about what the text might be saying (which can’t really be justified with debate), and various updates to that based on style, word choice, etc, where humans don’t necessarily have introspective access to what exactly in the text made them come to the conclusion.My guess is that’s not the limitation you’re running into here - I’d expect that to just be the depth.

There are other issues with text debates, like if the evidence is distributed across many quotes that each only provide a small amount of evidence - in this case the honest debater needs to have decent estimates for how much evidence each quote provides, so they can split their argument into something like ‘there are 10 quotes that weakly support position A’; ‘the evidence that these quotes provide is additive rather than redundant’.

[edited to fix links]

Update: We did a quick follow-up study adding counterarguments, turning this from single-turn to two-turn debate, as a quick way of probing whether more extensive full-transcript debate experiments on this task would work. The follow-up results were negative. 

Tweet thread here: https://twitter.com/sleepinyourhat/status/1585759654478422016

Direct paper link: https://arxiv.org/abs/2210.10860 (To appear at the NeurIPS ML Safety workshop.)

We're still broadly optimistic about debate, but not on this task, and not in this time-limited, discussion-limited setting, and we're doing a broader more fail-fast style search of other settings. Stay tuned for more methods and datasets.

I think that one of the key difficulties for debate research is having good tasks that call for more sophisticated protocols. I think this dataset seems great for that purpose, and having established a negative result for 1-turn debate seems like a good foundation for follow-up work exploring more sophisticated protocols. (It seems like a shame that people don't normally publish early-stage and negative results.)

In comparison with other datasets (e.g. in the negative results described by Beth), it seems like QuALITY is identifying pretty crisp failures and is within striking distance for modern ML. I haven't looked at the dataset beyond the samples in the paper but tentatively I'm pretty excited about more people working on it (and excited to see future work from your group!)

I do strongly suspect that multi-turn debates could handle these questions, and if not it would be a pretty significant update about debate / the nature of human reasoning / etc. I think it's possible those debates would have to get pretty complicated, and it's also quite plausible that it will be easier to get something else to work. In any case, i feel like the problem is a close enough match for what we care about that doing "whatever it takes" will probably generally be pretty interesting.

Do you have suggestions for domains where you do expect one-turn debate to work well, now that you've got these results?

If there are high-quality arguments for multiple answers, doesn't that "just" mean that the multiple-choice question is itself low-quality?