0 comments
Comments sorted by top scores.
comment by Vladimir_Nesov · 2023-02-07T04:43:49.579Z · LW(p) · GW(p)
Explaining the reasoning after the score helps in two ways:
First, it helps to select between the different possible scores if some choices are close
Would placing the reasoning before the score help, by having the reasoning in the context window when deciding a score? If the reasoning itself makes essential use of step-by-step, its outcome probably can't be anticipated immediately. (But this might be less reliable, if reasoning derails the script and never gets to issuing a score.)
Replies from: quentin-feuillade-montixi, quentin-feuillade-montixi↑ comment by Quentin FEUILLADE--MONTIXI (quentin-feuillade-montixi) · 2023-02-07T16:43:54.220Z · LW(p) · GW(p)
Also i didn't use step-by-step reasoning but a detailed reasoning because there where too much weird behaviors. I think that if it where a step-by-step reasoning, it would have indeed help to place the reasoning before.
↑ comment by Quentin FEUILLADE--MONTIXI (quentin-feuillade-montixi) · 2023-02-07T16:35:16.914Z · LW(p) · GW(p)
"But this is really not stable and the ordering of the explanation didn't seem to help the score that much. I will work on a post investigating this in more detail."
This was making the metric very unstable and this doesn't seems to help the score. I am working on a metric to evaluate that but i think this doesn't help. I have the intuition it might be some kind of search or complex heuristic inside LLMs (kind of LLM computing the explanation in advance and weighting the score with this precomputed explanation). I might be totally wrong but i am trying to think of a way to test that.
Replies from: Vladimir_Nesov↑ comment by Vladimir_Nesov · 2023-02-07T16:54:03.115Z · LW(p) · GW(p)
kind of LLM computing the explanation in advance and weighting the score with this precomputed score
I think this would work fine for things that don't essentially require step-by-step reasoning ("detailed reasoning" might still be accessing this capability), which is why the order from the post is not as bad as writing down the bottom line before the argument [LW · GW]. When you ask for the argument, it's being anticipated, as much as that's possible to do without seeing its prefix explicitly. But this should be a problem when the outcome of the argument can't be anticipated.
Replies from: quentin-feuillade-montixi↑ comment by Quentin FEUILLADE--MONTIXI (quentin-feuillade-montixi) · 2023-02-07T17:09:18.635Z · LW(p) · GW(p)
Good point, i should add this in limitation and futur directions. Do you have an example in mind?