0 comments
Comments sorted by top scores.
comment by Tristan Wegner (tristan-wegner) · 2023-11-25T10:31:23.005Z · LW(p) · GW(p)
I think you made a off by 100 error in Unlabeled Evaluation with all win rates <1%
Replies from: bruce-lee↑ comment by Bruce W. Lee (bruce-lee) · 2023-11-25T15:06:07.863Z · LW(p) · GW(p)
Thanks for pointing that out. Sometimes, the rows will not add up to 100 because there were some responses where the model refused to answer.
Replies from: tristan-wegner↑ comment by Tristan Wegner (tristan-wegner) · 2023-12-13T06:05:01.795Z · LW(p) · GW(p)
No. By off by 100 I meant of by a factor of 100 to small, NOT that they don't sum up to 100.
Replies from: bruce-lee↑ comment by Bruce W. Lee (bruce-lee) · 2023-12-14T01:29:29.436Z · LW(p) · GW(p)
Yeah, I see it. It's fixed now. Thanks!
comment by red75prime · 2023-11-24T09:45:43.059Z · LW(p) · GW(p)
This means that LLMs can inadvertently learn to replicate these biases in their outputs.
Or the network learns to trust more the tokens that were already "thought about" during generation.
Replies from: bruce-lee↑ comment by Bruce W. Lee (bruce-lee) · 2023-11-25T15:06:59.692Z · LW(p) · GW(p)
How is this possible? We are only inferencing