Testing for parallel reasoning in LLMs 2024-05-19T15:28:03.771Z

Comment by meemi (meeri) on Finishing The SB-1047 Documentary In 6 Weeks · 2024-10-31T07:18:30.422Z · LW · GW

I think this is a great project. I believe your documentary would have high impact via informing and inspiring AI policy discussions. You've already interviewed an impressive amount of relevant people. I admire your initiative to take on this project quickly, even before getting funding for it.

Comment by meemi (meeri) on A quick experiment on LMs’ inductive biases in performing search · 2024-04-15T16:03:56.783Z · LW · GW

Great post! I'm glad you did this experiment.

I've worked on experiments where I test gpt-3.5-turbo-0125 performance in computing iterates of a given permutation function in one forward pass. Previously my prompts had some of the instructions for the task after specifying the function. After reading your post, I altered my prompts so that all the instructions were given before the problem instance. As with your experiments, this noticeably improved performance, replicating your result that performance is better if instructions are given before the instance of the problem.