Reward hacking behavior can generalize across tasks 2024-05-28T16:33:50.674Z

Comment by Isaac Dunn (isaac-dunn) on Tips and Code for Empirical Research Workflows · 2025-01-21T20:43:19.013Z · LW · GW

I recently switched from using threads to using asyncio, even though I had never used asyncio before.

It was a combination of:

• Me using cheaper "batch" LLM API calls, which can take hours to return a result
• Therefore wanting to run many thousands of tasks in parallel from within one program (to make up for the slow sequential speed of each task)
• But at some point, the thread pool raised a generic "can't start a new thread" exception, without giving too much more information. It must have hit a limit somewhere (memory? hardcoded thread limit?), although I couldn't work out where.

Maybe the general point is that threads have more overhead, and if you're doing many thousands of things in parallel, asyncio can handle it more reliably.