post by [deleted] · · ? · GW · 0 comments

This is a link post for

0 comments

Comments sorted by top scores.

comment by saraprice · 2025-01-10T23:08:42.632Z · LW(p) · GW(p)

Parallelising experiments

  • It works well to run many different experiments at once with API-based experiments in a separate tmux pane/session.

I would link to some of the specific tools you list below for this and note that you've previously set up some generalizable/ easy-to-use infra for this but that plenty of tools already exist 

comment by saraprice · 2025-01-10T23:04:08.582Z · LW(p) · GW(p)

One notebook per paper figure

  • When you get to paper writing, it is very helpful to have one notebook per paper figure. This allows other collaborators to easily reproduce the figure and make changes as required. It also reduces the risk of horrible git merge conflicts if everyone is iterating in parallel.

After our BoN paper I actually kind of disagree with this take. I think I definitely leaned more towards liking to use the notebooks that had all the plots (cleanly). At the very least, I would put this much lower on the list

comment by saraprice · 2025-01-10T23:03:02.787Z · LW(p) · GW(p)

e.g. during the rebuttal period
specific that this is a paper rebuttal period