Estimating efficiency improvements in LLM pre-training 2024-01-19T19:32:45.124Z

Comment by Daan on Estimating efficiency improvements in LLM pre-training · 2024-01-20T10:07:03.503Z · LW · GW

Thanks for the comment! I agree that would be a good way to more systematically measure algorithmic efficiency improvements. You won't be able to infer the effects of differences in data quality though - or are you suggesting you think those are very limited anyway?