James Chua's Shortform

website to sum up resources / tweet thread/ discussion for our introspection paper

https://modelintrospection.com

Some people (my mentor ethan perez ) said my weekly MATS research update slides were nice. Some rough tips i have:

• mentors often have alot of projects they are working on. at the start of your slides,  recap the takeaways from last week, and any jargon you might have.
• Keep graphs simple. As a rule of thumb, it gets quite confusing when you have >= 4 categories / colors to look at. Are all these categories important? Maybe just show the most important two. Keep the other categories as a backup slide in case ethan wants the breakdown. One graph, one story to takeaway
• Highlight what to look at in the chart. E.g if you have a line chart on model loss, draw a red arrow that say "Model loss goes down - thats what we want!".
• Show the prompt of whatever you are calling the model with
• If you have someone to show to (e.g.  random people over lunch), show your slides. These people are going to have much less context on what you are working on, so if they can actually understand your slides, its a great signal that ethan is going to understand it. showing it to other ethan collaborators also helps - ask them to model what ethan would say.
• when i first started working with ethan and improving my slides, it took me around 2-3 days to do it. I suggest starting early. This seems a long time, but it includes asking my collaborators to critique my slides, and from their feedback i improved my plots + run more experiments to address the critique. i think it was a worthwhile investment! (after awhile i got better at this so i take less time to iterate on this process)