Poster Session on AI Safety

I agree that having a section on "what to do about it" is really useful for getting people interested. Otherwise you have a lot of unresolved tension.

Some quick comments based purely on the poster (which is probably the most important part of your funnel):

"Biological Anchors" is probably not a meaningful term for your audience.

We have a 50% chance of recreating that amount of relevant computation by 2060

This seems wrong in that we already have around brain training levels of computation [LW · GW] now or will soon - far before 2060. The remaining uncertainty is over software/algorithms, not hardware. We already have the hardware or are about to.

Once AI is capable of ML programming, it could improve its algorithms, making itself better at ML programming

This is overly specific - why only ML programming? What if the lowest hanging fruit is actually in cuda programming? Or just moving to different hardware? Or designing new hardware? Or better networking tech? Or one wierd trick to make a trillion dollars and quickly scale to more hardware? Etc etc. The idea that there are enormous gains in further optimization of ML architecture alone, and that this unending cornucopia of optimization low hanging fruit will still be bountiful and limitless by the time we actually get AGI - this suggests a very naive view of ML & neuroscience.

Just replace "ML programming" with "science and engineering R&D" or similar.

Training AI requires us to select an objective function to be maximized, yet coming up with an unproblematic objective function is really hard.

Many smart people will bounce hard off this, because they have many many examples where coming up with an unproblematic objective function isn't really hard at all. It's trivial to write the correct objective function for Chess or Go. It was trivial to design the correct utility function for atari, for minecraft even (which doesn't have a score!), it was also trivial for optimizing datacenter power usage, for generating high quality images from text, for every other modern example of DL, etc etc.

I would change this to something like:

"Training AI requires us to select an objective function to be maximized, yet coming up with an unproblematic objective function for AGI - agents with general intelligence beyond that of humans - seems really hard".

I don't think you need to view namedropping as an appeal to authority. The natural way to do it in a scholarly document, including a poster, would be to cite a source. That's giving the reader valuable information - a way to check out the authority behind it.

Of course, if the reader is familiar with the author cited and knows that their work is invariably strong, they might choose to take it on authority as a shortcut, but they have the info at hand to check into it if they wish.