Dual use of artificial-intelligence-powered drug discovery

I see your point as warning against approaches that are like "get the AI entangled with stuff about humans and hope that helps".

There are other approaches with a goal more like "make it possible for the humans to steer the thing and have scalable oversight over what's happening".

So my alternative take is: a solution to AI alignment should include the ability for the developers to notice if the utility function is borked by a minus sign!

And if you wouldn't notice something as wrong as a minus sign, you're probably in trouble about noticing other misalignment.

I had a long back-and-forth about that topic here [EA · GW]. Among other things, I disagree that "more or less orthogonal to CEV" is the default in the absence of alignment research, because people will presumably be trying to align their AIs, and I think there are will be obvious techniques which will work well enough to get out of the "random goal" regime, but not well enough for reliability.

There's a quick CFAR class that I taught sometimes, which was basically "if you understand a bug, you should be able to make it worse as well as better." [That is, suppose you develop an understanding of which drugs cause old cells to die; presumably you could use that understanding to develop a drug that causes them to stick around longer than they should, speeding up aging, or maybe to kill young cells, or so on.]

"how could it possibly be toxic in vivo, we had a scoring for toxicity in our combinational chemistry model!"

Usually when you're screening for tox effects in a candidate you're looking for off target effects (some metabolic process produces a toxic aniline compound which goes off into some other part of the body, usually the liver and breaks something), in this particular case, that isn't the whole picture. Galantamine (useful drug, originally said memantine which is taken with it but isn't in the same class) and VX (nerve agent) are both acetylcholinesterase inhibitors, a key difference is that VX is much better at it.

One way to achieve the aim in the paper would be to set the model to produce putative acetylcholinesterase inhibitors, rank them by estimated binding efficiency, then setting a breakpoint line between assessed 'safe' and assessed 'toxic'. Usually you'd be looking under the line (safe), in this case, they're looking above it (toxic).

My point was that in my opinion, being open to the possibility of the line having been placed in the wrong place (special efforts) is probably wise. This opens up an interesting question about research ethics--would doing experimental work to characterize edge cases in order to refine the model (location of the line) be legitimate or malign?