Multimodal Neurons in Artificial Neural Networks

post by Kaj_Sotala · 2021-03-05T09:01:53.996Z · LW · GW · 2 comments

This is a link post for https://distill.pub/2021/multimodal-neurons/

Contents

2 comments

A paper investigating how individual neurons in a CLIP model (an image/text neural net combining a ResNet vision model with a Transformer language model) respond to various abstract concepts. This shouldn't be very surprising after GPT-3 and DALL-E but still, identifying multimodal neurons feels scarily close to "neural net that understands abstract concepts" and thus AGI for my comfort.

Some individual neurons that they isolated (see the article for more):

They also document the fact that the multimodality allows for "typographic attacks", where labeling an item with a particular text causes the network to misclassify the item as an instance of the text.

2 comments

Comments sorted by top scores.

comment by Charlie Steiner · 2021-03-05T20:41:20.951Z · LW(p) · GW(p)

Their faces and poses images are solid gold. There should be Gravatars except random points in CLIP's face latent space.

comment by Richard_Kennaway · 2021-03-05T10:05:05.506Z · LW(p) · GW(p)

They also document the fact that the multimodality allows for "typographic attacks", where labeling an item with a particular text causes the network to misclassify the item as an instance of the text.

"These are not the droids you are looking for."