Human study on AI spear phishing campaigns 2025-01-03T15:11:14.765Z

Comment by Fred Heiding (fred010) on Human study on AI spear phishing campaigns · 2025-01-05T11:13:14.585Z · LW · GW

Great discussion. I’d add that it’s context-dependent and somewhat ambiguous. It’s noteworthy that our work shows that all tested AI models conflict with at least three of the eight prohibited AI practices outlined in the EU’s AI Act.

It’s also worth noting that the only real difference between sophisticated phishing and marketing can be the intention, making mitigation difficult. Actions from AI companies to prevent phishing might restrict legitimate use cases too much to be interesting.

Comment by Fred Heiding (fred010) on Human study on AI spear phishing campaigns · 2025-01-04T11:32:04.011Z · LW · GW

Thanks for your feedback! It’s just a matter of time before scammers maximize their use of AI. Hopefully, the defense community can use this time to optimize our head start. Stay tuned for our coming work!