[Linkpost] Mark Zuckerberg confronted about Meta's Llama 2 AI's ability to give users detailed guidance on making anthrax - Business Insider

post by mic (michael-chen) · 2023-09-26T12:05:57.396Z · LW · GW · 11 comments

This is a link post for https://www.businessinsider.com/mark-zuckerberg-senate-ai-forum-llama-2-gives-anthrax-instructions-2023-9

Several tech leaders descended upon Capitol Hill last week to discuss the rapid expansion of generative AI. It was a mostly staid meeting until the potential harms from Meta's new Llama 2 model came up.

During the discussion, attended by most of the Senate's 100 members, Tristan Harris, a co-founder of the Center for Humane Technology, said he recently had engineers take Meta's powerful large language model Llama 2 for a "test drive." After some prompting, Harris said that a chat with Llama 2 came back with a detailed walkthrough of how to create anthrax as a biological weapon, according to one person familiar with the forum and two senators present. That prompted a testy exchange between Harris and Mark Zuckerberg, co-founder and CEO of Meta, formerly known as Facebook. Most specifics of the exchange between Harris and Zuckerberg have not been previously reported, although Harris receiving directions from Llama 2 about an unidentified biological weapon was noted by The Washington Post.

Among the two dozen tech leaders at the forum were Elon Musk, owner of Twitter and CEO of Tesla and SpaceX; Sam Altman, CEO of OpenAI; Satya Nadella, CEO of Microsoft; Jensen Huang, CEO of Nvidia; and Sundar Pichai, CEO of Google.

The gathering was led by Senate Majority Leader Chuck Schumer, Democratic Sen. Martin Heinrich, and Republican Sens. Mike Rounds, and Todd Young, who all make up a new "artificial intelligence working group." The group formed earlier this year, a few months after OpenAI's ChatGPT bot became known the world over.

During the session, Zuckerberg attempted to downplay Harris' statement that Llama 2 can tell users how to make anthrax, saying anyone who was looking for such a guide could find out how to make anthrax on YouTube, according to both of the senators present. Harris rejected the argument, saying such guides do not come up on YouTube, and even if they did, the level of detail and guidance provided by Llama 2 was unique to such a powerful generative AI model. It's also largely an open-source model, meaning it's freely available to use and adapt.

"It was one of the only moments in the whole thing that was like, 'Oh,'" one of the senators present said, describing the exchange as having caught people's attention. "Twenty-four out of the 26 panelists there basically said exactly the same thing over and over: 'We need to protect AI innovation but with safeguards in place.'"

A Meta spokesperson declined to comment. Harris did not respond to requests for comment.

Beyond the brief snit between Harris and Zuckerberg, there was little in-depth discussion of the issues surrounding AI, according to all three of the people familiar with the meeting. Even the ability of Llama 2 to guide a prospective user on creating anthrax was not cause for any extended probing, the people said.

"It was, 'Ok, next speaker,' it moved right along," one of the senators present said.

Llama 2's power is well-known inside Meta. Its ability to turn up detailed instructions for creating a biological weapon like anthrax is to be expected, two people familiar with the company said.

"Really, this is going to be the case for every LLM of a certain size, unless you kneecap it for certain things," one of the people familiar with Meta said. "There will be edge cases. But the ones that are products, like ChatGPT, as opposed to open source releases, they just nerf it for this and that."

Still, AI tools trained on trillions of pieces of information scraped from the whole of the internet are difficult to control. Earlier this year, a user of a Discord bot created with ChatGPT was able to get the chemical recipe for napalm, a highly flammable liquid used as a military weapon. ChatGPT and Google's Bard are also known for serving up information to users that is incorrect, composed of misinformation, or simply made up, dubbed "hallucinations."

Are you a Meta employee or someone else with insight to share? Contact Kali Hays at khays@insider.com, on secure messaging app Signal at [phone number], or through Twitter DM at @hayskali. Reach out using a non-work device.

Get in touch with Bryan Metzger by email at bmetzger@insider.com or find him on Twitter at @metzgov.

11 comments

Comments sorted by top scores.