Why AI Isn’t Likely to Wipe Out Humanity With Bioweapons

Concerns about an AI-induced apocalypse are reaching a fever pitch, from Silicon Valley to Washington, DC.

One of the top theories about how a rogue AI will wipe us out? Biological weapons.

Earlier this summer, AI CEOs called for new laws to control the manufacturing of synthetic DNA. Then last month, researchers at Stanford University and the Arc Institute showed that AI can design new viral genomes. The idea gained steam last week after Anthropic released a report indicating that there had been efforts to use Claude in ways that “could support biological weapons development. “The report authors cite biological misuse as “one of the most serious risks of frontier AI models.” Shortly thereafter, chief executive officer Dario Amodei urged the government to help AI labs “pace the frontier.”

Despite the alarm around an AI-created killer plague, many scientists say the risk is fairly low on the list of the world’s concerns.

David Bellamy, a research scientist at the Institute of Foundation Models in Sunnyvale, California, does not see AI as a fundamentally new threat when it comes to the creation of bioweapons. The scientific community has grappled for decades with the question of whether to publish potentially dangerous biological research—known as the dual-use dilemma.

Before large language models, the internet itself, open-access science journals, and translation services like Google Translate made it easier for people to access biological information, including lab protocols, Bellamy says.

“AI is essentially a tool that can help both good actors, like scientists, and also threat actors to peruse information more quickly and define and source those protocols more quickly,” he adds. “But those capabilities are not really the bottleneck in the production of bioweapons.”

Before a bad actor could get to that point, though, most everyone agrees that models should have appropriate safeguards in place so that they don’t provide actionable, dangerous information to people.