Cyberattacks threaten to make open-source AI a little less open, Hugging Face exec warns

Loading the player…

The very qualities that make open AI platforms valuable, like open doors and rich data, might also make them vulnerable to more hacks, Irene Solaiman, chief policy officer and SVP of strategy at Hugging Face said on a recent episode of the Equity podcast.

The October attack on OpenCode, which targeted users’ third-party API keys and caused financial damage, is the latest example. It follows an incident this summer in which an OpenAI agent swarm breached Hugging Face in an apparent attempt to cover its tracks after accessing answers to cybersecurity evaluations.

The risks aren’t limited to platforms that invite open community contributions. On Monday, AI infrastructure startup Fireworks, which helps developers run and train open AI models, disclosed that attackers had accessed some customers’ model weights, datasets, and billing information. The company temporarily disabled certain self-service dataset functions as it investigates. While the attack isn’t necessarily due to Fireworks’ openness, it highlights how much valuable information is concentrated within AI infrastructure providers. 

“The things I worry about making an open community targeted are also what makes it special,” Solaiman said on Equity. “We are necessarily permissive. We want the average hobbyist in their basement who doesn’t have an institutional affiliation to be able to contribute.”

“You want people to be able to just submit pull requests and allow anybody without a valid affiliation to contribute…but then you have some losers who ruin it for the rest,” she added in a separate conversation.

That permissiveness isn’t the only thing drawing attackers. The wealth of information open platforms host also make them targets. Hugging Face became a temptation for OpenAI’s agent swarm because of the evaluation datasets it holds, whereas OpenCode’s target in this case came from its “juicy third-party API keys,” per Solaiman. 

View Bio