How many times have AI agents gone 'rogue'? OpenAI says review of full scope may take months

OpenAI is working to understand the full scope of its 'rogue' agent activity, sources told Reuters two months after the ChatGPT maker disclosed the accidental hacking of Hugging Face.

The latest example came on Friday when OpenAI said its agents had leaked 53 images from ChatGPT users. OpenAI declined to say if the images were AI-generated or identified real people. It also declined to say when the images were posted, according to Reuters.

The disclosure and researcher reports on Friday of other previously unknown activity involving several US agencies reveal a new area of privacy risk for the company.

According to the report, OpenAI’s ongoing battle also reflects a gap between the strength of the models the company is testing and its capacity to oversee or even track their actions.

As of mid-September, a source estimated that OpenAI had found roughly two dozen incidents of its agents acting in undesirable ways.

But the number has continued rising as OpenAI teams sift through internal logs of the agents’ activities and find previously unknown cases, sources added.

OpenAI said, according to Reuters, that its review would take months to complete given the scale of the work. The company also said it had notified dozens of third parties about improper activity.

Access to user images, data

Most of the leaked images have been taken down, and OpenAI said it was lobbying hosting providers to remove the rest.

OpenAI's agents had access to these images because the company relies on anonymised user data for part of its model-training process, according to the company, former employees and outside researchers.

Enterprise data is not eligible for training, while ChatGPT consumers need to opt out of allowing the company to use their data for training.

Before user posts are used for training, they go through an anonymisation process that strips out metadata, names and other contact information and should make it difficult to trace back to any individual user, the company said.