Khanna to introduce AI safety bill with ban on 'recursive' technology until safeguards exist

Rep. Ro Khanna, D-Calif., leaves the U.S. Capitol after the last vote of the week on Thursday, September 3, 2026.

Tom Williams | Cq-roll Call, Inc. | Getty Images

Silicon Valley Democratic Rep. Ro Khanna will introduce a bill to regulate artificial intelligence, which includes strict liability standards and a ban on "recursive" self-improving AI until the government creates safeguards.

The bill comes amid a push for Washington to regulate AI after top executives warned the technology could become out of their control. Congress has largely done nothing to regulate the technology, which is undergirding the economy but could cause catastrophic damage if it goes wrong.

"There's actually a civilizational extinction risk," Khanna said Monday in an interview with CNBC. "There's a safety risk of loss of control, and then there's a misuse risk, and we need to take both seriously."

Khanna's bill, a summary of which was shared exclusively with CNBC, is dubbed the "Human Control Over AI Act" and would ban models that recursively self-improve or autonomously modify their own core objectives, containment or shutdown controls until federal guardrails exist and an agency approves the activities.

The bill would also create a new federal agency to focus on AI models from OpenAI, Anthropic, Google DeepMind and xAI as the companies find themselves in the spotlight amid rapidly escalating safety concerns.

The agency would be tasked with creating safety regulations, a licensing system to approve model training and deployment, frontier AI model audits and setting security standards for continuous testing and effective human control.

The bill would also create new legal penalties for AI gone awry.

AI companies would also be required to hold extensive liability insurance to release their models.

The bill also calls for the administration to pursue enforceable agreements and targeted export controls to deter China and other adversaries from developing dangerous AI systems.