Anthropic warns of ‘catastrophic threat to humanity’ as it eyes $2 trillion valuation in biggest-ever IPO

Artificial intelligence (AI) company Anthropic filed its S-1 paperwork on Monday (local time) as it gears up for a Wall Street debut. The firm, which is eyeing one of history’s biggest stock market debuts, has once again warned of the potential dangers its AI systems could pose to humanity.

The prospectus, filed on Monday, devoted over 90 pages to risk factors, including potential threats to Anthropic’s ability to safely develop the technology. However, it also suggested that the company is best placed to address these risks, describing its approach as a “different approach from that of a typical company”.

What Anthropic's prospectus reveals about AI risks

According to the filing, the AI firm said that concerns have been raised by policymakers, regulators, and researchers, along with people within the organisation, that AI systems pose a significant risk to humanity.

It said advanced AI systems "could, if not developed, deployed, and operated with appropriate safeguards, oversight, and security measures, pose catastrophic or existential risks to humanity, particularly if such systems were misused, replicated, weaponised, or allowed to operate with a high degree of autonomy".

Citing recent cybersecurity incidents as an example, Anthropic said, "AI can be misused by those who wish to do harm, for example to conduct cyberattacks or to design biological weapons." It further said the company has strict guardrails and trusted access programmes to block misuse of its technology, including the Claude models.

It also warned of power, water, and climate-related risks and wrote, "The physical effects of climate change may increase our operating costs over time, as higher average temperatures can increase the power consumption and water usage required to cool equipment at the facilities on which we depend."

Anthropic IPO: A look at key figures behind the firm's ambitions

(with agency inputs)

Swati Gandhi