
So what’s the deal with Anthropic’s extraordinary S-1? More than just financial risks that affect investors, it also describes potentially catastrophic harms to the public. Some suggest we should read the S-1 is an expression of Anthropic’s laudable safety-first philosophy. Or, less charitably, that its leaders are doomer weirdos, or maybe they’re just trying to overcompensate for racing to dangerous levels of AI capability.
But setting aside Anthropic’s motivations, what does the S-1 actually mean?
Anthropic’s S-1 extensively details the risks of its AI models, ranging from their potential use to develop dangerous weapons to the possibility that increasingly autonomous systems could take harmful actions with limited human oversight.
The S-1 also acknowledges that its safeguards against misuse are incomplete and can fail. These disclosures are intended for investors. However, they have implications for the public as well. The S-1 provides a detailed, written, and formal record of serious risks that one of the leading AI developers has itself identified.
The filing raises a question that goes beyond the ordinary question of whether investors can price the risks of shares they are buying: What legal obligations does the company have to address these risks?
Anthropic’s status as a public benefit corporation already gives its board a responsibility to consider public benefit interests, and it gives investors who pass a certain ownership threshold a vehicle to file derivative suits if the board doesn’t meet those obligations. The S-1 could also be used as evidence in a future lawsuit—by victims of an incident, and under certain circumstances the shareholders afterwards—if the company fails to address risks it has disclosed.
However, corporate governance and litigation after harm occurs have limits. On their own, a company’s disclosures to investors do not establish what safety measures the company must take. That’s why policymakers should still establish clear, enforceable obligations that can hold AI developers accountable and require that they assess and mitigate serious risks before harm occurs.
While Anthropic’s S-1 can be a starting point for that discussion, the public needs more than a company’s acknowledgment that its technology may be dangerous.