WORLDTECHAnthropic urges California to let courts block risky AI models
Anthropic, an artificial intelligence company based in San Francisco, has urged California Governor Gavin Newsom to give the state power to ask a court to halt AI models that present a serious danger. The position appears in a letter first reported by Politico, and it arrives as the state considers how to tighten its AI safety rules.
What the letter proposes
In the letter, dated 15 October, Anthropic argues that a model found to carry serious risk should be stoppable through the judicial system. The company's wording holds that once a review identifies such a risk, government ought to be empowered to petition a court for a block on the model.
Under the company's proposal, independent evaluators would pass risk findings to California's attorney general. Anthropic also calls for internal safety auditors, risk reports every six months, and outside safety auditors embedded within leading AI firms.
Link to the September executive order
The letter answers an executive order Newsom signed in September. That order created a working group charged with recommending, within two months, ways to strengthen California's AI safety laws.
A question running through the wider debate is whether developers must be able to switch their models off.
What is confirmed and what is not
An Anthropic spokesperson confirmed that the letter is genuine. Newsom's office had not responded immediately to requests for comment, so the state's stance on the proposals is not yet known.
Open questions
- Will California's working group recommend a court-based power to block AI models?
- Must AI developers be able to switch their models off?
- How will the state's response to the September executive order shape the final rules?