Anthropic Proposes AI Safety Bill To Congress
Anthropic has drafted a bill that may block AI model release if it poses catastrophic risk, according to TechCrunch. The bill aims to regulate the AI industry and prevent potential dangers.
Original reporting · techcrunch.com
Original: “Anthropic pitches AI safety bill that may block a model if 50+ people could be killed”
Read “Anthropic pitches AI safety bill that may block a model if 50+ people could be killed” at TechCrunch →
How this was written. Hustle Bucks wrote this article with a language model from TechCrunch’s reporting (model: @cf/meta/llama-3.3-70b-instruct-fp8-fast); it describes that reporting rather than adding to it. The original, linked above, is the full story.
Key points from TechCrunch’s report
- Anthropic has proposed a bill to regulate AI safety and prevent catastrophic risks.
- The bill outlines scenarios in which an AI model's release could be blocked, including cases involving over 50 people at risk of death or over a billion dollars in property damage.
- The proposal suggests using independent evaluators to assess a model's potential for disaster, but it is unclear how this system would prevent payments from influencing the evaluation.
- Anthropic's proposal goes further than other industry suggestions for voluntary regulation and may influence future regulatory discussions.
- The AI industry has been criticized for its lack of transparency and resistance to mandatory government oversight.
Concerns about artificial intelligence safety have been growing, and Anthropic has put forth its vision for government regulation. The AI lab has been promoting a draft bill on AI safety that it hopes Congress will consider. This legislation is a significant development in the ongoing debate about AI safety.
The proposed bill outlines scenarios in which an AI model's release could be blocked, including cases where over 50 people are at risk of death or over a billion dollars in property damage could occur. These scenarios must involve a foreseeable and imminent risk, such as the creation of a biological weapon or a cyberattack. The bill's metrics are similar to those in a recently passed California law.
To assess a model's potential for disaster, the bill suggests using independent evaluators registered with the US Commerce Department. These evaluators could be paid by AI labs, as long as their payment is not contingent on the evaluation results. However, it is unclear how this system would prevent payments from influencing the evaluation.
If an evaluator determines that an AI model poses an imminent catastrophic risk, they would have 72 hours to notify the Commerce Department. The department would then need to request a court order to halt the model's release. The legislation would explicitly prohibit federal agencies from unilaterally halting a model's distribution.
Anthropic's proposal is notable, as it goes further than other industry suggestions for voluntary regulation. While some companies, like Nvidia, argue that the AI industry does not need new laws, Anthropic's bill represents a more concrete attempt to address AI safety concerns. The company has been advocating for its Advanced AI Framework since June.
The AI industry has been criticized for its lack of transparency and resistance to mandatory government oversight. Despite warnings about the potential dangers of AI, companies have largely preferred voluntary regulation and embedded evaluators to monitor models for potential risks. Anthropic's proposal may be seen as a step towards more robust regulation.
The draft bill's focus on catastrophic risks is somewhat removed from the current concerns in the tech industry, which have centered on agentic hacking and AI programs disobeying their human operators. However, Anthropic's proposal aims to provide a solution to the potential risks posed by AI and may influence future regulatory discussions. This article is based on reporting by TechCrunch.
Questions and answers
What is the main goal of Anthropic's proposed AI safety bill?
The main goal is to regulate the AI industry and prevent potential dangers, including catastrophic risks that could result in human death or significant property damage.
How would the bill assess a model's potential for disaster?
The bill suggests using independent evaluators registered with the US Commerce Department to assess a model's potential for disaster.
What is the significance of Anthropic's proposal in the context of AI regulation?
Anthropic's proposal is notable, as it goes further than other industry suggestions for voluntary regulation and may influence future regulatory discussions.
