Anthropic partners with US government to prevent AI nuclear leaks
What's the story
In a first, leading artificial intelligence (AI) firm Anthropic has partnered with US Department of Energy's (DOE) nuclear experts.
The collaboration is focused on making sure that Anthropic's AI models do not accidentally leak sensitive information about nuclear weapons.
The initiative, which started in April and was revealed by Anthropic to Axios, is a major step in AI security.
Security assessment
DOE experts test Anthropic's AI model for potential misuse
The DOE's National Nuclear Security Administration (NNSA) is conducting a "red-teaming" exercise on Anthropic's AI model, Claude 3 Sonnet.
The process involves experts trying to exploit or break a system to find potential weaknesses.
The main focus of this exercise is to see if Claude's responses could be manipulated for creating nuclear weapons or accessing other harmful nuclear applications.
Ongoing evaluation
Anthropic's AI model undergoes rigorous security tests
The security assessment project will continue until February, during which the NNSA will also assess the updated Claude 3.5 Sonnet introduced in June.
To prepare Claude for these intensive government-focused security tests, Anthropic has leveraged its partnership with Amazon Web Services (AWS).
However, given the sensitive nature of this work, Anthropic has not yet revealed any findings from this pilot program.
Information sharing
Plans to share security assessment results
Anthropic intends to share the findings of its security assessment with scientific labs and other organizations, encouraging independent testing to avoid the potential misuse of AI models.
Marina Favaro, Anthropic's national security policy lead, stressed the need for partnerships between tech companies and federal agencies in evaluating national security risks.
Security focus
AI at the heart of national security discussions
Wendin Smith from the NNSA, emphasized that AI has become a focal point in important national security conversations.
She said that their agency is well-prepared to assess possible risks posed by AI, especially those pertaining to nuclear and radiological safety.
This partnership comes in line with President Joe Biden's recent national security memo calling for AI safety assessments in classified settings.
Industry partnerships
Developers seek government contracts amid safety concerns
As AI developers battle for government contracts, Anthropic recently teamed up with Palantir and AWS to provide Claude to US intelligence agencies.
Other giants like OpenAI have also joined forces with organizations like NASA and the Treasury Department.
However, it remains unclear how these partnerships will hold up in the face of looming political changes in Washington.