Join our newsletters for the latest updates and exclusive content on leading AI developments.
As the chief information security officer at Anthropic, Jason Clinton has a multitude of responsibilities. Reporting directly to CEO Dario Amodei, he manages a small team handling everything from data security to physical security at the Google and Amazon-supported startup. Anthropic, recognized for its large language models Claude and Claude 2, has raised over $7 billion and employs about 300 people.
Clinton’s primary focus is safeguarding Claude’s model weights, a massive, terabyte-sized file, to prevent it from falling into the wrong hands. In machine learning, especially deep neural networks, these weights are vital as they enable the network to learn and make predictions. Their final values after training determine the model’s performance.
According to a new report from the Rand Corporation, model weights are incredibly important because they embody the results of extensive and costly training processes. If accessed by malicious actors, these weights could allow the unauthorized use of the full model at a fraction of the training cost.
Clinton emphasized the importance of this task, revealing that he spends almost half his time focused on protecting this file, which receives the highest priority and security resources in the organization. Anthropic’s concern is not just about intellectual property but the potential misuse of the technology by bad actors, which could have severe consequences.
The White House has highlighted this concern in its Executive Order on AI, requiring companies to document ownership and protection measures of model weights. OpenAI, for example, has committed to not distributing its model weights outside its organization and select partners to maintain control over sensitive information.
A report from Rand identified around 40 attack vectors that could be used to steal model weights, from physical system access to compromising credentials. This reinforces the non-theoretical nature of the threats, showing that these methods are actively executed.
The debate over open foundation models continues, with some experts arguing that open-source models offer benefits like combating market concentration and fostering innovation. Examples like Meta’s Llama 2 show how open models can enable downstream modifications and scrutiny. However, the risks of misuse, particularly for creating harmful applications like deepfake scams, remain significant.
Proponents of open models argue that transparency increases security, allowing a broader community to identify and fix vulnerabilities. However, some believe the risks outweigh the benefits, particularly for powerful models with significant national security implications.
Clinton mentions that protecting Anthropic’s models is a continuous task, complicated by a shortage of security experts in the field. The fast-evolving nature of AI technology means that security strategies must constantly adapt to new developments.
He predicts that the future will require more frequent security updates to counteract potential vulnerabilities. This shift necessitates a mindset change in IT security, from occasional updates to continuous patching.
Clinton stresses the importance of balancing security with enabling research progress. Ensuring that researchers can move quickly without compromising security is critical for advancing AI while safeguarding its powerful capabilities.
Stay informed with our daily updates on AI news. Subscribe now!