A few days ago, a US AI policy expert mentioned to me that ignoring the influence of effective altruism (EA) means missing a crucial part of the AI story. Reflecting on this, I realized I partially overlooked this aspect in my recent work.
Last week, I thought I’d hit the mark with an article about why top AI labs and think tanks are intensely focused on securing large language model (LLM) weights. It seemed relevant and clear, considering the White House’s recent AI Executive Order. This order mandates that companies document the ownership and security measures for LLM model weights to the federal government.
In my piece, I interviewed Jason Clinton, the Chief Information Security Officer at Anthropic. He emphasized that securing the model weights for Claude, Anthropic’s LLM, is his top priority due to threats from criminals, terrorists, and nations. If attackers access the entire neural network file, it’s a major breach. Other leading AI firms share this concern; OpenAI’s new “Preparedness Framework” also addresses controlling access to crucial information like model weights.
I also talked to Sella Nevo and Dan Lahav, co-authors of a RAND Corporation report on securing AI model weights. Nevo, who leads RAND’s Meselson Center focusing on bio-threats and new technologies, highlighted that AI models could gain national security importance within two years, posing risks like misuse for developing biological weapons.
Despite this, I missed a significant angle: the expanding web of connections from the EA community within AI security and policy. Though I’ve spent a lot of time trying to understand effective altruism—a movement that uses evidence and reasoning to maximize benefits and has gained a cult-like reputation— I didn’t consider its relevance for this piece.
The notable connections include:
1. EA’s recent headlines tied to the firing of OpenAI CEO Sam Altman, with EA-connected individuals involved.
2. Anthropic has links to EA, including significant investment from FTX, associated with EA.
3. Politico reported that RAND researchers influenced the White House’s AI Executive Order and highlighted significant funding from Open Philanthropy, an EA organization.
These connections extend further. RAND CEO Jason Matheny and senior scientist Jeff Alstott have EA ties and have worked in high-level government roles. Matheny is also connected to Anthropic’s Long-Term Benefit Trust. Furthermore, Nevo and other authors of the RAND report are deeply involved in the EA community, and their work is funded by organizations like Open Philanthropy.
Nevo emphasized that EA has been central to the AI safety discussion for years. Individuals working in AI security have likely interacted with this community. He critiqued articles with conspiratorial tones suggesting wrongdoing by RAND and pointed out that RAND has consistently provided valuable research for decades. He clarified that neither he nor his center was directly involved with the Executive Order’s security rules, although their work may have had indirect influence.
Nevo’s Meselson Center primarily focuses on bio-surveillance, DNA screening, and the AI-biology intersection, operating with a small team but with plans to expand.
Does the EA presence in AI security matter? For those advocating for transparency in technology and policy, yes. EA’s influence is shaping long-term AI policy and development. According to the AI policy expert I spoke with, there are indeed ideological agendas at play in AI, contrary to what many may believe.