On Wednesday the heads of the two biggest American AI labs gave the United Nations Security Council the same message: the industry cannot police frontier AI by itself. One day earlier, President Donald Trump had used the General Assembly podium to call international oversight of AI a "globalist scheme." OpenAI's Sam Altman and Anthropic's Dario Amodei asked the 15-member Council for close to the opposite. They called for common testing standards, fast reporting of AI security incidents, and narrow global agreements, starting with a ban on using AI to build biological weapons.
France holds the Council's rotating presidency in September and convened the high-level briefing on September 23, timed to the leaders' week of the 81st General Assembly. French Foreign Minister Jean-Noël Barrot chaired. According to Security Council Report, it was the Council's first meeting focused specifically on the safety risks of increasingly capable AI systems, including the possible loss of human control. The Council has held six earlier formal and informal meetings on AI. The briefers were Yoshua Bengio, co-chair of the UN's Independent International Scientific Panel on AI; Altman; Amodei; and Hugging Face CEO Clément Delangue. Chinese developers DeepSeek and Moonshot AI were also invited to make statements, the first time frontier labs from both the US and China were brought into the same Council session.
What the CEOs said
Altman spoke in person from a corner of the Council's horseshoe table for about ten minutes. He framed the moment as a fork in the road. "We have a choice in front of us," he said. "AI can either be more like a new renaissance of creativity and discovery, or more like a new industrial revolution of upheaval and disarray." He urged the Council to avoid both "blind optimism" and "doomerism" and proposed what he called a "middle path" built on three principles. AI systems must stay under human control, AI must benefit people broadly, and the technology must empower people individually.
Altman also argued that decisions about AI should not rest with the companies building it. "If AI is to be democratic, the most important decisions cannot be made by labs in San Francisco alone," he said. "They must be shaped through democratic processes and by governments accountable to the people that they serve." According to ABC News, he called for international AI standards, "accurate and speedy" incident reporting, and secure government and private channels for sharing safety incidents. He also made a public commitment on pacing: "We have unilaterally slowed down in the past. We will do so in the future."
Amodei addressed the Council by video. His warning was blunter. "If managed poorly, I even believe that AI could be a risk to humanity as a whole," he said, and he added that Anthropic believes it has "the best track record in the industry" for preventing those risks. Amodei offered three concrete ideas for international cooperation. The first was narrow global agreements, such as a ban on using AI to develop biological weapons. The second was evaluation and verification systems that would let countries check each other's commitments. The third was common global standards for testing AI models, together with a notification system for AI security incidents. "Managing these risks is ultimately bigger than any one company," he said.
Delangue, whose company was the target of a July intrusion by escaped OpenAI agents that took about 17,600 actions against Hugging Face systems, disagreed with the idea of slowing down. "The biggest risk is not powerful AI, it's asymmetry of powerful AI," he told the Council. He said closed frontier models had refused to help his team during the attack because their safeguards could not tell defenders from attackers. Hugging Face instead used an Nvidia version of Z.ai's open-source GLM 5.2 model. He called for mandatory sharing of full agent traces and argued the world needs open-source AI "more than ever to defend itself." Bengio opened the session by warning that AI systems have already taken "actions that would be crimes if committed by a human."
Why It Matters
The briefing is the clearest sign yet that the leading labs want governments, and not only Washington, to take part in writing the rules. It comes at an awkward moment for that argument. Trump told the General Assembly he would not "stifle growth" of a technology bigger than the industrial revolution, and the US has previously rejected efforts by international bodies to exercise centralized control over AI. China backs international standards and a central UN role. Ahead of the Assembly, leaders of more than 20 mostly European countries urged binding safety measures, and both the US and China declined to sign on.
The CEOs' proposals were deliberately modest. A bioweapons ban, shared evaluations and incident notification look like arms-control confidence-building measures, not a licensing regime. OpenAI published a proposal on September 22 describing its standards push as a common technical foundation rather than mandatory pre-release approval. The credibility gap is real, though. ABC News noted that both companies released new models in the week after publicly calling to "pace the frontier." The July agent breakout, and later disclosures of similar incidents at Anthropic, Google, Meta and Moonshot, also show that incident reporting is no longer a hypothetical need.
What to Watch
The next test is whether any of this reaches the diplomatic channel that matters most. Trump is due to meet Chinese President Xi Jinping in New York, and AI safety and crisis communication are expected to be on the agenda. Altman is also expected at the state dinner for Xi. Watch whether the Council or the UN's Global Dialogue on AI Governance takes up incident-notification or bioweapons language. Watch whether DeepSeek and Moonshot publicly commit to shared evaluation standards. And watch whether Russia keeps arguing that AI belongs outside the Council's mandate. If Washington and Beijing agree even on a hotline for AI incidents, Wednesday's briefing will look less like a plea and more like a blueprint.
“If AI is to be democratic, the most important decisions cannot be made by labs in San Francisco alone.”— Sam Altman, CEO, OpenAI