These standards would ideally be backstopped by “regulation that targets all US frontier AI companies” that don’t voluntarily comply, Amodei said, and presumably similar regulations in other democracies. That kind of regulation might be hard to achieve under the current US administration, though, as President Trump wrote on social media Monday morning that “the only control or ‘guardrails’ that AI needs is a STRONG AND SMART (High IQ!) PRESIDENT, and the USA has that, in spades!”
Speaker of the House Mike Johnson, for his part, acknowledged in televised interviews this weekend that “we have to put up some guardrails, some safety measures in place to ensure that AI doesn’t run away…” At the same time, he added that “we don’t need everybody to panic right now” and wanted to “resist Congress jumping in and imposing some sort of emergency moratorium.”
David Sacks, who serves as co-chair of the president’s Council of Advisors on Science & Technology, wrote on social media this weekend to urge the frontier model makers to self-regulate rather than wait for Washington. “The easiest way not to build superintelligence is for you to agree not to build it,” he wrote.
The China problem
OK, so let’s say you’ve somehow gotten every democratic country and all the massive AI companies they contain to agree to some sort of universal safety and pacing standards. You still have to worry about the open-weight models coming out of China, whose capabilities are only a few months behind those of the major corporate AI behemoths.
In his essay, Amodei proposes a number of different levels of international agreements on AI safety, with the highest being “a full pacing, or even ‘pause,’ in which participating governments agree to substantially limit the overall rate of AI development.” Amodei acknowledges that high-level agreement is “unlikely to actually happen any time soon,” especially because the stakes are so high. In fact, Amodei reasons that “AI could be so powerful that such a defection [from China] could lead to their geopolitical dominance,” which certainly would make the collective action problem a bit more acute.
