California Wants a Kill Switch for Frontier Models — Newsom's Order Jabs at Washington
The feds are asleep, so California hit the snooze button itself — and the thing it's bolting onto frontier AI is, literally, a "pull the plug" switch.
Three key facts
On September 18, California Governor Gavin Newsom signed an executive order accelerating the nation's first independent third-party oversight and audit regime for AI companies, and advancing the creation of an emergency shutoff mechanism — a "kill switch" — for frontier models. The order convenes a group of leading experts who must deliver, within two months, a guide for strengthening California's AI safety laws.
Two hard requirements under consideration: first, independent third parties writing the safety plans for frontier AI companies; second, mandating that companies develop an emergency shutoff for frontier models. This isn't out of nowhere — just a week earlier Newsom signed the law creating this independent oversight, and the executive order is putting the pedal down on that law.
The trigger is stated bluntly: recent alarming incidents including the Hugging Face attack, plus the fact that "the federal government has done nothing." The order names it outright: no federal law requires AI companies to report dangerous incidents when they happen, and the White House has rejected calls to strengthen oversight. Newsom's framing is to act "thoughtfully but with urgent velocity."
Wang Dou's Take
The moment "kill switch" gets carried out of sci-fi and into a state government's official document, it's a signal in itself: California has already assumed frontier models will go wrong. The debate is no longer whether to guard against it, but who pulls the plug when it happens — and whether pulling it does anything. Fitting a model with an emergency stop sounds like putting brakes on a rocket; the hard part was never installing it, it's who holds the switch in that moment and whether anyone obeys when it's thrown.
The genuinely sharp part is "independent third parties write the safety plans." Until now, the safety plan was written by the company, scored by the company, and published by the company — player and referee in one body. California is now moving the referee outside the building. That sets up a neat contrast with OpenAI's own misalignment disclosure framework from these same days — one scrambling to set its own standard, the other outsourcing the standard to third parties. Which side is more afraid of being regulated is obvious at a glance.
Finally, the political ledger. Newsom uses "federal inaction" as the target throughout — openly regulating AI, quietly stockpiling a national persona for 2028: California's framework should become the "national baseline." So this order is both regulation and campaign ad. It bets that by the day Washington wakes up, the whole country will be copying California's homework.
Source: California Governor's Office · CalMatters