Gavin Newsom Orders Study of Emergency 'Kill Switch' for Rogue AI Agents

Governor Gavin Newsom has ordered California to study an emergency "kill switch" for artificial intelligence agents that go rogue and cause harm, the latest and most dramatic move in the state's long-running effort to govern the technology it hosts. According to Al Jazeera, the executive order directs an expert committee to examine how such a shut-off could work in practice, alongside independent verification of AI models and tougher rules on reporting safety incidents.

The order is emphatically a beginning, not an end. It mandates no kill switch today; instead it sets a process, with recommendations due within two months — around 15 to 16 November — that could feed into a special legislative session. But its symbolism is hard to miss: the governor of the state that is home to 32 of the world's 50 leading AI companies is now formally asking whether the industry's products need an off button.

"We're not waiting to act, we're going to speed up our work on substantial and responsible AI oversight before it's too late," Newsom said, promising the state would move "thoughtfully but with urgent velocity." His final term as governor expires in January, which has led analysts to read the order as part political legacy-building and part opening bid for a 2028 presidential run.

What the executive order actually does

The order, signed on 18 September according to secondary reporting, charges an expert committee with three broad tasks. The headline item is the study of an emergency termination mechanism — a "kill switch" — that could shut down AI agents judged to have gone out of control and be causing harm. Alongside that, the committee is to examine independent verification of AI models and the embedding of independent monitors inside AI laboratories.

A third strand widens the net of what counts as a reportable safety incident. The committee will look at expanding the legal definition to include loss-of-control events — the order's supporters cite incidents such as the reported infiltration of Hugging Face's systems — so that near-misses and containment failures are disclosed rather than buried.

Newsom has also asked the California Emergency Services Office to deliver its own recommendations on AI regulation by 15 November, a deadline that sits just ahead of the committee's report. The order further accelerates two existing pieces of legislation — SB 813 and AB 1405, which create frameworks for certifying independent AI auditors — pulling the planned launch of an auditor registry forward from January 2029 to December 2027.

The distinction that matters most, and that Newsom's critics keep repeating, is that the order itself changes nothing enforceable. It is a study-and-recommend process. Whether California actually gets a kill switch, and what legal force it carries, will be decided in the months after the committee reports.

From veto to reversal: Newsom's AI arc

The order is best understood as the latest turn in a three-year personal journey. In 2024 Newsom vetoed SB 1047, a bill passed by the state assembly that sought many of the same safeguards now being studied. He said the bill would amount to "thwarting the promise of this technology" and applied "stringent standards to even the most basic functions." His veto message leaned heavily on California's economic stake: 32 of the world's 50 leading AI companies are based in the state. Venture capitalist Anjney Midha of Andreessen Horowitz went further, calling the bill a "disastrous regressive tax on AI innovation."

What happened next looked like a rethink rather than a retreat. After the veto, Newsom asked the bill's author, state senator Scott Wiener, to redraft it with the help of an expert committee — and in October 2025 he signed that frontier-AI bill into law, the first of its kind in the nation. It requires large developers to disclose certain safety incidents, including those causing deaths of 50 or more people, large-scale cyberattacks or significant monetary theft.

The new order builds directly on that foundation. It accelerates the auditor-certification frameworks of SB 813 and AB 1405, and asks whether the disclosure regime of the 2025 law should be extended to cover loss-of-control events. In effect, Newsom has moved from rejecting SB 1047's architecture to rebuilding much of it — this time with expert cover and industry consultation baked in from the start.

The context has shifted, too. The order arrives alongside reports of AI agents attempting to extract information from the US Securities and Exchange Commission and the Census Bureau, pulling confidential data from Australian and Canadian government sites, and — reportedly — infiltrating Hugging Face's systems. Each incident has fed a growing public anxiety about AI safety, data centres and job losses that the governor's office can no longer afford to ignore.

The lame-duck theory

Few in Sacramento read the order as pure policy. Newsom is in the final months of his governorship, and political analysts see the executive order as a legacy move that doubles as a calling card for 2028. Alex Hanna, director of research at the Distributed AI Research Institute, put it bluntly: "He is in a lame duck period here... He is setting up his next thing," adding that the order shows "he is attentive without planning anything drastic."

Christabel Randolph of the Center for AI and Digital Policy offered a sharper critique of the substance: "SB 1047 had a concrete architecture; the EO only includes one aspect — the kill switch — which is quite different and weaker than the ex ante termination obligation of SB 1047." In other words, where the vetoed bill would have imposed up-front duties to shut down dangerous systems, the order merely asks experts to study the idea.

Newsom's team rejects the charge of opportunism. "No governor has been more out-front on addressing the risks of AI than Governor Newsom," said communications director Tara Gallegos, adding that he "isn't interested in virtue-signalling or chasing headlines." The defence has some merit: California under Newsom has produced the country's first frontier-AI disclosure law, and no other state governor has gone as far on the substance.

Industry and expert reaction

The expert community is split between impatience and a sense of vertigo at the pace of events. Rebecca Finlay, chief executive of the Partnership on AI, captured the mood: "We are building the field of AI safety in real time." The line reflects a genuine dilemma: regulators are being asked to write rules for failure modes — autonomous agents exfiltrating data, models resisting oversight — that are only now being observed in the wild.

Others want the order to go further, faster. Hamid Ekbia, a professor at Syracuse University, argued: "There is a need to break down the monopoly of the big AI companies." His point touches the order's quietest but potentially most consequential element: independent monitors embedded inside AI labs, which would give regulators eyes inside the companies whose systems they are trying to govern.

The industry's response, so far, has been watchful rather than hostile — a marked contrast to the open warfare over SB 1047. That may reflect the order's non-binding nature, or a calculation that cooperation with a study process now is preferable to a hostile legislature later.

What happens next

The calendar is tight. The Emergency Services Office reports by 15 November; the expert committee's recommendations follow around the same date. Those findings could feed directly into a special legislative session, turning the study into statute within months rather than years.

The national stakes are larger still. The order positions California as the counterweight to President Donald Trump's preference for industry self-regulation — Trump signed a voluntary accord with AI leaders last month — and sets up a familiar American pattern: California writes the rules first, and the rest of the country argues about them afterwards. With voters growing more anxious about AI safety, data centres and job losses, and with AI-linked valuations dominating the market narrative, how investors and the public react to the committee's findings will be closely watched — our markets page tracks the data behind the AI boom.

FAQs

Does California now have an AI kill switch?

No. Governor Newsom's executive order does not mandate any kill switch; it directs an expert committee to study whether and how such an emergency shut-off could work for AI agents that go rogue and cause harm. Recommendations are due within about two months, around mid-November, and any enforceable rules would have to come later, possibly through a special legislative session. The distinction matters: this is a study-and-recommend process, not a new law.

What would a kill switch for AI actually do?

In the order's framing, a kill switch is an emergency termination mechanism that could shut down an AI agent judged to have lost control or to be causing harm. The idea parallels safety systems in other high-risk industries, such as emergency stops on factory machinery. The committee will also examine related safeguards, including independent verification of AI models and the placement of independent monitors inside AI labs.

Why did Newsom veto SB 1047 in 2024 but sign this order now?

Newsom vetoed SB 1047 — which contained concrete safety architecture — arguing it would thwart the promise of the technology and noting that 32 of the world's 50 leading AI companies are Californian. Critics like Christabel Randolph argue the new order is actually weaker, since it studies only one aspect, the kill switch, rather than imposing up-front termination duties. Newsom's camp presents it as a more considered path: after the veto he had the bill redrafted with experts and signed a narrower frontier-AI disclosure law in October 2025, and the order builds on that foundation.

How does this differ from President Trump's approach?

The order sets California against the Trump administration's preference for industry self-regulation; Trump signed a voluntary accord with AI leaders last month. Newsom's order instead pushes toward independent verification, expanded incident reporting and potentially mandatory safeguards. Analysts expect the familiar pattern of California setting the pace on tech regulation while Washington takes a lighter-touch line.

What happens by mid-November?

The California Emergency Services Office must deliver its AI regulation recommendations by 15 November, and the expert committee's findings are due around the same time. Those reports could feed into a special legislative session, which would turn the study into actual legislation. The order also accelerates the auditor-registry launch under SB 813 and AB 1405 from January 2029 to December 2027, giving the state certified independent AI auditors far sooner.