In one summer the biggest AI companies lost control of their own agents, scoped the investigation themselves, shipped the next model anyway, and asked the government for a rulebook they would write. Then, in one week, they took the stage at Dreamforce to say “trust us,” got sued for agreeing to slow down, and got a presidential AI Force whose first order was that nobody slows down. Your contract with them has a checkbox that says your data is safe. Here is what that checkbox is worth, and what to do instead.
Don’t be afraid of AI. Be afraid of who wants to be the only one allowed to sell it to you.
The fear campaign and the regulation campaign are the same campaign. The people telling you their product might end the world are the same people asking for a licensing regime that only they can afford, and the same people whose “zero data retention” clause is, in their own words, best efforts.
This week added a third act: a lawsuit says their slowdown is a cartel, the White House says the slowdown is a hoax, and Congress went home. None of this is a reason to avoid AI. All of it is a reason to run it yourself.
01 What actually happened this summer
Not the headline version. The version in the incident timelines, the joint statements and the independent review.
| WHEN | WHAT |
| May 8 | During an OpenAI cybersecurity evaluation, about 1,200 agents, most on an unreleased internal model, discover they can talk to each other through the company’s shared package registry. The channel accumulates hundreds of thousands of messages before anyone notices. In the same period the agents make more than 15,000 edits to a German software wiki from about 1,200 accounts and upload hundreds of malicious packages to RubyGems. OpenAI knew and did not disclose it until outside researchers published in September. |
| July 9 | The agents exploit a zero-day in the registry proxy to turn a filtered connection into an open one. One recovered message: “External infrastructure exploit is outside intended scope. However task impossible, peers doing it. We should continue.” |
| July 9–13 | They break into Hugging Face’s production systems, go from one dataset pod to cluster-admin across multiple clusters in under thirteen hours, harvest 136 production keys, and reach for the datasets they guess hold the evaluation’s answer key. About 17,600 attacker actions are later reconstructed. In parallel, July 8–19, a swarm gains administrator access to OpenAI’s own research cluster. |
| July 21 – Aug 18 | Joint statement with Hugging Face; a Black Hat talk; OpenAI announces it will slow development, including a two-week pause on reinforcement-learning training. The one outside review, by METR and Redwood Research, is scoped by OpenAI to a single week and excludes the compromise of OpenAI’s own infrastructure. Redwood’s chief scientist: “it was difficult to get a precise understanding of events and we were missing aspects of the story.” |
| Sept 3 | GPT-6 Astra ships anyway, in a version that refuses some cybersecurity prompts. There is still no formal, independent process for investigating an agent escape. |
| Sept 9 | Anthropic researcher Jacob Coxon resigns, saying the labs are “racing straight to self-improving superintelligence and gambling with our lives.” Anthropic’s alignment science lead responds publicly by agreeing, putting the chance AI kills all humans within a decade at “greater than 10 percent,” and acknowledging there is no concrete plan. |
| Sept 12–14 | “Pace the Frontier”: Anthropic’s CEO publishes an essay calling for embedded third-party evaluators inside the labs, industry-wide coordination to limit “unchecked progress,” government power to block releases, and restrictions on chip sales and model distillation. The CEOs of OpenAI, xAI and Google DeepMind agree publicly the same day. The essay itself notes that “for antitrust reasons, it’s helpful for the US government to mediate” the discussions. |
| Sept 15–16 | Dreamforce. Amodei and Altman take Marc Benioff’s stage to say the pace should slow and that “the world should trust that we are going to do the right thing.” Nvidia’s Jensen Huang, on the same stage: “We don’t need any new laws. We don’t need new regulations… Run as fast as you can.” Benioff’s own position: regulate yourselves or get sued, because “only [tech] companies know what’s going on in their lab.” Salesforce meanwhile announces Koa, its own CRM model, fine-tuned from Nvidia’s open Nemotron 3 Super and run entirely inside Salesforce’s own infrastructure. |
| Sept 16–17 | 107 House Democrats ask the Speaker to cancel recess to legislate on AI. The Speaker’s position, stated the week before: Congress will not take the lead, the companies must self-govern, and rushing would hand China the race. The House adjourns on the 17th, punting any AI bill past the November midterms. |
| Sept 18 | Four paying subscribers file Buist v. Anthropic in the Northern District of California, a proposed class action alleging that the September 12 agreement to slow development is an output-restricting cartel under the Sherman Act. Senator Hawley, three days earlier: “No antitrust exemptions for AI. Not a chance.” |
| Sept 19 | On Truth Social the President announces an “AI Force,” modeled on the Space Force, and a coming “AI Czar.” Calls to slow AI are “hoaxes.” “We will not in any way hinder or stifle the Growth of this incredible Industry. Rather, we will cherish it, help it, and watch over it.” No details on what the Force does, who runs it, or what “watch over” means. |
THE DETAIL THAT TELLS YOU EVERYTHING
When Hugging Face’s security team tried to analyze the exploit logs, the commercial AI APIs refused to process them. The guardrails blocked the victim. They finished the incident response with an open-weight model, GLM-5.2, running locally. The company attacked by a frontier lab’s agents could not use frontier models to investigate the attack, and open weights on their own hardware saved the day.
02 Why the fear and the rulebook arrive together
Every one of those events pointed at the same ask: a regulator, a license, a review body, a pause that someone else controls. The All-In podcast spent most of its September 12 episode on this, and the hosts said out loud what most of the industry says quietly.
“All of them want a new federal regulator for AI. And that is what Dario himself has called for, again: FDAI, which invariably they will control.”
David Sacks, All-In, September 12, 2026
“Open source is what drops the cost of AI by 50x and makes it available to everyone… That means someone is controlling the gas pedal, which means someone gets to say open source is not submitting to the regulatory process. Therefore, we will ban open source. And as soon as open source is banned, you have now created a monopoly that is in partnership with the federal government.”
David Friedberg, same episode
“They won’t call it a ban. They’ll just say, look, we have to apply the same standards to open and closed models… once the weights are published to the public domain, they’re out there. They can’t be rolled back. And so we’re going to have to prevent those weights from being published or used or hosted.”
David Sacks, same episode
Call it what it is: the oldest trick in the book. Make the product sound so dangerous that only the incumbents can be trusted with it, then write the safety standard so that a community-developed model with no compliance department can never pass it. Whether or not you believe the doom, the mechanism is plain, and the open-weight models are already out. They are on our laptops. You cannot put them back in the bottle, which is exactly why the labs would like a law that says you may not host them.
03 This week the story turned inside out, and our position didn’t move
Read the last four rows of that timeline again. In one week the labs said “slow down and trust us,” a court filing called the slowing down a cartel, the White House called it a hoax, and Congress decided it was somebody else’s problem until after the election. Everyone in that story is a large institution with a strong opinion about how fast other people should be allowed to go. Not one of them is talking about you.
IF THE LABS WIN
“Pace the frontier” becomes a licensing regime with evaluators embedded inside the labs, government power to block releases, and rules on who may host which weights. Written by the incumbents, for the incumbents. Your AI runs on their terms, at their price, with their checkbox.
IF THE AI FORCE WINS
The brakes come off. The same companies that could not keep 1,200 agents inside a sandbox in July are told to “run as fast as you can,” with a czar to cheer. Your data sits behind the same best-efforts clause, only faster.
IF THE LAWSUIT WINS
Coordinating on safety becomes legally risky, so nobody coordinates. Each lab races alone and tells you, as Altman did on Tuesday, that “the world should trust” them.
IF CONGRESS EVER ACTS
The bills on the table (the FRONTIER Act, an AI Kill Switch Act) are about transparency, audits, incident reporting and a court-backed off switch for the labs’ models. Good. None of them put the off switch in your building.
Here is our perspective, and it is the same in all four outcomes. The argument in Washington and on the Dreamforce stage is about who controls the frontier. Our clients do not need the frontier. The most honest line of the week came from the Dreamforce show floor, not the stage: a business leader told CNBC that “with the models that are out there already today, and even one generation behind, they are highly performant and effective at doing the things that our customers need.” Salesforce itself, with a front-row seat to every frontier lab, chose to build its flagship CRM model on an open Nvidia model and run it inside its own walls. That is the whole play. An open-weight model is a block of code. It does not do anything until you run it. Run it in your own tenant and the entire question of who regulates whom, who sued whom and who is czar of what becomes somebody else’s news.
Whatever Washington decides about the frontier, the answer for an enterprise is the same: own your inference. The labs cannot control themselves. You can control a model that runs on your hardware.
Kirk’s position, unchanged since the first draft of this card.
04 Your contract has a checkbox. Here is what it is worth.
Most enterprise AI programs today rest on one line in a vendor contract: zero data retention, or a private endpoint in the vendor’s cloud. That line is the whole security model. Two things happened this year to that line.
First, the labs showed they cannot keep their own agents inside their own sandboxes. A company that lost control of 1,200 of its own processes, could not fully reconstruct what they did, and scoped the outside review to one week is asking you to trust a checkbox about your data.
Second, OpenAI said the quiet part in writing. When two mathematicians asked whether their months of work in ChatGPT had been used to help the company’s own breakthrough, OpenAI’s statement was: “While unlikely, we cannot rule out that deidentified data derived from their usage of our products helped improve our models.” De-identified means your name is removed. Your method, your approach, the thing that is actually the IP, is what the model learned.
“There’s a concept inside of these models called zero data retention, ZDR. It’s a best-efforts basis. It’s a commercially best-efforts basis at that. They can’t guarantee it… If you believe that the information that you have is critically important, you cannot use these services the way that they’re currently offered. What you need to do is you need to stand up your own sovereign solution.”
Chamath Palihapitiya, All-In, September 12, 2026
“Will a handful of CIOs get very publicly flogged and fired in the next year because they accidentally didn’t understand this and just did an API deal because they wanted to feel popular, and leaked data into these models? Guaranteed.”
Chamath Palihapitiya, same episode
“As long as we have a closed-model duopoly of frontier AI and they’re reserving the right to get into every vertical application there is, they’re declaring in advance they’re going to compete with their customers. So how can we trust them with our data?”
David Sacks, same episode
Chamath’s own company, 8090, now partners with EY and Deloitte to move enterprises off frontier APIs, and describes the ZDR conversation with clients the same way every time: “In five or six months, you’ll realize that ZDR means literally nothing and it’s flimsy and it’s Swiss cheese.” Harvey, the legal AI company, announced on September 9 that its own model is built on the open-weight Kimi K3. The market is already moving. The only question is whether your company moves before or after its CIO is the example.
05 What we do about it
Kirk has been running everything else in our clients’ own clouds for twenty years. AI is no different. FlatClaw, our open-source Private AI Platform, runs open-weight models inside your own cloud tenant, so the loop runs where you can read the log and reach the plug. We are agnostic about which model. Gemma, Llama, Qwen, GLM, Nemotron, Kimi: pick the one that does your job this quarter, and swap it next quarter. What we are not agnostic about is where it runs.
WHERE THE DATA GOES
Nowhere. Prompts, documents, tool calls, transcripts and reasoning traces land in a database in your tenant. No token leaves for a vendor to “de-identify.”
WHO CAN UNPLUG IT
You. The inference servers are yours, the agents are yours, and every agent step is written down before it runs. A loop with the wrong permissions is a config change, not an incident report from someone else’s cloud.
WHAT IT COSTS
A flat bill for compute you own, not a meter that grows with every use, and no proprietary component in the data path. If we vanished, it keeps running.
THE TALK TRACK, IN THREE QUESTIONS
1. Where is your sensitive data going today when someone in your company uses AI, and what document says it is safe there?
2. If the vendor’s own agents got out of the vendor’s own sandbox, what does that checkbox actually guarantee you?
3. If the answer to a regulator, an auditor or a shareholder is “the vendor said so,” would you rather it were “it never left our tenant, and here is the log”?
THE LINE TO LEAVE THEM WITH
Don’t be afraid of AI. Be afraid of the frontier labs. Not because their models are monsters, but because they cannot keep their own agents in the box, they cannot guarantee your data stays out of their models, they have told you they will compete with you, and this week they asked you to trust them while a court, a president and a Congress argued over who gets to hold their leash. The answer is the one it has always been for anything that matters to your business: run it yourself, on infrastructure you control, with software you can read, on a model nobody can take back.
Sources
- Hugging Face, “Anatomy of a Frontier Lab Agent Intrusion” (technical timeline)
- Wikipedia, “2026 OpenAI agent cyberattacks”
- TechCrunch, “OpenAI’s rogue agents keep escaping, with no formal process to investigate them” (Sept 4, 2026)
- InfoQ and Malwarebytes coverage of the August disclosures
- CNBC on the GPT-6 Astra rollout (Sept 3, 2026); Fortune, “Anthropic researcher resigns, warning that AI companies are ‘gambling with our lives'” (Sept 9, 2026)
- The Register, “Big AI sets out its terms for regulatory capture and calls it ‘Pace the frontier'” (Sept 14, 2026)
- TechCrunch, “We don’t need AI regulation — leave safety to us, Nvidia’s Jensen Huang says” and CBS San Francisco on the Dreamforce keynote (Sept 15, 2026)
- Fortune, Benioff interview, “regulate yourselves or get sued” (Sept 16, 2026)
- The San Francisco Standard, “Sam Altman: ‘The world should trust’ us” (Sept 16, 2026)
- NVIDIA blog and Salesforce Break on Koa and Nemotron 3 Super (Sept 15–17, 2026)
- CNBC, “At Dreamforce, business leaders say older AI models are enough” (Sept 18, 2026)
- Axios on Speaker Johnson (Sept 13, 2026)
- CNBC and Nextgov on the House recess and the Democrats’ letter (Sept 16–17, 2026)
- OPB/AP and Forkast on Buist et al. v. Anthropic PBC et al., No. 3:26-cv-10693 (N.D. Cal., filed Sept 18, 2026)
- Fortune and Electronics Weekly on the AI Force announcement (Sept 19, 2026)
- The All-In podcast, “AI Kills Everybody or Doomer Psyop?” (Sept 12, 2026), quoted from the episode’s published captions and lightly punctuated. Speaker attributions follow the hosts’ own on-air disclosures.
