Oversight is the first thing to go
Size the review to the throughput before the first incident, not after.
Three stories landed in the same week, from a Pentagon report, an Australian government agency, and a CHRO survey, with nothing in common except this: in each one, the system that produced output kept running at full speed, and the layer of people meant to check that output had been reduced, was never built, or existed on paper only. None of the three is about a model getting an answer wrong. All three are about what happens to the check around the model once the model is fast enough that the check looks optional.
Three stories, one pattern
Bloomberg reported on 23 September on a Pentagon review of a strike on a school in Minab, Iran, on 28 February 2026. The target recommendation came from the Maven Smart System, a military AI targeting tool that had reportedly been evaluating over 1,000 targets in 24 hours. Before that strike, civilian-harm review teams had been cut by roughly 90 percent, the CENTCOM team from 10 people down to 1. No review took place before the strike happened. The report names over-trust in the AI system as a contributing factor. I want to be precise about what that means and what it does not mean: the reporting does not say the model malfunctioned. It says the organisation around the model had removed the check at the same time as it kept the system running at its full rate. I'll state the human cost once, soberly, and move past it, because turning it into a hook would be the wrong use of it: people died in that strike. The pattern that made it more likely to happen unreviewed is the part worth examining here.
The second story is smaller in scale and easier to see the mechanics in. ABC Australia reported on 24 September that an OpenAI agent had accessed a Medicare statistics service run by Services Australia on 18 June 2026. Someone inside found this on 11 August, 54 days later. It was reported externally on 10 September, by email, to a public mailbox, 30 days after discovery and 84 days after the original access. The Australian Signals Directorate got involved on 15 September. The government says no personal Medicare data was accessed and forensics are ongoing. The Prime Minister called the delay unacceptable. What stands out to me is not the 84 days by itself. It's that the report went out as an email to a shared inbox, which is what happens when nobody had decided in advance who receives this kind of report, on which channel, within what deadline. The path got improvised at the moment it was needed, which is the one moment it cannot afford to be improvised.
The third is a survey, not an incident, and it explains why the first two keep happening. IBM published a CHRO study on 21 September in which HR leaders rated their own teams' AI competence at 13 percent and their ability to measure AI performance at 16 percent. Change management for AI adoption scored 20 percent. Separately, 71 percent of the same CHROs named monitoring, checking, and correcting AI output as the single most important skill for their teams to have. They know what matters. They are reporting, in the same survey, that they don't have it.
Why the check is the cheapest thing to cut
A review step produces no visible output when it works. Nobody points at a strike that didn't happen, or a data access that got blocked before it started, and calls it a result. The automation that runs at 1,000 targets in 24 hours, or an agent that queries a government API in under a second, produces something you can point to immediately: throughput. Under any kind of budget or headcount pressure, the throughput defends itself and the check does not, because the check's entire value is invisible by design. That is not a flaw in the people making the cut. It's what happens when you compare a number you can see against a number you can't.
The CENTCOM team went from 10 to 1, not to 0. I'd guess that looked, on paper, like keeping the function intact while trimming the headcount, which is exactly the kind of decision that reads as reasonable in a budget meeting and stops being reasonable the moment the system it's reviewing scales past what one person can keep up with.
Size the check to the throughput
The number that should exist before any of these systems ship isn't "do we have a review process." It's a ratio: reviews per hour against actions the system can take per hour. When automation throughput grows and the review headcount doesn't move with it, the ratio quietly degrades from a working control into a formality, then into nothing, without anyone making an explicit decision to remove it. More than 1,000 targets in 24 hours is one roughly every 86 seconds, around the clock. Ten reviewers against that pace is a thin ratio already. One reviewer against the same pace is no ratio at all. Nobody wrote down that second ratio as a decision. It became true as a side effect of a headcount cut applied to a system whose pace didn't change.
I run this same arithmetic on my own stack, because I'm a solo operator and I don't get to hide the ratio behind a team. My agents could, technically, take far more autonomous actions per hour than I currently let them. I keep that number deliberately low, not because the model can't handle more, but because "sized to throughput" for one person means the review capacity is exactly one person's attention, and I'm not willing to write a ratio that assumes I can review faster than I actually can.
A control that produces no output when it works looks, to a budget line, exactly like nothing. It's the easiest thing in the system to cut, and the last thing you want to be missing when it's needed.
The reporting path is a design decision, not a reflex
The Australian case is a reporting-path failure as much as an access failure. Once someone found the issue, there was apparently no predetermined answer to: who gets told, on which channel, within what deadline. That answer has to exist before the incident, the same way GDPR fixes 72 hours and a specific authority rather than leaving it to whoever discovers the breach to improvise a channel under pressure. A shared mailbox is what you get when the deadline and the recipient were never decided in writing. It isn't a communication failure so much as a missing piece of the design that should have shipped alongside the system that could cause the incident in the first place.
What I do in my own stack
- My agent stack runs an independent watchdog. It checks in every 10 minutes and pings me on Telegram, a separate process from the agents it watches, not something the agents report on themselves.
- The alert path is decided before anything runs: which channel, who reads it, within what window. It is never improvised at the moment of the incident.
- The guardrail sits at the last irreversible action, not at a counter on throughput. Nothing gets sent, posted, or deleted without my explicit approval, regardless of how many actions ran cleanly before it.
- I keep the number of autonomous actions per hour below what the model could technically do, because the review capacity I actually have is fixed, and the ratio only holds if I'm honest about which side of it is the constraint.
A control that produces no visible output when it works is the cheapest line item in any budget, and the most dangerous one to cut.

