Who Owns the Override?

The pitch for AI on the 911 floor and in the back of a medic unit always sounds responsible. The vendor says the tool only recommends, the human still decides, there is always a person in the loop. It is the line that closes the deal, because it lets a careful department adopt fast technology without feeling like it handed over the call. Regulators heard the same line. They just stopped taking it on faith.

Human in the loop is a sentence, not a safeguard.

What Europe Just Put in Writing

The EU AI Act now classifies AI that evaluates and classifies emergency calls, that dispatches or prioritizes first response, and that runs emergency patient triage as high-risk (Annex III High-Risk AI Systems, https://artificialintelligenceact.eu/annex/3/). That classification is not a label. It pulls in a set of obligations, and the one that matters most here is the human-oversight duty in Article 14, which says a deployer has to assign oversight to a real person with the competence, the training, and the authority to understand the system, question it, and override it (EU AI Act Article 14, Human Oversight, https://artificialintelligenceact.eu/article/14/). The transparency rules land in August 2026, with the broader high-risk obligations phasing in behind them.

Strip away the regulatory language and here is what changed. Somebody wrote down, in law, that the human in the loop cannot be a rubber stamp. They have to be trained well enough to spot a bad recommendation and stand high enough to throw it out. The emergency-services groups reading the Act came to the same conclusion: agencies will have to build real oversight, with competent people behind a human-machine interface that lets them actually intervene, not just watch (How the AI Act Will Shape the Future of Emergency Services, EENA, https://eena.org/blog/how-the-ai-act-will-shape-the-future-of-emergency-services/).

It is European law. It does not bind a department in Oregon or Ohio. But procurement language travels, and our own regulators are circling the same questions. This is the clearest written statement yet of where the accountability is supposed to sit, and it is worth reading before someone writes the American version for us.

The Loop We Already Pretend We Have

Walk into most comms centers and ask who can override the AI triage assist, and you will get a confident answer: the call-taker, of course, the human is always in control. Then ask what training that call-taker got on when to distrust the tool, and how their shift is measured. The answers get quieter. The call-taker is measured on call volume and handle time. Nobody ever taught them that the recommendation is sometimes wrong, or gave them cover to slow down and argue with it.

A human in the loop sounds like a safeguard. Too often it is just a seating chart.

That gap is the whole problem, and it is not a technology problem. The tool can be excellent. The oversight can still be a fiction, because the person we named as the safeguard has neither the training to catch the error nor the protected authority to act on it when they do. We have built a lot of these loops already, in dispatch, in clinical decision support, in the documentation tools creeping into the ePCR. Most of them are loops in name. The human is in the diagram. The human is not really in the decision.

The Four Questions That Make Oversight Real

I do not know exactly what the right override policy looks like in every department. I know the four questions it has to answer, and an agency that cannot answer all four has not actually put a human in the loop, no matter what the contract says.

The first is authority. Who, by name and by role, is allowed to countermand the AI, and is that written down somewhere a new hire can find it. Not “the team,” not “supervision,” a defined role with the standing to say no and make it stick. If overriding the tool means going up two levels and explaining yourself first, nobody will do it at three in the morning, and the override that lives only on the org chart does not exist.

The second is competence. What training makes that authority real. A person with the power to override and no idea when to use it is worse than no safeguard at all, because they create the appearance of one. The role has to come with deliberate training on the tool’s failure modes, the kinds of calls where it tends to be wrong, and the skill to run the problem independently when something feels off. That last part is the hard one, because the tool itself erodes that skill through daily use, which means the competence has to be maintained on purpose, not assumed.

The third is protection. What shields the override authority from the metrics that quietly kill it. Every efficiency case for AI triage runs on doing more with fewer people, and that pressure lands directly on the person whose job is to slow down and second-guess the machine. If your numbers reward speed and your policy rewards scrutiny, speed wins every shift, and the oversight gets hollowed out without anyone deciding to hollow it out. The protection has to be explicit: this role is allowed to be slower, and using the override is never the thing that shows up against someone at review.

The fourth is documentation. When the human overrides, or chooses not to, is there a record of who, what, and why. Not to punish the call, but to prove the loop was real, to learn where the tool fails, and to stand behind the decision later when someone asks. An override that leaves no trace is indistinguishable from no override at all, and in two years, when the question comes from a courtroom instead of a QA review, the trace is the only thing that answers it.

What “Override” Actually Means on a 911 Floor

Put the four questions on a real call. An AI assist scores an incoming complaint and recommends a BLS response to what it reads as low-acuity chest discomfort. The call-taker has a feeling. Something in the caller’s voice, a detail the model flattened, the kind of pattern a seasoned ear catches and a classifier misses.

Now run the four questions. Does that call-taker have the named authority to bump the response over the tool’s recommendation. Were they trained to recognize this exact failure, the atypical presentation the model under-reads. Is their shift built so that taking the slower, safer path does not ding their numbers. And when they make the call, does the system capture that they overrode, and why. If the answer to all four is yes, you have a human in the loop. If the answer to any of them is no, you have a recommendation engine with a person sitting next to it, and the next bad outcome is going to find the gap you left open.

That is the difference between oversight as a promise and oversight as a position. One is a word in a contract. The other is a person you can name, trained to disagree, protected when they do, and documented after.

The Person, Not the Promise

We are about to adopt a lot of tools that decide things, or come close enough that the difference stops mattering at three in the morning. The instinct will be to reassure ourselves with the same sentence the vendors use, that there is always a human in the loop. The work is making that sentence true before we need it to be. A department that names the override, trains it, protects it, and documents it stays the author of its own calls even while it runs someone else’s software. A department that takes the phrase on faith finds out, on the worst call, that the loop was empty the whole time.

The machine can recommend. Someone still has to be able to say no, and mean it. That someone is not a line in a procurement document. It is a person you can name, and right now, in most departments, you cannot.

Robert Grand is a Battalion Chief at Eugene Springfield Fire with 24 years of service. He writes Frontline Intelligence, a newsletter on operational doctrine, technology, and leadership in Fire & EMS.

If this landed, share it with someone in Fire & EMS who needs to hear it.