Human in the Loop Just Became a Spec. Fire & EMS Doesn’t Have One.
Robert Grand · Battalion Chief who still runs calls
Across finance, healthcare, insurance, and law, “human in the loop” is hardening from a phrase you say in a procurement meeting into an engineering requirement with teeth. The pattern everyone is converging on is hybrid: the machine handles speed and pattern recognition, and a person sits at defined decision points to affirm, tailor, or override. The research even has a name for the failure mode it’s designing around. They call it “appropriate reliance,” a decision-maker who neither rubber-stamps the machine nor waves it off. From the outside it reads like a technical problem other people are busy solving.
It isn’t. It’s a doctrine problem, and it’s ours.
Our service has never written down where judgment lives.
What the Rest of the High-Stakes World Just Decided
For years, “human in the loop” was a comfort phrase. You said it to make a tool sound safe enough to buy, and nobody ever defined the loop. Now the serious sectors are defining it, because regulators and courts have started asking them to (Future of Human-in-the-Loop AI 2026, https://parseur.com/blog/future-of-hitl-ai). The work being restructured isn’t the decision itself. It’s the structure wrapped around the decision: which calls require a human checkpoint, what the machine has to show the human to justify its recommendation, and what gets recorded when the human goes a different way.
The research is unusually blunt about why this matters. A human paired with a machine can land in one of two ditches. One is automation bias, where the person defaults to whatever the system says because the system is usually right and arguing with it is tiring. The other is reflexive dismissal, where the person ignores a good recommendation out of pride or distrust. Both produce worse outcomes than either the human or the machine alone. The entire point of the spec is to keep your people out of both ditches, on purpose, by design.
Healthcare is building this into clinical decision support deliberately, shaping the system so it sharpens the clinician’s judgment instead of quietly standing in for it (Human in the Loop: Designing AI That Enhances Rather Than Replaces Clinical Judgment, https://censinet.com/perspectives/human-in-the-loop-designing-ai-enhances-clinical-judgment). They’re treating the override as a first-class event worth capturing, not an exception worth ignoring. We haven’t had that conversation. Most of us haven’t even had the meeting where it gets scheduled.
What a Judgment Call Actually Looks Like on a Run
Picture a clinical decision-support tool that recommends a transport destination. It weighs the patient’s presentation against real-time hospital capability and capacity, and it tells your medic where to go. Most of the time it’s right, and it’s faster than the medic working it out cold. Then one night the medic looks at the patient, looks at the recommendation, and goes somewhere else. Correctly. The machine optimized for the tidy inputs. The medic read the room.
Here’s the part that should bother you. In most departments, that override leaves no trace. Nobody recorded what the tool advised, why the medic overruled it, or who turned out to be right. The single most valuable moment in that whole interaction, a trained human catching what the algorithm missed, evaporates the second the call clears.
Same thing one seat over in dispatch. An assist recommends a response level off the intake, the call-taker senses it’s wrong for this caller, and dials it up. Real save. Unlogged. Or the quieter, worse version: the recommendation under-triages a calm caller having a genuine cardiac event, the words didn’t match the pattern, and nobody downstream ever learns the tool has a blind spot for stoic patients. We’re running judgment calls against a machine on every shift now, and we’ve built no structure to hold them.
The Gap Nobody Named
The reason is simple, and it’s uncomfortable. Protocol tells a medic what to do. It has never told a medic when to distrust a machine that’s advising them, because until very recently there was no machine in the decision. Our entire doctrine assumes the human is the only source of the recommendation. The instant a tool starts proposing the answer, that assumption breaks, and we have nothing written to replace it.
There’s no template for this. No accreditation language, no medical-director playbook, no command position that owns it. The other high-stakes fields are scrambling to define human-machine decision boundaries right now, and they at least have compliance departments and general counsel down the hall. We’d be building the governance from scratch, while the vendors are already in the lobby with the tools.
I’ll be honest about where I am on this. I don’t know yet what the right policy looks like in our service. I know what the first three questions are. Which decisions does a tool get to advise on, and which does it never touch. What does the tool have to show a medic or a call-taker before they’re allowed to lean on it. And when our person overrides it, where does that decision go so the rest of us can learn from it later.
What “Human in the Loop” Actually Means When You Write It Down
Said out loud in a meeting, “human in the loop” means nothing. Written down as a spec, it means three concrete things.
It means decision boundaries. Somebody, and it’s probably a working group with your medical director in it, decides on purpose where the tool advises versus where it has no business being. That’s a clinical and operational judgment, not an IT setting, and the people who own protocol need to own this too.
It means an explanation standard. A recommendation a medic can’t understand is a recommendation a medic can’t responsibly act on, or defend later. If the tool can’t show its reasoning in terms a paramedic can follow at two in the morning, it doesn’t belong in the decision. That standard alone will tell you which vendors to take seriously and which to walk past.
And it means the override gets captured. Not to hang the medic who overruled the machine. To protect that medic, and to teach the next one. The override is the most instructive data your service is going to generate in this entire era, and right now you’re throwing every instance of it in the trash.
That feeling that this is bureaucracy slowing down a good tool? That’s a habit, not a fact. The spec isn’t friction. It’s the only thing that makes the tool safe to trust.
Somebody Has to Own It
Here’s the reframe I keep coming back to. Writing this spec isn’t really about liability, though it will cover you there too. It’s about deciding, on purpose, what your people are for. When a machine can produce the recommendation, the human’s value moves entirely to the judgment around it: when to trust it, when to throw it out, and the discipline to record which one you did and why. That’s not a smaller job than the one your medics and call-takers do now. It’s a harder one.
The old problem was that we had no tools worth governing. That problem is solving itself fast. The new problem is that the tools are arriving ahead of the doctrine, and doctrine that shows up after the tools is just paperwork written in a courtroom. We get to write it first. Most fields didn’t get that chance.
Robert Grand is a Battalion Chief at Eugene Springfield Fire with 24 years of service. He writes Frontline Intelligence, a newsletter on operational doctrine, technology, and leadership in Fire & EMS.
If this landed, share it with someone in Fire & EMS who needs to hear it.
Subscribe
From the floor, not the vendor booth. Two times a week.