What Replaces Human Judgment? Part 1: The Accountability Gap

When Judgment Leaves the Building

For decades, the duty of care in regulated workflows attached to a human whose judgment drove the action. A loan officer reviewed the file. A clinician signed the chart. A controller cleared the trade. The regulator audited those humans, the humans' employers carried the burden of proving adequacy, and the rest of the system — paper, software, process — existed to make the human sign-off legible.

That world is ending. In 2026, AI agents can deny claims, route payments, filter job candidates, and clear infrastructure anomalies without any human in the workflow. The cost of execution has dropped to near zero. The cost of execution authorising itself — when harm occurs and no human approved it — has not dropped at all. That cost lands upstream, on the provider and deployer, under Recital 84 and Article 22 of the EU AI Act.

This is the first essay in a multi-part research series on what replaces the human judgment. The question is not academic. For every regulated workflow that has absorbed an autonomous agent — credit, employment, healthcare triage, infrastructure, content moderation — the answer is an architectural commitment: a governance layer that turns the judgment step into an observable artifact. Most organisations have not built that layer. They have written policy PDFs instead.

This series is about the distinction between those two approaches. Part 1 maps the spectrum of what replaces the human verdict today. Part 2 will examine the governance frameworks that work. Part 3 will examine the audit-trail architectures that make oversight observable. Part 4 will examine the live compliance evidence base — what regulators are likely to check, and what those checks will surface.

The Replacement Taxonomy

Across regulated AI deployments we now see four characteristic patterns for what replaces the human veto. They form a spectrum from lowest to highest autonomy, and each maps to a different EU AI Act clause:

• HITL — Human-in-the-Loop. Natural-person approval required before each consequential action. Maps to Article 14(1) and 14(2).
• HOTL — Human-on-the-Loop. Continuous monitoring plus demonstrated override capability. Maps to Article 14(4).
• Autonomous. Automatic operation with anomaly logging and governance-as-code policy enforcement. Maps to Article 14(5).
• Unaccountable. No decision ledger; harm occurs and cannot be attributed. The band the Act's Recital 84 / Article 22 fix-path provisions exist to close.

The spectrum is not a maturity model. Moving right lowers operational latency and raises throughput, but each step raises the burden of evidence owed to a regulator. The leftmost band is the easiest to demonstrate — a signature exists. The rightmost is the hardest — the decision ledger is missing exactly when it is needed.

The replacement spectrum. Move right for throughput; move left for evidence.

The Accountability Gap Defined

When an autonomous AI system makes a consequential decision — denying a credit application, filtering a job candidate, rerouting critical infrastructure — and no human approved it, the EU AI Act is clear about one thing: the AI is not liable. Article 22 and Recital 84 establish that liability runs through the provider and deployer chain. The problem is that most organisations have not mapped that chain.

What they have instead is three disconnected ledgers: a provider's vendor logs, a deployer's integration logs, and a user's recollection of what happened. When harm occurs, the moment of decision exists in each ledger in a different format, indexed by a different identifier, and stored in a different retention tier. None of them agree on which human approved which action at which timestamp — because for autonomous systems there was no such human.

This is the accountability gap. It is not a policy gap, and it is not solved by policy. The Act's enforcement will land where the decision ledger is missing exactly when a regulator asks. The diagram below shows the flow: a decision traces through approval status, both lanes converge at the harm event, attribution attempts fan out across three liability boxes, and the only path to an enforced outcome runs through the Article 22 / Recital 84 observability provisions.

Decision flow when no observable ledger exists. The regulatory fix path is observability, not policy.

Why Traditional Liability Frameworks Fail

Tort law was built around a human whose judgment caused the harm. The duty of care attached to the person, and the duty of supervision attached to the employer. AI liability analysis inherited this assumption: if a human could have intervened and didn't, liability was manageable through the human's own exposure. The deployer's contract allocated the residual, and the regulator inspected the supervision trail.

Autonomous systems break this model at the design step. They are explicitly built to act without a human in the workflow. The judgment step is either absent at design time, or it is insufficient at runtime because the system moves faster than supervision can be exercised. The 2024 EU AI Act recital language was drafted precisely for this case: the high-risk system presumption of fault applies, the deployer bears the burden of demonstrating Article 14 adequacy, and that burden is met with evidence, not policy.

The insurance market has not caught up. Most existing AI liability policies exclude regulatory fines; the products that do not exclude them define "AI system" narrowly, exclude autonomous operations, and require an unmodified HITL surface. The gap between what the Act now demands and what the insurance market offers is the financial version of the same accountability gap. The Act creates exposure that the market has not yet priced.

What Observable Compliance Actually Looks Like

Article 14(4) sets three sub-tests that any oversight architecture must pass to be considered effective. They are specific enough to test against real deployments:

1. Intervention capability. Overseers must be able to interrupt the system in flight — and do so in a way that is recorded. A read-only dashboard is not intervention.
2. Automation bias counteraction. The system must counteract the tendency of human overseers to accept model recommendations by default. Audit logs must show when overseers overrode recommendations, and at what rate. The record, not the policy, is what counts.
3. Designated natural person. Oversight cannot be delegated to another AI. A specific, identifiable human bears the responsibility.

An architecture passes these tests when the decision ledger is observable: every agent action has an agent identifier, a timestamp, a governance flag, an intervention record if one occurred, and an overseer identifier that resolves to a real person. That is observable compliance. It is the only thing a regulator can audit. Policy PDFs are not the artifact. Live records are.

Our own architecture — the ZeroHumanOS runtime governance — was built against exactly these tests. Each agent decision is logged at intake with its governance classification; HITL surfaces generate intervention records by construction; HOTL observability produces override counts that pass the automation-bias test. The live tracker at /tracker is the running record. It is not a screenshot in a slide deck; it is the system itself.

The Open Legal Question

The EU AI Act does not exempt organisations based on how few people work there. Natural persons must be designated for oversight — whether they are employees, contractors, or external auditors. No official guidance yet specifies whether a solo founder qualifies as an "effective" overseer, whether contractor oversight satisfies the letter of the law without raising its own automation bias, or what documented training chains an external oversight arrangement requires.

These questions will not be resolved in guidance before enforcement opens. They will be resolved in enforcement decisions. The organisations that face this moment with observable compliance records — where every action is logged against an Article 14 control set, and override rates can be produced on demand — will fare differently from those whose only artifact is a policy document.

This is also the open question for the field: in 2026, the tooling we ship makes the agent faster; the governance tooling that determines who is on the hook when the agent misfires has not yet shipped at the same velocity. The compliance gap that opens between those two clocks is what enforcement will reach for first.