“Human in the loop” is not a control by itself. It becomes a control only when the human has a defined role, the evidence needed to decide, and the authority to stop or redirect the system.
gate 01
gate 02
NIST’s AI Risk Management Framework emphasizes defining and documenting human-AI roles, responsibilities, and oversight. That sounds procedural, but it has a direct architectural consequence: the system must know when to pause, what to present, who can decide, and how that decision changes execution.
Put gates at decision boundaries
Do not ask for approval after every model call. That destroys the throughput autonomy was meant to create and trains reviewers to click through. Place gates where one of three things changes:
- Intent: the system is about to commit to a scope, requirement, or tradeoff that shapes everything downstream.
- Exposure: the next action affects customers, production data, money, permissions, or an external system.
- Confidence: evidence is missing, checks disagree, or the system has left the conditions under which it was evaluated.
In a product build, useful gates often appear after the scoped specification, before production deployment, and when a safety or quality check fails. Routine implementation between those points can proceed autonomously inside defined constraints.
Design an approval packet
A reviewer should not need to reconstruct the run from a transcript. Every gate needs a compact packet that supports a real decision:
- what the system proposes to do;
- what changed since the last approval;
- the assumptions and unresolved risks;
- the evidence that required checks passed;
- the blast radius and rollback path; and
- the exact choices available to the reviewer.
The choices should be structured: approve, reject with a reason, request a specific revision, or escalate. Free-form comments can add context, but they should not be the only way to steer the system.
Match oversight to risk
Not every build or action needs the same gate. Use risk tiers based on reversibility, data sensitivity, user impact, financial impact, and how well the behavior is covered by evaluations.
This lets the system move quickly on familiar work while preserving attention for consequential decisions. The gate policy itself should be versioned and testable, just like application logic.
Give the human real ownership
A gate fails when the reviewer is accountable but powerless. The person approving scope needs enough product context to challenge it. The person approving a release needs access to test results, security findings, deployment impact, and rollback readiness.
Define who can approve each class of decision, who may override, and what requires two-person review. Record the approver, evidence shown, decision, reason, and resulting system action. That trace supports incident review and reveals whether gates are producing useful corrections.
Use decisions as evaluation data
Human corrections are expensive and valuable. Capture them as structured feedback. If reviewers repeatedly reject the same missing edge case, add it to the planning eval. If they always approve a low-risk class unchanged, consider automating that gate. If reviewers disagree, the policy or success criteria may be underspecified.
The objective is not to remove humans from the system. It is to move their judgment to the smallest number of moments where context, accountability, and consequence meet.