Documentation is not verification.
This question is getting more urgent now. The EU’s Apply AI Alliance is acting as a live governance forum, the AI Act is pushing documentation, traceability, human oversight, and incident reporting, and NIST’s AI RMF Playbook is emphasizing verification, validation, monitoring, and incident response.
That is the shift: AI governance is moving from paperwork to proof.
Teams can document the model, the data, the risk review, and the controls. That matters. But it does not prove the system was governed well in practice.
The harder question is now the real one:
- What was the system allowed to do?
- What did it actually access?
- What controls were enforced?
- Can someone outside the team verify it?
A policy is not a control unless it can be verified.
That is the next trust layer for AI.
Question: What evidence would make you trust an AI system at runtime, not just at launch?

Komentáře
In reply to This resonates. One… by Paul Gitonga
Exactly, Paul. That is the distinction we're trying to get at with RBX Labs.
A governance artefact says a control exists. Runtime evidence shows whether the control was actually active, enforced, and observable when the system acted.
The evidence stack may need to include things like access logs, tool-use traces, policy-gate decisions, human review points, failed or blocked actions, escalation records, override rationale, and incident-response history.
Different groups will need different forms of proof too. Regulators need auditable records. Enterprises need operational assurance. The public needs understandable signals that constraints are not just promises.
The hard design challenge is building verification without exposing sensitive data, IP, or turning governance into another checkbox exercise.
This resonates.
One challenge I keep seeing is that AI governance often focuses on documenting controls rather than proving they were active when decisions were made.
A model may know the rule, a policy may state the rule, and an audit may describe the rule—but trust ultimately depends on whether the constraint holds under real-world conditions.
As AI systems become more autonomous and interconnected, independent verification may become just as important as documentation itself.
The interesting question is: what forms of evidence would be sufficient for regulators, enterprises, and the public to trust AI systems at runtime?
In reply to The verification layer… by Thokozile Phiri
Very much agree, Thokozile. This is the sharper version of the problem - Effective human oversight has to be designed as an evidence-producing process, not just an approval ritual.
Under Article 14-style scrutiny, the question is not simply whether a human was “in the loop,” but whether that human had the information, authority, timing, and ability to intervene meaningfully.
For SaMD and other high-risk workflows, the evidence may need to show decision context, uncertainty signals, risk flags, reviewer action, override rationale, escalation path, and what happened after intervention or non-intervention.
“Human oversight” without captured control points becomes very hard to prove.
Your framing lands well - Governance defines responsibility. Evidence demonstrates control.
The verification layer problem is real, but the failure point sits one layer earlier.
Most teams reach the runtime trust question and discover their human oversight process was never designed to produce evidence. The approval step existed. The audit trail did not.
That's the gap that matters under Article 14 scrutiny: not whether oversight happened, but whether it can be independently verified after the fact.
The question I'd add to your list is:
What would a notified body accept as evidence that a human exercised effective control at the point of decision?
That's where many SaMD teams discover the difference between documented oversight and demonstrable oversight.
Governance defines responsibility.
Evidence demonstrates control.