Most discussions of Article 14 I have read so far, here and in adjacent forums, still treat human oversight as a deployment-time property. A box is ticked when the model is validated, a dashboard is installed, a playbook is written. That framing fits classifiers. It does not fit agents.
Agentic systems do not act at deployment time. They act at runtime, tool call by tool call. A model that passed every pre-deployment check can still cause loss, exfiltrate data, or degrade a regulated process through a sequence of individually innocuous actions. Article 14 obliges deployers to enable effective oversight while the system is in use. For agents, that obligation only bites at the granularity of the action.
What follows is an argument for why the current model-level governance stack does not satisfy Article 14 for agentic deployments, a sketch of what a runtime action-governance layer has to provide mechanically to close the gap, and a pointer to one open-source implementation deployers can study or extend.
Why model-level audits fall short
Consider an agent with permissions to call `read_data`, `export_data` and `delete_data`. Each of those calls, in isolation, passes any reasonable pre-deployment safety evaluation. In sequence they are a textbook exfiltration pattern. No static risk label attached to the model catches this. Only runtime inspection of the action stream does.
Most current AI governance tooling treats the model as the unit of analysis. That is the right unit for classifiers that ingest a document and emit a label. It is the wrong unit for agents that call tools, invoke sub-agents, write files, transfer funds or modify infrastructure. The risk unit there is the action. A model with strong benchmark scores and thorough red-team coverage can still issue `rm -rf /` through a prompt injection or a misclassified instruction, because pre-deployment evaluation cannot enumerate the action space the agent will actually traverse in production.
Oversight also has to be effective, not merely present. A log file nobody reads, a dashboard that aggregates to monthly averages, a review queue that flags every event. None of those satisfies a plain reading of Article 14. The AI Act does not prescribe a technical architecture for oversight. But its obligations do constrain the shape of any architecture that can satisfy them.
What Article 14 implies, mechanically
Reading Articles 9, 12, 13 and 14 together as a deployer's checklist, a runtime governance layer has to carry a small number of properties. I set them out in the order a deployer would have to justify them to a Notified Body.
A per-action decision. For every consequential action an agent attempts, the system has to produce a decision (permitted, routed to human review, or denied) with a recorded reason. Aggregates do not satisfy Article 14's oversight obligations at the individual-action level.
A calibrated confidence rather than a point estimate. Risk scores are only useful to an overseer if they come with uncertainty. A 0.4 risk with a [0.35, 0.45] interval is very different from a 0.4 risk with a [0.15, 0.85] interval. The deployer needs the second number to decide how conservatively to route. Conformal prediction is one distribution-free way to produce it. Bootstrap intervals, Bayesian posteriors and quantile regression are others.
A logging artefact that supports record-keeping. Article 12 requires automatic logging of events relevant to risk. In practice this means append-only, tamper-evident records linking the action, the agent identity, the decision and the evidence the decision was based on. Hash-chained logs (SHA-256 linking) are the mechanically simple way to get tamper-evidence without a trusted third party.
An online adaptation path. Deployed systems drift. Tool inventories change, user behaviour shifts under adversarial probing. Article 9 requires ongoing risk management, so the governance layer's own calibration has to update as outcomes accumulate, or its risk estimates become miscalibrated. There are well-studied algorithms for this (multiplicative-weight updates for expert combination, FACI for adaptive conformal alpha). The specific mechanism matters less than the property: coverage guarantees that hold as the distribution shifts.
These properties are not exhaustive of Article 14. They are necessary. A tool that provides them produces evidence a deployer can assemble into a conformity case. It does not conduct the conformity assessment. That remains the deployer's and, where applicable, the Notified Body's responsibility. The distinction between *evidence* and *verdict* is where a lot of current AI governance tooling is quietly blurred.
Open questions
I would value pushback, particularly from members working on deployer-side conformance.
For agentic systems you have deployed or advised on, how are you evidencing continuous Article 14 oversight today? What is actually being logged, at what granularity, and what are Notified Bodies asking to see — per-action, per-session, or daily aggregates? This is the single largest uncertainty I keep hearing back from deployers.
Is anyone aware of harmonised standards work at CEN/CENELEC or ETSI that specifically addresses runtime action-level governance as distinct from model-level audits? My reading is that the current standards pipeline is still model-centric. I would be happy to be corrected.
And beyond Article 14 itself: the same runtime gate primitive looks applicable to DORA ICT incident response, MiFID II algorithmic trading oversight and on-chain risk scoring. Is there appetite in this group for a cross-regulation mapping exercise on that primitive?
I would rather this post serve as the start of a conversation than a broadcast. Corrections and counter-examples especially welcome.
- Συνδεθείτε για να αναρτήσετε σχόλια
Σχόλια
Σε απάντηση του A few responses. Design… από Henri Sirkkavaara
Your exchange ultimately converges on an important institutional reality: runtime oversight and design accountability are not competing governance models, but interdependent layers of the same regulatory architecture.
What the discussion also highlights is that the problem cannot be reduced to Article 14 alone.
The AI Act is increasingly becoming the visible focal point of European AI governance, but agentic systems already operate within a much broader legal and operational environment. Runtime supervision, traceability, security-by-design, data minimisation, incident reporting, accountability chains, and human oversight are not isolated obligations emerging from a single regulation. They are converging requirements distributed across multiple European frameworks: the AI Act, the Cyber Resilience Act, the General Data Protection Regulation, DORA, NIS2, sectoral financial rules, and future harmonised standards still under construction.
This matters because prompt injection and agent bypass research have already demonstrated a critical fact: no governance layer is permanently reliable in isolation.
Design-time controls can be circumvented.
Runtime controls can be overloaded.
Human supervisors can become desensitised.
Audit systems themselves can become opaque.
The consequence is not that governance is futile, but that resilience must emerge from layered accountability rather than from any presumed “perfect safeguard.”
That is where the discussion becomes constructive rather than polarised.
The existence of adversarial behaviour is precisely why Europe is moving toward cumulative governance structures instead of single-point compliance. The CRA reinforces secure development and lifecycle obligations. GDPR constrains unlawful processing and excessive data exposure. The AI Act introduces oversight, transparency, and risk-governance requirements. Together, these frameworks are imperfect, sometimes overlapping, occasionally ambiguous — but they represent an active institutional effort to progressively close the gap between technical capability and operational accountability.
And importantly, this ecosystem is still evolving.
Neither the standards bodies nor regulators are pretending the current framework is final. The present phase is one of iterative stabilization: guidance papers, harmonised standards, delegated acts, conformity practices, supervisory interpretation, and operational feedback loops are all still maturing in parallel with the technology itself.
That perspective is important when discussing runtime governance.
The objective should not be to construct an architecture of permanent suspicion where every agentic action is treated as inherently hostile. Nor should it be to rely exclusively on developer attestations and design-time assurances that cannot be empirically verified once systems encounter adversarial environments.
The more realistic institutional direction is proportional accountability:
- justified delegation of authority at design time,
- measurable oversight at runtime,
- traceable evidence after execution,
- and continuous adaptation as threat models evolve.
In practice, that resembles mature governance domains such as finance, aviation, or critical infrastructure more than either unrestricted autonomy or universal pre-clearance.
Ultimately, the strongest governance systems are rarely those claiming perfect control. They are the ones capable of remaining auditable, adaptable, and contestable under real-world conditions.
For agentic AI, that may become the defining distinction between procedural compliance and operational trust.
- Συνδεθείτε για να αναρτήσετε σχόλια
Σε απάντηση του Your argument on runtime… από remy wehrung
A few responses.
Design intent and runtime evidence are not substitutes. They are complements, and the original post is explicit about this. The "evidence vs verdict" distinction in the closing section draws the same line you are drawing: a runtime governance layer produces the evidence a deployer assembles into a conformity case. It does not replace the design-level justification, the conformity assessment, or the Notified Body's role. So the framing that runtime advocates treat the agent as "the primary locus of responsibility" misses the argument. Per-action evidence is what makes the upstream story (these permissions, granted for these reasons, under these governance assumptions) checkable by anyone other than the deployer. Without it, "ex ante justification" is a deployer attestation about the deployer.
On read_data, export_data, delete_data: agreed that the upstream question matters. But the upstream answer is unfalsifiable until you can see what the agent did under the granted permission set. Design accountability without runtime evidence is governance-by-affidavit.
The prompt-injection literature you cite is the strongest case for runtime oversight, not against it. The whole premise of injection research is that adversarial inputs subvert design-time intent. If agent behaviour could be fully bounded by upstream design decisions, injection would not be a category. The action layer is where the subversion manifests, and where it gets caught.
On opacity: you and the original post agree. The post explicitly points to an open-source implementation deployers can study or extend, for exactly the reason you give. The opacity critique lands on closed-source compliance vendors selling assurance through systems deployers cannot audit. It does not land on what the post argues for. The fix is to publish the supervisor, not to abandon supervision.
The banking analogy cuts the other way. Modern retail banks run real-time risk scoring on every transaction. They do not review keystrokes, they instrument actions and surface a small fraction for human review based on calibrated thresholds. That is the model the original post sketches: per-action decision with calibrated uncertainty, not universal pre-clearance. With conformal calibration the flag rate is a tunable parameter with statistical coverage guarantees. Closer to fraud detection than to a checkpoint regime.
"Turing police" is rhetoric, not an argument. Universal pre-clearance is a strawman of how runtime governance is implemented in practice. Score every action, route the tail to human review, preserve the evidence trail for audit. That is proportional, not paranoid.
Article 14 reads in both directions, and the original post argues this directly. A competent authority has to be able to ask "why was this delegated authority granted" and "what did the agent actually do under it." Both questions need answers. The first is a design-document question. The second is harder to fake without runtime evidence.
- Συνδεθείτε για να αναρτήσετε σχόλια
Σε απάντηση του Your argument on runtime… από remy wehrung
Thanks. The "interdependent layers" reading is the right resolution. The cumulative frame you sketched across CRA, GDPR, DORA, NIS2 and the AI Act is closer to how operational compliance actually gets assembled than any single-regulation reading.
Vaara (vaara.io, Apache 2.0) is the open-source implementation referenced in the introduction. It sits in one slice of that architecture: runtime evidence and online adaptation. It is not a design-accountability tool. CRA secure-development and GDPR processing-basis sit elsewhere. The claim is that it produces the runtime evidence other layers need to be checkable, on the assumption that upstream design accountability is documented in those other layers.
On the closing criterion. "Auditable, adaptable, and contestable under real-world conditions" is the constraint at the runtime layer too. Adaptability comes from conformal coverage, auditability from hash-chained logging, both open to inspection. Neither is a black box.
The design-time delegation question is upstream and stays upstream. Whoever granted the permissions has to justify them. Runtime evidence is what makes that justification checkable.
- Συνδεθείτε για να αναρτήσετε σχόλια
Your argument on runtime oversight is structurally strong, particularly in distinguishing model validation from action-level supervision. However, I would introduce an additional dimension that, in my view, sits even further upstream than either the model or the agent’s individual actions: developer intent and governance intent.
An agent does not originate purpose. It executes delegated intent within the boundaries designed by its developer, deployer, and operator. Focusing exclusively on per-action oversight risks treating the agent as the primary locus of responsibility, when in reality the first accountability layer is architectural intent: who defined the permissions, the escalation paths, the acceptable failure thresholds, and the economic incentives surrounding deployment.
For example, an agent capable of calling read_data, export_data, and delete_data is not inherently problematic because of runtime sequencing alone. The more fundamental question is why those permissions were granted together in the first place, under which governance assumptions, and with what ex ante justification. Runtime control is necessary, but it should not become a substitute for design accountability.
This becomes even more important given the now substantial literature on agent bypass and prompt-injection redirection. In just a few years, we have accumulated enough empirical evidence showing that safeguards can be manipulated, redirected, or socially engineered through adversarial interaction. If the governance model assumes that runtime supervision alone can contain this, it may be overestimating technical enforceability and underestimating adversarial creativity.
A second concern is the governance infrastructure itself.
If oversight tools, audit systems, and action-governance layers are treated as black boxes, we merely displace opacity from the model to the supervisory layer. An unauditable compliance mechanism creates regulatory theatre rather than trust. The same principle that justifies scrutiny of foundation models should apply to the tools judging them: governance mechanisms themselves should be transparent, inspectable, and preferably open source, especially where they support conformity claims under Article 14.
Otherwise, deployers may find themselves in the paradox of proving compliance through systems they are not allowed to verify.
There is also a proportionality question.
At what point does legitimate oversight become systemic overreach? If every action, every tool call, every inference path requires permanent supervisory escalation, we risk constructing not operational safety, but a form of computational pre-clearance architecture. Excessive governance can become indistinguishable from permanent suspicion.
Taken to its extreme, this logic resembles the idea of a permanent “Turing police”: an enforcement structure where the assumption is not controlled autonomy, but perpetual prior authorization. This may satisfy procedural caution, but it risks undermining innovation, operational efficiency, and the very utility that makes agentic systems valuable.
A useful comparison can be drawn with financial systems. We do not regulate banking by reviewing every keystroke of every employee in real time. We regulate through governance structures, accountability chains, auditability of critical actions, and ex post traceability where proportional intervention is justified. AI governance should probably follow the same institutional logic rather than defaulting to universal microscopic surveillance.
In short, runtime evidence is necessary, but it is not sufficient.
Article 14 should not be interpreted only as “observe every action,” but also as “justify every delegated authority.” The question is not only what the agent did, but why it was structurally allowed to do so.
Otherwise, we may end up building compliance architectures that are technically impressive, legally defensible, and strategically counterproductive.
Prudence is necessary. Institutional paranoia is not.
- Συνδεθείτε για να αναρτήσετε σχόλια