The Two Boundaries: Why Behavioral AI Governance Fails Structurally
Source: https://www.semanticscholar.org/paper/b32c7671f795085a9430ed9a4c066d72b1636a6e ↗
Full text: open-access via OpenAlex ↗
McCann's central move is elegant and underexploited: Rice's theorem (1953) already proves that no behavioral layer added on top of a Turing-complete system can ever fully govern its effects — the gap between what a system can do and what governance covers is undecidable in the general case.
This reframes 'AI governance' from a policy problem into an architectural one, where safety theater is not an implementation failure but a structural inevitability when expressiveness and governance boundaries are defined independently.
The proposed solution — 'coterminous governance,' achieved by separating computation from effect at design time rather than layering oversight afterward — translates a theoretical result into a testable architectural criterion with mechanized proofs.
For product directors, this is the argument that explains why every post-hoc content filter, every bolt-on moderation layer, and every compliance dashboard is fighting a battle that formal logic guarantees it will lose at the edges.
The citation count is low and the work is new, but the abstract delivers a genuine analytical architecture grounded in a classical theorem rather than a policy framework, which clears the bar the library sets for AI governance entries.
Read alongside Peterson on compliance design and Falk/Tsoukalas on the organizational consequences of AI deployment.
Central argument
McCann argues that AI effect governance—controlling what actions an AI system performs in the world, not just what it outputs—fails structurally because the expressiveness boundary (what the system can do) and the governance boundary (what policies cover) are defined independently and inevitably diverge. This divergence produces two failure regions: ungoverned capabilities that create real risk, and governance theater covering capabilities that don't exist. He proves via Rice's theorem that this gap is undecidable for any Turing-complete system using behavioral governance, and proposes 'coterminous governance'—where the two boundaries are made identical by architectural design, specifically by separating computation from effect execution—as the only structurally sound alternative.
Critique
The paper's most significant blind spot is the gap between architectural prescription and organizational reality: McCann acknowledges that structural governance says nothing about whether the policy itself is correct, but the coterminous governance model implicitly assumes that the full expressiveness boundary of an AI system can be known and formalized at design time—an assumption that becomes increasingly strained as agentic systems acquire tools dynamically or interact with third-party APIs whose semantics evolve. The framework may prove that behavioral governance is undecidable in the limit, but it does not demonstrate that coterminous governance is achievable at scale in production systems rather than in constrained research architectures.
Why it matters for product
For a CPO, the ungoverned capability region is not an abstract security concern but a direct product liability and trust problem: every new tool or integration added to an AI-powered product expands the expressiveness boundary in ways that existing governance policies—compliance checklists, content filters, monitoring dashboards—will not automatically cover, meaning product decisions about capability expansion carry compounding governance debt. McCann's framework also has organizational design implications: the separation of computation from effect that coterminous governance requires cannot be owned by a security or compliance team bolted on post-delivery; it must be an architectural constraint embedded in how product and engineering define what an AI agent is allowed to express, making governance a first-order input to platform and API design decisions, not a review gate at the end.