Chapter 5: AI Governance — Approval Regulation, Technical Levers, and the Coordination Problem

Why Governance Belongs in an AI Safety Series

Chapters 1–4 of this playlist were about what’s true inside an AI system — its capability, its objective, its internal cognition, and its security posture. None of those, on their own, decide whether a particular system gets built, who gets to run it, what evidence has to exist before it ships, or what happens when the evidence is bad.

Those decisions are governance. And governance is where the technical case for AI safety stops being a research program and starts being something the world actually has to do something about.

Two pieces of context to set this chapter:

1. Governance is multi-stakeholder by construction. No single lab, regulator, or country can implement the governance described below unilaterally — and the moment one tries, the value of unilateral action becomes a function of whether others follow. That’s the coordination problem, and it’s load-bearing.
2. There is a technical side to AI governance. Governance isn’t only law and policy. It also includes the technical instruments that make policy enforceable: evaluations, audits, compute monitoring, structured access, watermarking, model registries, incident reporting. Reuel et al.’s Open Problems in Technical AI Governance (2024) is the field map for this side.

This chapter is structured around three layers — the technical toolkit, the regulatory architecture (with a deep dive on Ezell’s approval-regulation proposal), and the coordination problem — and ends with what each implies for engineers, lab leadership, and policymakers.

The Three Layers of AI Governance

The layers are mutually dependent. Approval regulation without evaluations is a rubber stamp. Evaluations without an institutional consumer are an academic exercise. International coordination without enforceable domestic regimes is a press release. Each chapter section needs the others to be load-bearing.

Layer 1: Technical AI Governance — The Toolkit

Reuel et al.’s 2024 Open Problems in Technical AI Governance is the cleanest taxonomy of what’s actually needed to make any of this implementable. The central observation: almost every plausible AI governance regime depends on technical capabilities that don’t currently exist at the required level of reliability. Building those capabilities is its own research program, sitting between ML and policy.

The toolkit, organized:

Why “open problems” is the right framing. Reuel et al. don’t claim these tools exist and just need adopting. They claim each tool has technical research questions blocking its production use — and that without that research, governance regimes will either be unenforceable or rely on developers’ good faith. The technical AI governance program is precisely the work of closing those gaps.

The implication for an engineer reading this: a lot of the highest-leverage governance work is engineering, not law. Better evals, better watermarks, better attestation primitives, better incident schemas. The supply of these is a binding constraint on every regulatory regime that depends on them.

Layer 2: Regulatory Architecture — Five Live Models

The instruments above can be assembled into very different regulatory architectures. Roughly five families are active in 2026.

These aren’t mutually exclusive — most realistic regimes blend several. The interesting design question is which combination, and the answer depends on the maturity of the underlying technical instruments (Layer 1) and the political appetite for binding rules (Layer 3).

Deep Dive: Approval Regulation — Ezell’s Certified Safe

Carson Ezell’s Certified Safe: A Schematic for Approval Regulation of Frontier AI is the most fully-worked-out proposal in the family. It deserves a careful walk-through because the structure illuminates what governance has to be able to do, not just what it would forbid.

The core proposal: frontier AI systems above defined thresholds must obtain regulator approval before deployment, and in stronger versions before development past a certain point. The schematic specifies who, what, and how.

The pharma analogy isn’t decorative — it’s load-bearing. FDA approval works (imperfectly) because:

• The agency is technically credible.
• The submission package (the “NDA” in FDA-speak, or “safety case” in AI) is structured, auditable, and explicit about evidence and uncertainty.
• The regulatory decision is binding, and there are real consequences for circumvention.
• Approval is for a defined indication, not the molecule in general.
• Post-market surveillance is mandatory and consequential.

Ezell’s claim is that the same five properties are achievable for AI — and that without something close to them, the alternative regimes leave the most consequential decisions in the hands of the developers themselves. The strength of the proposal is that it forces governance to confront what evidence would be sufficient to deploy a frontier system — and that question, once asked seriously, exposes how much technical AI governance work still has to be done before any regulator could answer it.

What approval regulation requires that doesn’t yet exist

The honest reading of Certified Safe is that it’s a target, not a current capability. The preconditions are exactly the open problems Reuel et al. identified:

The honest summary of where approval regulation stands. It is the most coherent end-state proposal for the highest-stakes systems. It is also currently impossible to implement at full strength because the technical and institutional preconditions don’t exist yet. The realistic path is building those preconditions in parallel with bridging regimes — voluntary RSPs, AI Safety Institutes, sector-specific rules — that buy time to mature the toolkit.

Layer 3: The Coordination Problem

Domestic regulation has one fundamental limit: it doesn’t bind anyone outside the jurisdiction. For frontier AI, where the marginal cost of moving compute and talent across borders is non-trivial but finite, that limit is load-bearing.

The instruments for cross-border coordination on AI are still early-stage:

Why this matters for the technical layer. Almost every Layer 1 instrument is more useful if it works across borders — a safety eval whose results are mutually recognized, an incident database that aggregates across jurisdictions, a model registry with international scope, watermarking that’s recognized cross-platform. The technical work isn’t separate from the international coordination work; it’s the substrate that makes coordination implementable.

What This Implies for Practice

Common Confusions

At a Glance

Key Takeaways

• Three layers, mutually dependent. Technical instruments enable regulatory architecture; regulatory architecture is undercut by lack of international coordination; international coordination has nothing to enforce without domestic instruments. Move on all three or move on none.

• Technical AI governance is the rate-limiting step. Reliable evaluations, structured access, compute attestation, watermarking, model registries, incident reporting — every governance regime depends on these. Most don’t yet exist at the level of reliability needed. Building them is engineering work disguised as policy work.

• Voluntary RSPs are the current bridge regime. They’re imperfect — voluntary, self-reported, vulnerable to competitive pressure. They’re also the only thing currently load-bearing in the absence of mature approval regulation. Their credibility now shapes the rules later.

• Approval regulation is the most coherent end-state for high-stakes systems. Ezell’s Certified Safe schematic — pre-deployment safety case, independent review, conditional approval, post-market surveillance, real enforcement — generalizes a regime that’s worked imperfectly-but-consequentially in pharma. The preconditions don’t yet exist; the proposal is a target, with TAIG as the runway.

• Compute governance bites on a real chokepoint. High-end accelerators and the equipment to make them are concentrated, identifiable, and controllable in ways that data and weights aren’t. Export controls, training-run reporting, and chip-level attestation are the toolset.

• The coordination problem is real and partial cooperation is the realistic plan. Full multilateralism is a target, not a base case. Domestic regimes that survive partial international participation — backed by export controls, structured-access denials, and import restrictions — are what’s actually achievable in the medium term.

• Engineers are governance contributors. The TAIG agenda is not separate from ML research — it’s an applied subset of it. Better evals, better audits, better attestation primitives are policy infrastructure built in code.

• Borrow ruthlessly from older safety fields. Aviation, nuclear, and pharma all developed structured safety cases, mandatory incident reporting, regulator capacity, and international coordination over decades. AI doesn’t have decades. It does have the option to copy what worked rather than invent from scratch.

Further Reading

• Reuel, Bucknall, Casper, Fist, Soder et al., “Open Problems in Technical AI Governance” (2024) — the field map for the technical instruments side of governance.
• Ezell, “Certified Safe: A Schematic for Approval Regulation of Frontier AI” — the most fully-developed proposal for FDA-style pre-deployment AI regulation.
• Sastry, Heim, Belfield, Anderljung et al., “Computing Power and the Governance of Artificial Intelligence” (2024) — the canonical case for compute as a governance lever.
• Anthropic, “Responsible Scaling Policy”; OpenAI, “Preparedness Framework”; Google DeepMind, “Frontier Safety Framework” — the reference implementations of voluntary if-then commitments.
• EU AI Act (Regulation 2024/1689) — the most comprehensive horizontal framework, with general-purpose-AI specific obligations.
• UK AI Safety Institute and US AI Safety Institute publications — the early operational record of pre-deployment evaluation regimes.
• OECD AI Incident Monitor and MITRE ATLAS — the early infrastructure for cross-organization incident sharing.