Mechanical Verification: Why We Don't Trust the AI to Check Itself

When you use AI to generate educational content at scale, a natural instinct is to use the same AI to review its own output. We tried that. It doesn’t work. The same model that drifts on terminology will not notice it drifted. The same model that misapplies a Bloom verb will confidently assert the verb is correct.

Stay Learning is a course-authoring pipeline built entirely from Copilot agents: one designs the audience profile, another writes curriculum, another produces lesson plans, another writes the prose. Each agent has detailed instruction files specifying exactly what to produce. Those instructions help — but the first hand-written course we ran through the pipeline surfaced four structural defects that no instruction file caught.

So we wrote a validator. Not an agent. A Python script.

What goes wrong without mechanical checks

AI-generated courses drift in predictable ways:

Terminology inconsistency. Module 1 introduces “container image” and Module 2 starts saying “Docker image” without declaration. The glossary says one thing; the prose says another.
Outcome–content mismatch. An outcome says the learner will “compare” two approaches (Bloom: analyze), but the lesson only describes one.
Bloom level misapplication. Outcomes using “understand” or “appreciate” — words that sound like learning objectives but aren’t measurable and don’t belong to any Bloom level.
Broken cross-references. A lesson plan covers outcome m01-l02-o3, but that id doesn’t exist in outcomes.yaml.

Instruction files tell the agent what to do. They cannot verify, after the fact, that the agent did it.

The validator

validate.py is ~800 lines of deterministic Python. It loads every YAML artefact in a course directory, cross-references them, and emits PASS, WARN, or FAIL for each check. It runs automatically after every file write via a PostToolUse hook:

{
  "hooks": {
    "PostToolUse": [
      {
        "type": "command",
        "command": "./.github/hooks/validate-course-state.sh",
        "timeout": 30
      }
    ]
  }
}

No agent has the execute tool. The hook is shell-level infrastructure that agents cannot bypass or suppress.

What it checks

Structural integrity

Every lesson id must match m<NN>-l<NN> and be unique within the course. Every outcome id referenced by an assessment item or lesson plan must exist in outcomes.yaml. Every module referenced in a checkpoint must exist in course.yaml.

  FAIL  [id] formative id names the lesson owning m01-l02-o1 [m01-l03-a1]

This catches the common case where an agent copies a block from one lesson and forgets to update the id prefix.

Outcome coverage

Every outcome must be covered by at least one plan section. Every non-exempt section (introduction, summary, next_lesson are exempt) must map to at least one outcome. An outcome that nothing teaches is dead weight. A section that covers no outcome is filler.

Bloom verb enforcement

Outcomes must use verbs from a controlled vocabulary at the declared Bloom level. A set of banned words catches the most common mistakes:

FALLBACK_BANNED = {"understand", "know", "learn", "appreciate", "comprehend", "grasp"}

These are not measurable. You cannot observe someone “understanding.” You can observe them explaining, comparing, or implementing — and those map to specific Bloom levels. The validator rejects any outcome containing a banned verb before the rest of the pipeline ever sees it.

Assessment ceilings

An assessment item must sit at or below its outcome’s Bloom level, never above. If an outcome is at apply, a formative item cannot ask the learner to evaluate:

  FAIL  [asm] bloom 'evaluate' does not exceed outcome 'apply' [m01-l02-a1]

This prevents assessments from testing skills the course never taught.

Cognitive load budgets

Each module declares a cognitive load band during curriculum design: low, medium, or high. These map to term ceilings:

LOAD_BANDS = {"low": 6, "medium": 10, "high": 15}

The count is derived from lesson plans at validation time — it is never stored as a number that could drift from reality. If Module 1 is rated low but its plans introduce 9 new terms, it fails.

Rubric quality

Rubric levels must describe observable properties of the work, not the marker’s opinion. The validator flags vague quality words:

RUBRIC_VAGUE = re.compile(
    r"\b(good|well|excellent|poor|adequate|satisfactory|properly|correctly|"
    r"appropriately|effectively|thoroughly|sufficient|strong|weak)\b",
    re.I,
)

“Code is well-structured” fails because two markers will disagree on what “well-structured” means. “Code separates HTTP handling from business logic into distinct modules” passes because it’s verifiable.

Prose quality

The house voice lives in house-style.yaml, and most of it is specified by worked exemplars rather than rules — an adjective like “friendly” is re-interpreted every run, a passage is not. But the file also carries a countable floor, and that floor is enforced. Educational prose should not be patronising:

BANNED_PROSE = re.compile(r"\b(just|obviously|of course)\b", re.I)

“Just run the container” implies the learner should already know how. If they did, they wouldn’t be reading the lesson.

Sentence length is checked in the same pass, with code fences stripped first so a long line of shell doesn’t count against the prose:

  FAIL  [md] average sentence 21.4 words < 20 [m01-l02]

The cap is a 20-word average and a 30-word maximum. Emoji in educational content also fails — it adds no information and undermines technical register.

If you change the limits in house-style.yaml, change the validator in the same edit. Two files stating the same number is the classic place for a spec to drift away from what is actually enforced.

Diagram sanity

Mermaid diagrams are validated for:

Node count (max 12 — past this, automatic layout breaks down)
Label length (max 30 chars for nodes, 24 for edges)
Correct opening keyword for the declared diagram type (a flowchart that opens with sequenceDiagram fails)

Retrieval and spacing

A term introduced in Module 1 and never referenced again is a sign that either the term is unnecessary or the course missed a retrieval opportunity. A module after the first that reuses zero prior terminology fails — it means the course isn’t building on itself.

How it integrates

The severity model is simple:

Level	Meaning	Effect
`FAIL`	Structural violation	Blocks the agent immediately
`WARN`	Valid mid-write, wrong at rest	Informational during a run
`--strict`	Promotes `WARN` to `FAIL`	Used in CI or post-run validation

When the validator emits failures, the hook injects a system message telling the agent to fix them before continuing:

if [[ $status -ne 0 ]]; then
    # Inject failures as a system message the agent must address
    print(json.dumps({"decision": "continue", "systemMessage": message}))
fi

The agent cannot proceed past its current turn without resolving every FAIL. This is not a suggestion — it’s a gate.

Why this matters

Instruction files are necessary but insufficient. They tell an agent what to produce. They cannot verify the result against the rest of the course state. A lesson planner that perfectly follows its instructions can still produce an outcome id that doesn’t exist, a Bloom level that doesn’t match, or a term count that exceeds the budget — because those are cross-file invariants that only emerge from reading the whole course as a graph.

The validator makes the pipeline trustworthy without requiring a human to audit every structural decision. It runs in under a second, it has no false negatives by construction (it checks data, not intent), and it catches errors in the same turn they’re introduced — before they propagate.

If you’re building AI pipelines that produce structured artefacts, the lesson is straightforward: don’t ask the AI to verify its own output. Write deterministic code that does it instead. The AI generates; the machine verifies. That division of labour is what makes the system reliable.