What Makes a Good Course: Instructional Design as Code

Most AI-generated courses are bad. Not stylistically bad — structurally bad. They read fluently, cover the topic, and teach almost nothing. The failure mode is always the same: the model writes content first, then retrofits outcomes to match whatever it produced. The result is a course that describes a subject without ever committing to what the learner will be able to do afterward.

The Stay Learning pipeline takes a different approach. It encodes instructional design research as machine-enforced constraints — not suggestions in a system prompt, but validators that run after every write and block progress when the structure is wrong.

Here’s what that means in practice.

Backward Design: Destination Before Route

The core method is backward design, from Wiggins and McTighe. Work in this order:

Destination — what must the learner be able to do afterwards?
Evidence — how would you know they can do it?
Route — only now choose lessons, examples, and activities.

Most weak courses are built in reverse: content first, outcomes retrofitted. The symptom is a lesson whose outcomes could be swapped with another lesson’s and nobody would notice — because the outcomes don’t actually constrain anything.

The pipeline makes this reversal impossible. The stage dependency order is:

audience → curriculum → outcomes → assessment → lesson plan → content

Each stage refuses to run if its upstream artefact is missing. Assessment is designed before lesson planning on purpose: the evidence that proves an outcome is decided before the lesson that teaches it, not inferred afterward from whatever prose the model happened to produce. If you can’t turn an outcome’s evidence field into something a person can be handed, the outcome was never measurable, and the pipeline sends you back to fix it.

Bloom’s Taxonomy as a Machine-Enforced Constraint

Every learning outcome must start with exactly one observable verb from a machine-readable verb data file. The validator checks verb-to-level agreement automatically.

Banned verbs: understand, know, learn, appreciate, comprehend, grasp, realise, believe. These name internal states you cannot observe. An outcome beginning with “Understand how containers isolate processes” passes no test — you cannot hand a learner a task and watch them “understand.” Replace it with “Explain how namespace isolation restricts a container’s view of the host” and now you have something assessable.

The levelling rules matter too:

Bloom level must trend upward across a module. A later lesson sitting below an earlier one signals the sequence is wrong.
A course that never leaves remember and understand produces learners who recite and cannot act. A course that opens at evaluate produces learners who bluff.
Three to five outcomes per lesson. More than five usually means the lesson is two lessons.

This isn’t taxonomy as decoration on a slide deck. It’s a constraint the validator enforces on every outcome in outcomes.yaml, cross-referenced against the verbs file.

Cognitive Load Is Budgeted, Not Guessed

Each module carries a cognitive load rating — low, medium, or high — with a justification naming the load sources (new mental models, new terms, new tools, new syntax). The rating caps new terminology for the module:

Rating	Max new terms
Low	6
Medium	10
High	15

The validator enforces this cap against the lesson plans once they exist. Two consecutive high-load modules is a sequencing problem, and the pipeline flags it — separate them with a consolidation lesson or reorder.

Pacing is estimated bottom-up: section minutes summed upward, never a lesson total divided down (which hides overloaded sections). Practice needs at least as much time as theory. Total course time must land within 15% of the learner’s stated budget. If it doesn’t, the answer is to cut scope, not compress lessons.

Retrieval Practice and Spacing: Counted, Not Reviewed

Two findings from cognitive science, both countable:

Retrieval practice: a term the course never asks the learner to produce has been shown, not taught. Between exercises and quizzes, every glossary term must be asked for at or after the lesson that introduces it.

Spacing: the same term must reappear in a later lesson — not just in its own quiz (that’s recognition minutes after reading, not recall). A term introduced in lesson 3 needs to surface again in lesson 4 or beyond.

The pipeline checks both mechanically:

A term defined in the glossary and never asked for in any exercise or quiz → fails validation.
A module after the first that reuses none of the earlier modules’ terminology → fails. It’s a separate course sharing a folder.

These don’t need a “reviewer agent” making subjective judgements. They’re countable properties of the artefact graph.

Assessment Alignment

assessment.yaml carries four kinds of items: diagnostic, formative, checkpoint, and summative. The constraints:

At least one formative item per outcome. This is what makes an outcome a commitment rather than a claim.
No item above its outcome’s Bloom level. You can assess at or below — a recall check before an analyze item confirms the learner has the parts. Assessing above is a broken contract: the course promised one thing and tested another.
Diagnostics are tied to the audience profile. Each diagnostic item probes one entry from common_misconceptions or prerequisites.assumed in audience.yaml, quoted verbatim so the link can’t drift.
One checkpoint per module, drawing on that module’s outcomes, preferring items that require two lessons combined.

The key insight: assessment is written before content because an assessment written after prose can only test what the prose happened to cover. It can never reveal that the lesson missed its outcome — the two agree by construction.

Example Quality: Typed and Validated

Examples are where AI courses fail hardest. The model produces a code block that restates the definition, calls it a “demonstration,” and moves on. The pipeline prevents this with five typed example kinds, each carrying distinct required fields:

Kind	Shows	Required fields
`worked`	Complete solution with reasoning for each step	`steps` (≥ 3)
`completion`	Partial solution the learner finishes	`steps` (≥ 3), `learner_completes`
`contrasting`	Cases differing in one dimension	`cases` (≥ 2), `varies`
`non_example`	A case that looks right but fails the definition	`why_not`
`analogy`	Mapping onto prior knowledge	`maps_to`, `breaks_down_at`

A kind cannot carry another kind’s fields. A worked example with one step is “a definition with a colon,” not a procedure. An analogy as the only example in a lesson has explained what the thing is like and never shown the thing.

Any outcome at apply or above requires at least one worked or completion example — the kinds that actually show how the thing is done.

Voice Is Pinned by Example, Not by Adjective

Structure can be counted. Voice mostly cannot, which is exactly why it needs its own artefact. house-style.yaml is read by the four agents that write anything a learner sees — lesson writer, exercise generator, quiz generator, project designer. There is one voice for every course and all four surfaces, so a learner moving from a lesson to its exercises does not change author halfway through a session.

The interesting part is how it specifies that voice. Telling a model to be “friendly and clear” achieves nothing reproducible: an adjective is re-interpreted from scratch on every run. A worked passage is not. So each surface carries a matched pair — one passage in the target voice, one in a voice to avoid, on the same subject, with a why_not explaining the difference.

The counter-exemplar for lesson prose defines six drivetrain components before saying what any of them is for. Its why_not is precise about the failure:

It defines six parts before saying what any of them is for, so the reader carries the nouns with nowhere to put them. It never states the point — that gearing trades force against distance — so the final instruction arrives as an assertion rather than a conclusion the reader has already reached.

That is a far tighter constraint than “write clearly,” and it survives being read by a different model next month.

Underneath the exemplars sits a countable floor, and that part is enforced. Sentences average under 20 words and none exceeds 30. Paragraphs run one to five sentences — a one-sentence paragraph is a device, not a defect. Three words are banned outright:

banned_words:
  - just
  - obviously
  - of course

Each one tells the reader their difficulty is not real. “Just run the container” implies they should already know how; if they did, they would not be reading the lesson.

Note what the limits deliberately exclude. They apply to lesson_prose and exercise_task only — not to rubric levels or the why_wrong on a quiz option. Those are single dense judgements, and holding them to a 20-word average would buy brevity by trading away the observable detail that makes a rubric level markable at all. A style rule applied everywhere is a style rule that damages something.

Why Enforcement Matters

These aren’t suggestions in a system prompt that the model can acknowledge and ignore. They’re checked by a validator that runs after every file write via a post-tool-use hook. The validator returns FAIL (blocks progress) or WARN (valid mid-procedure, wrong once the run stops). --strict mode turns warnings into failures for CI.

The practical effect: an AI that “understands” backward design but skips it gets stopped at the first write. An outcome with a banned verb never reaches the lesson planner. A module that blows its terminology budget can’t produce content until something is cut.

The design philosophy is simple: anything you can count, count. Anything you can validate, validate. Where a property resists counting — whether an analogy is apt, whether an explanation lands — pin it with a worked exemplar instead of an adjective, and validate the floor beneath it. Everything else is structure, and structure is checkable.

A good course isn’t one where someone intended to apply instructional design. It’s one where the structure makes it impossible not to.