The basis for each check
The evidence behind every check.
The fair question, when somebody tells you their tool finds problems in your course, is: on what basis? This page answers it, check by check.
Before the checks: none of this research validates Learning Preflight. The literature describes how people learn, and where instruction and assessment fail. We designed checks around those failure modes. That translation is ours, and it is arguable. Every check below says what its evidence supports and, just as important, what it does not.
Six evidence-linked checks, and one expert judgment
A Preflight runs seven instructional tests. Six of them are below, and each is aimed at a failure mode that a recognised body of learning or assessment research describes.
The seventh is instruction quality: whether the explanation, the example, the feedback or the guidance is actually sufficient. It is a craft judgment. We report it because it matters, and it is not on this page because we do not present it as though a published finding validates our scoring of it.
01. The anchor: objectives
What we check. Whether the course states what a learner will be able to do, rather than what the course will cover, and whether every stated objective has instruction you can point to.
This is a precondition rather than a finding. Before anything can be aligned, the course needs a defensible statement of what learners are expected to be able to do. Without it there is nothing to align instruction to, and the five checks that follow have no denominator. When a course states no objectives, which happens more often than you would expect, we reverse-engineer them and say so in the report.
What this supportsObjectives are the reference point the other checks are measured against.
What it does not supportThat objectives we reverse-engineered are the ones the author intended.
02. Capability coverage at the required cognitive level
What we check. Whether content pitched at explain is being asked to satisfy an objective that requires apply.
Bloom's revised taxonomy and the frameworks that followed it draw the same distinction: recognition and recall are different operations from application and analysis, and instruction that supports one does not automatically support the other. A course can cover every topic on the list and still leave the learner unable to do the thing.
What this supportsThat the cognitive level of the instruction and the level named in the objective are worth comparing.
What it does not supportThat our reading of the level is correct, or that matching it predicts what a learner will be able to do.
03. Practice, with feedback
What we check. Whether every objective has at least one opportunity to do the thing, and whether the response explains why an answer was wrong rather than only that it was.
Research generally finds that retrieval practice improves retention relative to restudying, and that feedback can substantially improve the value of practice, particularly where it helps a learner understand why a response was incorrect. This is why our rubric does not count practice without explanatory feedback as practice at all.
What this supportsThat practice and explanatory feedback are relevant design conditions to check for.
What it does not supportThat our threshold for adequate practice is validated, or that meeting it predicts learner outcomes.
04. Assessment alignment
What we check. Whether each item maps to a stated objective at a matching cognitive level, and whether items give themselves away through implausible distractors or answers telegraphed in the stem.
Constructive alignment is the underlying idea: objectives, instruction and assessment have to be pointed at the same thing, or the assessment measures something other than what was taught.
What this supportsThat misalignment between an item and its objective is a defect worth reporting.
What it does not supportThat an aligned assessment is a psychometrically sound one. We do not run item analysis.
05. Unstated prerequisites
What we check. Knowledge the course requires but never teaches or signposts.
Unfamiliar prerequisite knowledge increases the processing demands placed on a learner and can make new instruction harder to integrate. Cognitive load theory is the usual frame for this. It is why unstated prerequisites are a critical issue in our rubric rather than a note.
What this supportsThat a term used but never defined is a real obstacle rather than a stylistic preference.
What it does not supportA specific claim about how much load any individual learner experiences.
06. Obtainable without demonstrating
What we check. Whether a learner can obtain the completion record without demonstrating the capability.
This is our hard gate, and it is the one check that is not primarily a research question. It is close to axiomatic: if two learners, one capable and one not, can produce the same passing record, the assessment cannot distinguish capability. A course whose assessment cannot separate them cannot produce a record that separates them either.
What this supportsThat the record does not depend on the capability. This is established from the package itself.
What it does not supportThat any particular learner took the shortcut, or how many did.
The checks are a system, not a menu
Presented as a list, these look like six independent things to fix, and you could reasonably pick two or three. That does not work, because each layer constrains the next:
- A good assessment cannot rescue an objective the course never taught.
- Practice cannot repair missing prerequisite knowledge.
- Retrieval practice does nothing when there is nothing yet to retrieve.
- More examples do not necessarily help an expert who needed edge cases rather than explanation. Worked examples can strongly support novices, while the same level of guidance may become redundant or inefficient for a more experienced learner.
The checks are sequenced because the failures are sequenced. That is why they are an order rather than a checklist.
What this page does not claim
That our checks are validated instruments. They are aimed at documented failure modes. We have not run a study comparing our findings to human reviewers or to learner outcomes, so we make no accuracy claim about the score and neither should anyone citing us.
That passing these checks means training works. It means specific, known defects are absent. That is a screen, not a measurement, which is why our verdict is "Cleared for Human Pilot" and never "proven effective."
See what these six checks find in your course
They are not abstract. Every run we publish shows them applied to a real package, with the evidence and the verdict. If you want to apply them yourself first, the self-audit is the same six questions asked of you rather than of your course.
Published runs · Score your own course · How Preflight works