The rubric
How a Preflight is scored.
Published in full, here and inside every report. You should know how the number was reached before you are asked to act on it, and you should be able to argue with it.
The seven dimensions
Objective alignment
Percentage of stated objectives with traceable supporting instruction, weighted by objective criticality.
Capability coverage
Percentage of derived KSATs addressed at the required cognitive level. A KSAT requiring “apply” is not satisfied by content that only explains.
Practice adequacy
Percentage of objectives with at least one aligned practice opportunity that provides feedback. Practice without feedback does not count.
Assessment alignment
Percentage of assessment items mapping to a stated objective at a matching cognitive level.
Synthetic learner performance
Median across the three instructional profiles, reported with the spread, and with the minimum-effort learner called out separately. Access and readability is scored below rather than blended in, because mixing “couldn’t reach the button” with “didn’t learn the concept” produces a figure that means nothing.
Access and readability
Reported separately and split. Operability, from the keyboard-only and touch-only profiles. Readability, from the second-language reader. Both are screens, not a conformance audit.
Critical issues
A count, not a percentage. Exactly three conditions qualify: a learner can get credit without demonstrating the capability; a prerequisite is required but never taught; an assessment item tests something never covered. Access and readability findings are reported and scored but are not critical issues, and do not count toward the guarantee.
The cut lines
Cleared for Human Pilot
All dimensions at 80 or above, and zero critical issues.
Ready with Remediation
No dimension below 60, three or fewer critical issues, and an accepted plan.
Not Ready for Deployment
Any dimension below 60, or four or more critical issues, or the minimum-effort learner passes the assessment. That last clause is absolute.
The top verdict is Cleared for Human Pilot rather than “passed” or “proven effective”, and the wording is deliberate. It means the course is ready to put in front of real learners with evidence behind it. It does not mean the course has been shown to work.
What repetition does and does not control for
Each profile runs three times and we report the median. Where the three runs differ by more than 15 points on any dimension, we mark it low confidence, needs human review rather than reporting a clean number.
Three runs surface instability. They do not surface systematic error: a model that is confidently wrong will be wrong the same way three times and score as high confidence. That is a real limit of this method, it is why every critical issue is verified against the package by hand rather than accepted from a run, and it is why we have not made an accuracy claim about the score.
What we don’t claim
The score is a screen, not a measurement
Learning Preflight is a structured review of instructional design. It identifies defects, missing practice, misaligned assessment, untaught prerequisites, items that give themselves away. It does not measure learning effectiveness, and the numbers in the report are not a prediction of how real learners will perform.
We have not run a validation study comparing our findings to human reviewers or to learner outcomes. Until we have, we will not make an accuracy claim about the score, and you should treat any vendor who does without one with suspicion.
Accessibility: what we do and don’t do
We screen for operability failures, and we score them. Two of our six profiles are a keyboard-only user and a touch-only mobile learner. Those findings get their own reported dimension rather than disappearing into an average. On a public-sector course we published, this surfaced a live WCAG 2.2 SC 2.5.8 target-size failure.
We do not perform a conformance audit. We do not test with real assistive technology, we do not issue a VPAT or ACR, and we do not evaluate every success criterion. Simulated learners cannot substitute for a screen-reader user.
So: a Preflight can tell you a course has an accessibility problem. It can never tell you a course is accessible. If conformance is your binding requirement, commission a specialist audit.
← Back to Learning Preflight · See the rubric applied to a real course