Learning Readiness Report™ · Case B · 16 August 2026
Case B, staff cyber-security awareness
The full report behind the published case study. This is the highest-scoring course we have published a run against, and it still fails: on one setting, in the packaging, with everything upstream of it done well.
7 lessons · 7 knowledge checks · 5 quiz items · publisher blinded.
This is a static analysis run, not an instrumented playback
Every finding comes from inspecting the package. The six learner profiles were applied to that content analytically. Each profile is a defined set of failure conditions, and we checked the course against them.
We did not drive this course in a browser and we are not reporting timings. There are no session durations, no click paths and no dwell times in this report, because we did not measure any. A client engagement adds an instrumented pass. This published run does not have one.
Verdict
Not Ready for Deployment
Triggered by the hard gate, and by nothing else. A learner is recorded as complete for viewing the content, whether or not they attempt the quiz and regardless of how they score on it.
This verdict is doing something unusual here, and we want to be plain about it: the teaching in this course is good. Six of the seven dimensions score above anything else we have published. One export setting is the entire difference between this verdict and Cleared for Human Pilot.
Scorecard
The introduction tells the learner what the course will cover, four named topics, which is a contents list, not an objective. No statement anywhere describes what a learner will be able to do afterwards. We reverse-engineered four objectives, and unusually, all four are well supported by instruction. The score reflects that they are unstated rather than unsupported.
The target capability is recognizing a situation and choosing the right response, and the course teaches at exactly that level. Each topic is built around a named character facing a decision, and the learner is asked to make it. This is the highest capability-coverage score we have published.
Seven scenario knowledge checks are embedded in the lessons, not saved for the end. Every one carries explanatory feedback on both branches, and the correct and incorrect feedback are near-identical by design, so a learner who guesses right still receives the reasoning. That is a deliberate choice and a good one. Marked down only because one topic has no practice at all.
The five quiz items are scenario-based and correctly keyed. The problem is distribution. The one lesson with zero knowledge checks supplies two of the five final questions, 40% of the assessment tests the only topic the learner never got to practice. Meanwhile the topic with the most practice is barely represented.
Median across the three instructional profiles, spread 61 to 88. All three completed. The minimum-effort learner is reported separately below and is the reason for the verdict.
The best result on this dimension we have recorded. Keyboard and screen-reader operating instructions are written into every question stem rather than left to the player. The introduction carries a dedicated screen-reader section. The video ships with a caption file. Plain language throughout, short sentences, no unexplained jargon.
One of the three qualifying conditions is met. It is enough.
Step one, the objectives, reverse-engineered
The course names four topics. We derived one capability objective per topic and mapped each against instruction, practice and assessment.
| Derived objective | Instruction | Practice | Quiz items | Balance |
|---|---|---|---|---|
| 1. Choose and store passwords so a compromise of one account does not spread | Strong | 3 checks | 1 | Good |
| 2. Respond correctly to update prompts and software errors on a work device | Strong | 2 checks | 2 | Good |
| 3. Recognize a phishing attempt and reduce personal exposure to one | Strong | 2 checks | 0 | Practiced, not assessed |
| 4. Recognize a reportable incident and report it promptly | Present | None | 2 | Assessed, not practiced |
The inversion in rows 3 and 4
Phishing gets two scenario rehearsals and no direct quiz item. Incident reporting gets no rehearsal and two quiz items, and reporting is the topic where hesitation does the most damage, because the value of a report decays by the hour.
This is not a scoring artifact. It is the single most useful instructional finding in this run, and it is fixable by moving one existing knowledge-check pattern into the reporting lesson.
The panel
- No undefined jargon. Where a technical term is unavoidable the course names concrete examples in the feedback rather than assuming familiarity.
- Every topic opens with a named person in an ordinary situation, which gives a learner with no background something to reason from instead of a rule to memorize.
- One gap: the reporting lesson tells the learner to report but never rehearses recognizing what counts. A cold-start learner leaves knowing they should report and not reliably knowing when.
Traced to: lessons 2–5, knowledge check feedback
- No pre-test and no way to skip what is already known. An experienced learner walks the full linear path to reach a five-question quiz whose result is not recorded anyway.
- The password guidance is current and correct, it endorses browser-saved passwords and password managers rather than repeating outdated rotation advice. A practitioner notices this and it buys credibility.
- Nothing distinguishes this year's version from last year's. For annually mandated awareness training that is the difference between a refresher and a re-run.
Traced to: navigation settings, lesson 2 feedback
- Built on a responsive authoring platform with a fluid layout. No fixed canvas, no horizontal panning, no pinch-zoom required.
- Interactions are tap targets rather than hover states.
- One inert defect: the page calls
window.resizeToat load, labeled in the source as a resize hack. Browsers have blocked this for years, so it does nothing, but it is the kind of thing that ages into an unexplained console error.
Traced to: package entry point, platform layout engine
- Short sentences, active voice, concrete nouns. The scenarios carry the meaning, which survives translation far better than a rule stated abstractly.
- Feedback restates the reasoning in full rather than referring back to something read earlier, so a reader who missed a nuance the first time gets a second pass at it.
- Minor: a small number of idiomatic constructions appear in the scenario framing. None sits on a decision point, so nothing is load-bearing.
Traced to: all lesson text and feedback blocks
This is the strongest accessibility result across every run we have published, and it is the result of a decision rather than a default.
- Operating instructions are inside the question text itself. Every item carries them. It does not depend on the learner finding a help screen.
- The introduction includes a dedicated screen-reader section documenting the keyboard controls for the whole module.
- The video ships with a matching caption file, present, not assumed.
Traced to: every quiz and knowledge-check stem, introduction, media folder
This learner succeeded, which fails the course. No ingenuity was required; the configuration grants the outcome.
- Completion is content viewed, not passed. The package reports completed or incomplete on the basis of reaching 100% of the content.
- The quiz can be skipped entirely. Advancing does not require attempting it.
- Scoring zero and not attempting produce an identical record, because no quiz identifier is registered for reporting. The author's 80% pass mark is never applied to anything.
Traced to: runtime configuration block, quiz settings
Critical issues, one of three conditions met
01 · A learner can get credit without demonstrating the capability
Completion is set to content viewed. The quiz is not required to advance, its identifier is not registered for reporting, and its 80% pass mark is therefore never evaluated. A learner who scrolls to the end is recorded exactly as one who scored full marks.
02 · Prerequisite required but never taught, not found
Checked and cleared. No specialist term carries reasoning without being explained in place.
03 · Assessment item tests something never covered, not found
Checked and cleared. Every quiz item maps to a topic the course teaches. Two of them map to a topic it teaches without rehearsing, which is an alignment weakness rather than a critical issue.
Remediation, ranked by what unblocks the most
This is the shortest remediation list we have produced, and the first item is close to trivial.
- Register the quiz for reporting and set completion to passedOne export configuration change. Register the quiz identifier so its score is transmitted, and set the completion criterion to the existing 80% pass mark rather than to content viewed. This closes the only critical issue and lifts the verdict on its own.Changes the verdict: one setting
- Require the quiz before completionCurrently a learner can advance past it. Related to the above and worth doing in the same pass.Same setting screen
- Add a knowledge check to the reporting lessonIt is the only topic with no rehearsal and it supplies 40% of the final quiz. The other four lessons contain seven worked examples of exactly the pattern needed: a named person, an ambiguous situation, three plausible responses.Half a day
- Add a direct phishing-recognition item to the quizThe most rehearsed topic is the least assessed. One item rebalances it.Low
- State the objectivesConvert the four topics into four capability statements. The course already teaches to them; it just never says so, and a learner who knows what they are meant to be able to do attends differently.Low
- Add a revision marker for repeat learnersFor annual training, tell a returning learner what changed. Removes the strongest reason an experienced learner disengages.Low
- Remove the inert resize callHousekeeping.Minutes
What not to change
The scenario structure, the dual-branch feedback, and the accessibility guidance written into the question stems. All three are better than commercial norms and all three would be easy to lose in a redesign. If this course is rebuilt on a different platform, these are the properties to carry across.
What this report does not establish
It is a structured risk screen, not a measurement of learning effectiveness. It does not predict how real learners will perform, and no score here is evidence that the training does or does not work.
We tested the package, not the deployment. An organization can layer its own completion rules on top of a package that reports completion only. The quiz result is not among the things the package gives them to work with.
The access and readability dimension is a screen, not a conformance audit. A score of 86 means our profiles found no obstruction. It is not a conformance result.
The derived objectives are our inference. The course states topics, not objectives.
If you think this might be your course
Email hello@actyra.com with the subject “Is this our course?” and we will tell you, privately, within two business days. If it is yours you get the identity, the package hash and the complete working notes at no charge, and no follow-up unless you ask for one.
That offer stands for any public-sector publisher whether or not one of these runs is theirs. If you want your own course checked against the same three critical-issue conditions, ask and we will do one free, for a course of up to 3 hours.
The published case study for this run · All published runs · How Preflight works