For LMS administrators
What is the course actually telling your LMS?
A course can launch, track and report completion perfectly while the package underneath it is doing something very different from what everyone assumes. We have published examples. This is the evidence you can take upstream.
The position you’re in
You see the defect. You don’t own the budget.
Every one of the findings below came out of a package we examined, not out of a theory. You will recognize all of them, because they arrive in your inbox every month attached to a course somebody has already approved.
The package reports nothing at allNo score, no status, no interactions. The only file containing a reporting call is the generic API wrapper. Completion is issued for reaching the last page.
The answer key is in the page sourceFifteen questions, fifteen correct answers declared in client-side JavaScript, unlimited retries, and the code that would re-lock the Next button commented out.
A real quiz that isn't registered for reportingFive scenario questions, an author-set 80% pass mark, and an export configured for completed/incomplete. Scoring zero and not attempting produce the same record.
Success criterion set to "view"A pass mark of 8 out of 10 on the graded question, and both completion and success criteria set to 75% of slides viewed.
Fixed canvas, no viewport declaration1024×768, responsive: false, no media queries. It renders. It tracks. On a phone-sized viewport it becomes effectively unusable.
Every run is published in full, with the reasoning, the evidence and a ranked remediation list. They are written to be forwarded.
- Case A: full report · the one where nothing is reported at all
- Case B, full report · good course, quiz not wired to reporting
- Case C, full report · view-based success criterion
Why nobody upstream sees it
None of this is visible from the authoring tool.
This is the part worth putting in front of whoever declined your last request. In an authoring preview the course works. The quiz runs, the feedback appears, the pass mark sits in a settings panel looking authoritative. The defect exists only in the exported package, which is often the least-inspected artifact in the process, and the one your learners actually receive.
In two of our published runs, somebody deliberately set a pass mark and the export configuration silently overrode it. Different tools, different organizations, five years apart. Neither author did anything careless. Both courses shipped, and both reported a number that meant less than everyone assumed.
What the author sees
- Quiz works
- 80% pass mark
- Feedback works
- Preview runs
What the package says
- Completion = view
- Success = view
- Quiz not reported
What the organization sees
- Completed
Which is also why “our completion rate is 94%” is not a rebuttal
A high completion rate is consistent with a meaningful assessment and with no meaningful assessment at all. Completion rate alone cannot distinguish them. Nothing in the LMS distinguishes them. The distinction lives in the package.
What we can check
Conformance and reporting, which is the part you care about.
Most of our work is instructional. This part is not. It is a standards question, and it is the one we are best qualified to answer.
Standards work led by Brian Caudill, founder
- In the working groups when SCORM was being written
- One of only nine certified SCORM auditors in the world
- Built and sold JCA Solutions to Rustici Software
- Twenty-plus years across SCORM, AICC, xAPI and cmi5
- What the package actually reports, score, status, interactions, objectives, suspend data. Against SCORM 1.2, 2004, AICC, cmi5 or xAPI.
- What triggers completion and success, and whether those two are the same condition when they should not be.
- Whether the assessment result reaches your LRS in a form you can query, rather than a completion verb and nothing else.
- Whether the package survives your environment. A course that works in one LMS and silently stops reporting in another is a packaging fault, not a mystery.
For one course this is part of a Learning Preflight. For a library, it is the first stage of a Portfolio Intelligence engagement.
Free, and you can send it today
The trapped library study applied these rules to 73 packages from seven public-sector publishers, with the predictions and the falsification conditions published before a single package was opened. It is a citable number rather than an opinion, which is usually the better opening for the conversation you are about to have internally.
Making the case internally
Need to make the case internally?
- Start with one package. Choose an important mandatory course and establish exactly what it reports, what triggers completion, and whether anything evaluates the pass mark.
- Bring evidence, not suspicion. Package configuration, reporting behaviour and completion criteria are all things somebody can escalate. “I think something is wrong” is not.
- Use one of our published reports as the example. They carry no sales content and state their own limits on the first screen, and they are written so a non-technical stakeholder can follow the chain from package configuration to completion record.
- If the course is mandatory, the question has another owner. What the completion record is based on is the same finding asked from the compliance side.
Send us the package you already suspect.
We will tell you what it reports, what triggers completion, and whether the required completion record depends on demonstrating the capability.
Written for the person who opens the package · every finding cited here is from a published run · hello@actyra.com