Trapped library study
Pre-registration
Published before we opened a single package. The point of writing predictions down first is that it makes the result capable of embarrassing us.
What we predicted
- A substantial minority of packages will ship with no editable source. We expected somewhere between a quarter and a half.
- SCORM 1.2 will dominate, despite being superseded for two decades.
- Dead runtimes will cluster by publisher and era rather than being evenly distributed.
- Tracking will mostly work. We expected the failure to be in recoverability, not in the packaging standard itself.
What would have proved us wrong
- Fewer than 15% of packages lacking editable source would have falsified the central claim.
- An even spread of dead runtimes across publishers would have falsified prediction 3.
- Widespread tracking failure would have falsified prediction 4 and meant we were measuring something else entirely, badly built packages rather than trapped ones.
What we found
Predictions 1, 2 and 4 held. Prediction 3 held, and more sharply than expected, dead runtimes were almost entirely concentrated in a single publisher's older material.
The headline figure landed at 46.6%, at the top of our predicted range.
Why this is published
A number you cannot check is not a finding. Writing the predictions and the falsification conditions first is what separates a study from a marketing statistic, and it is why we will keep publishing them even when the result is less convenient than this one was.