The Progress Graph

At the end of each testing session, the Trainer displays your progress in a graph that looks something like this:

The graph shows a number of features, including:

  1. 'total' The total number of items in the vocab file.
  2. 'seen' The number of items that have been seen.
  3. 'known' The number of items with 'known' status (defined below).
  4. 'target' The current learning target (the number of items that should be known.
  5. 'secure' The approximate number of items in long-term memory, allowing for memory lapses.
  6. 'stale' The number of items due for revision.
  7. 'session counts' The total number of questions answered each session.

The Learning Journey

These terms make more sense if you consider the learning journey of an individual item. Each item starts off unseen. When first shown, it enters the learning pool - the size of this pool is dynamically adjusted each session, but often consists of about 20 items. Depending on how many other items are in the learning pool and the item's overall success rate, the item is usually shown fairly frequently, with revision intervals of a few minutes. It graduates to known when one of the following conditions is met:
  1. It has been answered correctly five times in a row.
  2. It has been answered correctly on at least 70% of ≥5 attempts.
  3. It has been answered correctly on at least 60% of ≥10 attempts.
  4. It has been answered correctly on at least 50% of ≥15 attempts.
These are fairly generous criteria, especially considering that half-marks are given for any question that is answered correctly with the help of a hint, or answered within 2-5 attempts, or for any item in which the only error was use of the wrong article (a gender error). Users wanting extra confidence in the 'known' status can select the checkbox 'Strict scoring', which adds the additional requirement that the most recent test on the item has to have been answered correctly. Note that, if an item achieves 'known' status with a low success rate, it will be scheduled for earlier revision.

Claimed items. For some easy items, and for items already known from other exposures to the target language, it is sometimes inefficient to wait for one of the 'known' criteria to be met. If you think you know an item at any stage, simply enter '@' as the answer, or click the 'Skip@' button. This is known as 'claiming' knowledge of an item, as compared to 'demonstrating' knowledge of an item. A claimed item is counted as known in every respect, and moves directly into the long-term revision schedule along with items that have earned their known status the hard way. At the moment an item is claimed, the Trainer asks for an indication of how well known the item is, and uses this to choose an appropriate revision interval. Subsequent revision intervals are based on actual performance with the item. Do not be shy about claiming items if you think you know them, because the scheduler is self-correcting. If the item is subsequently answered incorrectly, the 'known' status is usually revoked, unless the item has since passed one of the standard criteria. The item will not pass up the confirmation ladder unless it is repeatedly answered correctly, so it does not really matter how it initially achieved its known status. (Claiming is not possible when items come up for confirmation - this is purely a way of skipping the short-term schedule.)

Revision intervals. An item that has just graduated to known has a confirmation level of zero, and is usually scheduled for revision after a one-day interval. If it is answered successfully, it achieves confirmation-level one and is reviewed after a longer interval. Each successful revision advances it towards confirmation level ten, and each unsuccessful answer degrades the confirmation level by one - unless the 'known' criteria described above are no longer met, in which case it is taken out of the 'known' pool. (This is less punishing than many other schedulers, which puts every item back on the unknown pile.) The length of the review interval depends on the item's intrinsic difficulty, its history (success rate and confirmation level), and the memory model for the individual student. For each combination of confirmation level and difficulty, the Trainer tries to find a revision interval associated with a 90% success rate. For higher confirmation levels, this usually means a revision interval of many months.

Stale items. Items are considered 'stale' when they have passed their scheduled revision time without being answered. The staleness is usually expressed as an overdueness factor that describes, as a ratio, how long the item has waited for revision compared to the theoretical optimum. An item scheduled for revision at 5 days, for instance, is overdue by a factor of x2.0 at 10 days. Assuming that the item has not been reviewed for five days because the student was busy and did no revision at all, many other items might be five days late, as well. An item that was due for revision at 1 day might therefore be reviewed after 6 days, with an overdueness factor of x6.0. The Trainer automatically gives higher revision priority to items with higher overdueness factors. If the stale items are mounting up, it is recommended that students do a 'Confirmation session', consisting entirely of stale items.

Secure items. Not every stale item is forgotten, and not every forgotten item is stale. In fact, by definition, an item is considered stale when it has a 10% chance of being forgotten. The Trainer estimates the number of forgotten items by assuming that 90% of items which have just become stale (with an overdueness factor of ~1.0001) are still remembered, but a progressively lower proportion of items will be remembered after longer intervals, down to a worst-case scenario of 50% failure. It considers the number of stale items and the overall distribution of overdueness factors, and then calculates how many items are probably still known. This number is the 'secure' tally. The more items overdue for revision, the more the secure tally will fall below the known tally. On the other hand, the secure tally always equals the known tally if no items are stale. (That is, the Trainer gives full credit for items that are not yet stale, so as not to demoralise students who are following the revision schedule appropriately. In fact, an item that is nearly stale has an almost 10% chance of being forgotten, so the secure tally is an over-estimate.)

The Target. Students may set a learning target, such as 100 words per week, and this will be plotted on the progress graph in red. (The graph shown above is for a 9 year-old boy, learning German, closely following his target of 18 words in per week.) The difference between the current knowledge and the target is also displayed in the status bar at the bottom of the screen. The first figure after 'Target' is the known tally, relative to the target, and the second figure (in parentheses) is the secure tally, relative to target. It is recommended that students get the bracketed, target-relative, secure score to zero every day, if possible.

Zooming. The progress graph can be studied more closely by dragging the cursor across a rectangle of interest, or along the X or Y axis. Clicking in the X or Y margin will revert to the full view.