Ask separate follow-up questions about remembering the rule and using it at work. A delayed knowledge check can show what someone recalls under the check’s conditions. Evidence from a real task can show whether the person applied the rule when an opportunity arose. Neither should be used as a substitute for the other.
Start by deciding which question matters to the project. If employees must recall a critical step without a reference, test that condition appropriately. If they normally use an approved guide, observe whether they can find and apply it. A memory test can be a poor proxy for a reference-supported job.
Define the two outcomes before collecting data
For a fictional internal routing course, recall means explaining which request features determine the destination. Workplace use means applying that rule to an eligible request using normal tools, information and permissions.
The CDC’s guidance on evaluating training effectiveness distinguishes learning from learning transfer and recommends delayed evaluation after learners have had an opportunity to apply what they learned. Timing should fit the task and context. The plan below is an original example, not a universally validated schedule.
Do not assume that every employee will encounter the same number or type of cases. Opportunity is part of the interpretation. Someone who has not received a relevant request has not yet supplied evidence of workplace use, whether their quiz score is high or low.
Build a follow-up plan around the work cycle
Suppose routine requests usually occur during an employee’s normal workweek. You might plan a short delayed check after a week and an observation at the first suitable request after training. Those timings are illustrative, not evidence-based cutoffs. If the task occurs monthly, a one-week transfer check may be premature.
Collect the minimum evidence needed to answer the question. A brief scenario can check the decision rule. A de-identified work record or structured observation can check application. A short question about barriers can explain why an opportunity did not lead to use.
Illustrative follow-up plan. The task, timings and examples are fictional and must be adapted locally.
Evaluation question: Can the employee explain the rule later?
Timing tied to opportunity: At a planned delay after instruction
Evidence to collect: Short new case with an explanation of the deciding condition
Interpretation limit: Shows performance on that check, not actual workplace use
Evaluation question: Can the employee use the rule at work?
Timing tied to opportunity: First suitable eligible case after training
Evidence to collect: Observation or authorized de-identified work record
Interpretation limit: Depends on case difficulty, available support and resources
Evaluation question: Was there a chance to use it?
Timing tied to opportunity: At each follow-up
Evidence to collect: Whether an eligible case occurred and how many were reviewed
Interpretation limit: No opportunity is different from nonuse
Evaluation question: What helped or blocked use?
Timing tied to opportunity: After an attempt or a missed opportunity
Evidence to collect: Brief employee account plus checks of relevant conditions
Interpretation limit: Self-report may be incomplete and should not be treated as direct observation
Evaluation question: Is performance sustained?
Timing tied to opportunity: Later suitable cases across relevant conditions
Evidence to collect: Repeated bounded checks of the same criteria
Interpretation limit: Repetition strengthens the description but does not by itself establish causation
Use different prompts for recall and application
For the delayed check, give a new fictional request and ask: “Which condition determines the queue, and why?” If unaided recall is the target, state that the guide is not available during this check. If guide use is part of the skill, allow the current guide and describe the result accordingly.
For workplace use, ask: “What did the employee do on this eligible case?” Record the selected queue, the deciding information, any prompt supplied and whether the required resources were available. The evidence should describe an action, not only an intention to act.
A learner’s report that they plan to use the rule is useful for planning support, but it is not proof of later application. Similarly, a manager’s general impression that performance is better is weaker than a record tied to specified cases and criteria.
Read combinations of evidence carefully
A strong recall response with no workplace use may indicate missing opportunity, a tool barrier, conflicting expectations or a gap between the practice case and actual work. Investigate before assigning more training. A person can remember a rule and still be unable to act under the available conditions.
Successful workplace use with a weak unaided recall response may mean that the approved guide is supporting performance effectively. Whether that is a problem depends on the job requirement. Do not penalize reference use when reference use is the intended workflow.
Weak evidence on both measures may justify a closer look at prerequisites, instruction and working conditions. It still does not identify the cause automatically. Begin with a safe, representative case and ask the employee to explain the decision with appropriate support.
Avoid attributing every change to the course
Compare with relevant pretraining evidence when it is available and appropriate. If you only collect a posttraining measure, you can describe what people did afterward, but you cannot show how much they changed from their starting point. Even a before-and-after improvement can have other explanations, such as a revised form, a new manager prompt or a change in case mix.
Record important concurrent changes. If the organization repairs permissions and clarifies the routing guide at the same time as training, report the combined conditions. Do not present the resulting improvement as the isolated effect of the course without a design that supports that inference.
Avoid filling gaps with unsupported percentages. A small set of observed cases can support a practical next decision without becoming an organization-wide impact claim. State the number and type of cases if you report a rate, and keep the denominator clear.
Return the findings as actions
Organize the result around what should happen next. A knowledge gap may call for a short refresher and practice. An access barrier needs an operational owner. A lack of opportunity may require a later check or a suitable simulation. A secure routine performance may justify observing a relevant variation rather than repeating the same exercise.
Keep individual records private and proportionate to the agreed evaluation purpose. Use approved systems and avoid collecting sensitive work details when a de-identified description is enough.
For your next follow-up, write two questions on separate lines: “What do they remember?” and “What do they do when the task occurs?” Add a way to record opportunity and support. You will have a more honest account of learning and use, and a clearer basis for the next action.
