Miller’s pyramid and workplace assessment

“Can you sign this off?” Suppose a trainee asks you that towards the end of a busy shift. You’ve worked alongside them, discussed several patients and had no particular concerns. It would be easy enough to complete the form. But what, exactly, have you seen them do?

That question gets us to the heart of Miller’s pyramid. We can teach something thoroughly and still have only a limited idea of whether the learner can do it. They may be able to talk us through a task, or perform it under examination conditions. We need to think carefully about what those observations allow us to conclude about their work.

Knows, knows how, shows how, does

George Miller described the framework in 1990. Its four levels distinguish knowledge, the application of that knowledge, a demonstration and performance in practice. Norcini’s discussion of workplace assessment explains why looking at everyday clinical work adds something that examinations alone can’t provide.

Take handover as an example. A learner might know which information a handover should include. We could explore knows how by giving them a written case and asking what they would communicate and why. In a simulated encounter they can show how they would hand over to a colleague. To find out what they do, we need evidence from their clinical work.

There’s a useful difference between those last two. A rehearsed handover to an examiner may go very well. On the shop floor, the receiving clinician might be interrupted, information may still be arriving and someone has to establish who is taking responsibility for the patient. We need to see how the learner handles the work as it happens.

These levels describe the evidence we’ve collected about a task. Be careful about turning them into permanent labels for people. A registrar can be very capable in one part of the job and need considerable help with something unfamiliar.

Does higher always mean better?

The original article suggested that assessment becomes more valid the higher we go on the pyramid. We need to correct that. A single observation of clinical work can be a very poor basis for a broad judgement, however close to the top of the diagram it appears.

Cook and Beckman’s account of validity asks us to consider the interpretation of an assessment score and the evidence supporting it. Put more simply: what are we claiming this result tells us, and have we got enough evidence to make that claim?

If the question is whether a learner recognises an ECG pattern, a well-designed written test may be useful. If the question is whether they can explain an investigation to a worried patient, that test leaves something substantial unanswered. Choosing an assessment starts with deciding what we need to know.

We should also be clear about the consequence. A brief observation can give us plenty to discuss in a teaching conversation. A decision about progression or independent practice needs a broader view. That should draw on different encounters and other relevant evidence, including occasions when the accounts don’t agree.

Back to the form

Back to our trainee. The useful response might be to arrange to watch a handover. Decide beforehand what you’re looking at. Perhaps the learner wants help keeping the account concise without leaving out information the receiving clinician needs.

Then actually watch it. Afterwards, ask what they thought worked and discuss what you noticed. If you had to prompt them, record that. If the receiving team repeatedly interrupted, include the circumstances. “Good handover” is difficult to learn from; a description of where the listener became confused gives both of you somewhere to start.

We then need another opportunity to find out whether the feedback helped. Otherwise we’ve documented a moment and left the trainee to sort out the rest. The existing St Emlyn’s discussion of coaching and feedback is worth reading alongside this.

Where does simulation fit?

There’s an obvious difficulty in emergency medicine: some of the things we most need to prepare for happen infrequently, and the right assessor may be nowhere nearby when they do. Waiting for a suitable emergency is a poor educational plan.

Simulation lets us practise and observe those tasks. The original article made the case for in situ simulation, where familiar equipment and the usual working environment can bring out problems that a classroom exercise misses. That remains a good reason to use it.

It’s still a constructed situation, though. Putting a mannequin on a real resuscitation trolley doesn’t turn the assessment into evidence of routine clinical performance. Nor does the label “shows how” make it a lesser teaching experience. For a rare event, it may be our best chance to examine the task closely and repeat the difficult part.

Before completing the next assessment, pause over the claim you’re making. If you watched one handover, describe that handover and the help needed. It may be enough for a useful teaching conversation. Before making a broader judgement about the learner, look for evidence from other occasions.