You get an email from Sister Astir….
You need to have a word with Dr Dawdling please. He hardly saw any patients yesterday and there was a massive wait in minors. The nurses were tearing their hair out, even Jim, and he’s bald!
Can you have a word and sort him out please!!!
Some questions for you………..
- Are you a slow or fast emergency physician?
- Does speed matter?
- What does work rate tell us?
- Should we publish this data locally?
Why am I asking this? Well for several reasons really. At the moment it is because I have recently had several conversations with colleagues about work rate and work intensity in the ED. Most of these have revolved around the perception that some doctors (usually junior docs) do not work hard when on shift. This is something that has been said for many years and now that we have electronic databases and patient tracking systems it is increasingly easy to pull data out about who saw how many patients, of what type and when. I hear tales of performance league tables being pinned to the door in some departments with those at the bottom of the list being named and shamed, those at the being given special status and in one story even being given ipods for seeing the most number of patients during an attachment.
Does this worry you?
I hope it does, as it worries me a lot so let’s explore why.
This post is written from the perspective of an educational supervisor exploring numbers with trainees, but it is arguably just as valid for any one of us working in Emergency Medicine. It is very much from a UK perspective and I understand that pressure, payment, activity and productivity are considered differently around the world. What applies here may not be relevant to other health economies.
1. Are you slow or fast?
It’s a bit like asking if you are tall or short. Clearly there is going to be a range, with an average and it’s probably going to be roughly normally distributed. In other words to know if you are abnormal you sort of need to know what is normal. So what is normal for a junior doc in a UK ED? Well there is some data out there on this Aruni Sen and colleagues looked at grade and work rate in a North Wales ED and showed (amongst other things) that there is a significant variabilty between the numbers of patients seen within an equivalent grade. That’s also important as if you are going to make comparisons then it seems only fair that comparisons take place between docs at the same stage of training, doing the same shifts, and over a sufficiently long period that variation due to holiday, nights, leave etc. flattens out. That’s one of the reasons why we usually only look at data once a full cycle of shifts (at least 16 weeks, but usually more) has taken place.
So taking all that into account what we are talking about is a variability from the average. So what sort of difference would you expect to see amongst docs of a similar grade over a period of months? 10%, 20%, 50%,…………, 100%.
Have a guess 🙂
Well I have looked at data around this for sometime and the variability is pretty high. Typically the difference will be 80-100% if you have a larger enough number of docs and enough time.
Here is sample data from 41 docs at the same grade, same hospital, same time period, same shifts, same holidays. Thoughts???
The average here is about 660 patients but the range is from 467 to 1012 which is interesting. Sure there will be variation around a mean, but that is quite large. Interestingly the majority of docs are fairly consistent and the variability is greater at the high end rather than the low end. This variability is a fairly consistent feature amongst docs in the ED over the last few years that I’ve been able to look at the data.
So, such data (with all the caveats above) can give an idea of whether you see lots of patients. What is clear is that some docs see more patients than others, but is that the full story, and do those at the bottom end need to be told to speed up?
2. Do numbers matter?
Yes and no. I am pretty sure that the next patient you see does not care how many ‘other’ patients you see over a 4-month period and that’s right. What matters to patients is whether you are going to look after them and if you are going to do your best for them. Surely nobody can argue with this…., or can they? Perhaps there is an argument that an EP looks after a population of patients in the department as much as they look after the individual. Whilst Mrs Miggins may love you to spend lots of time discussing her chronic health problems, in a system with finite doctor resource the time spent with Mrs M is inevitably at the expense of the time you could be spending with other patients (who may be more/same/less in need). In other words there is an opportunity cost related to work rate.
There is also the issue of complexity and patient mix. It will come as no surprise that it takes less time to see a broken finger than it does to see a broken heart (physiological or psychological). If you spend all your time in resus then numbers will be small, but you might just be making the real difference to mortality and morbidity. Similarly you could be seeing loads of patients in the minors end doing the work that (some people) think you are paid for, but for which your advanced training may arguably be superfluous. You should be able to look at case mix when analysing the data. That’s actually quite tricky to do in a way that reflects the needs of individual patients, but you can certainly get some idea by looking at triage categories. This will at least give you an idea of whether you and your colleagues are seeing a similar mix of patients. We find this the easiest way to subdivide the categories, but you could use other classifications like discharge diagnosis I guess. Again, you should be looking for variability in these areas to see if some clinicians seem only to be seeing certain types of patients.
Let’s remember that the argument proposed above about Mrs Miggins is closely related to performance targets and in the UK to the 98% target on 4-hour performance. There is therefore a conflict between individual patient care which is a very clinician focused outcome and patient population performance which is a systems outcome. BOTH are patient related outcomes owing to the opportunity cost if a clinician is slow at seeing patients.
3. What does work rate tell us?
In simple terms it tells us how many and if related to triage category the type and range of patients our doctors see. So, what do we typically find? In truth the range of patients seen varies quite a lot and the type of patients seen does not. It just seems that some docs get through the patients faster than others. Difficulties then arise as a leap can be made to describing this as poor performance – which is probably wrong because it is the wrong question. Measuring the number of patients seen tells us how many but it does not tell us why. As educational supervisors we need to challenge ourselves to explore the why question.
quantity AND/OR quality?
Patients seen per hour does not tell us whether the quality of care changes and indeed a recent article looking at Paediatric Emergency Physicians suggests that productivity does not correlate with quality measures such as admission rates and revisit rates. Now we can and should argue whether those are the only quality measures we should consider, but it is at least interesting to see people looking at it.
Back in 2009 ACEP looked at this and the document on benchmarking and productivity is worth a read. It basically highlights the difficulties in setting any particular benchmark and suggests using oneself as a baseline and working from there, we arguably used a similar concept of self-reflection and performance in the personal NNT post here at St.Emlyn’s. Perhaps, but I also think that in many departments the benchmark might better be placed around an average at a particular grade and rota. ACEP have also looked at the concept of RVUs as explained here (sort of as I find this really tricky to get my head around), but basically a measure of the workload required to see certain types of patients. I think this has more of a place in the US system than here in the UK and it’s not really applicable to trainees.
So what of quality as an unmeasurable? Does fast or slow relate to quality of care given to our patients? I suspect not and in my very unscientific experience I see variability at both ends of the scale.
Ed – Er, fine, but I have a slow trainee though….
We all know who the slow docs are right? Well perhaps not. Again this is anecdotal but it’s often quite tricky to get a feel for productivity just by guessing. You do need the numbers to some extent as there are few behaviours you may recognise that illustrate the difference between perception and reality……
- The machine gun: Works in bursts. Attacks the patient queue with gusto, sees a whole bunch of patients then wanders off. Prone to case selection (takes the easy ones). Numbers often appear high, but nurses get frustrated when enters scarlet pimpernel mode (they seek him here, they seek him there) between bursts. Perception – lazy. Reality – numbers often high.
- The circulator: Often found walking purposefully between different areas of the department. Sees patients in all areas, but often waiting for a second (third, fourth) opinion. Perception – busy, hard worker. Reality – numbers average/low.
- The socialite: Always busy, always doing stuff, chatting with colleagues and popular with staff. If asked people rate this doc as a really good clinician and hard worker. Perception – one of us, great, team player. Reality – numbers typically low.
- The plodder: Doesn’t really like EM that much. Here on rotation. Not a team player, heading for a career in non emergency speciality. Sees all patients in order. Just gets on with it, not much chat. Perception – slow. Reality – numbers often high (these are the real surprises).
- The holistic practitioner. Never embraces EM as a concept. Takes full histories and does complete exams. Always does blood tests. Phones relatives, the police, the RSPCA, makes cups of tea for patients and anyone else. Prefers to ask for advice from speciality teams rather than EM seniors. Perception slow. Reality – glacial.
- The superstar. May or may not be career EM. Keen to learn, knows when to ask about decisions. Perception – excellent. Reality – excellent.
So all a bit tongue in cheek, but I suspect you will recognise some of these caricatures.
So should we do anything about work rate?
OK, so this is my opinion only. Firstly, before you embark on a conversation with a trainee about work rate you need to be sure of the facts and ensure that the numbers are real. Such an interest is usually precipitated by complaints from the nursing staff who rapidly form perceptions about whether juniors are working well. So, look at the numbers and then it’s a question of whether it is worth having the conversation? What might you get out of the conversation and how would you approach it?
Before speaking to the trainee I would pull a sample of clinical records from a few shifts. Basically all the records from complete shifts to get an impression of what is being seen, what is being done, when it is being seen and whether the quality of care (record keeping at least) is up to speed. This can give some impression of quality, but with obvious caveats.
In my practice I rarely have conversations around this but if I do they start off something like this..
“I’ve had a look at the numbers of patients the docs at your grade see in the ED and interestingly it appears that you see fewer patients than I would normally expect for someone at your level. Can we talk about this as I think it would help us understand how you are working and there may be aspects of your practice that we might change”
What comes out of that is interesting. Sometimes it is a clinical issue, sometimes about adapting to EM ways, but more often than not there may be personal, health, confidence or skills issues that come out of the conversation and that is something we can work with.
So numbers are not a problem, but they may be a symptom of something else. Any exploration of this with a trainee should not be predicated on making them faster. That should not be a goal of supervision, rather a side effect of investigating what is happening at the moment.
Whilst putting this together I spoke with my colleagues and we came up with the following graphical description of how we sometimes see trainees in terms of speed and safety. Again this is slightly tongue in cheek and we might argue that the Y-axis is better described as competence (which will vary with grade, experience, role etc.).
Whilst we all want fast, clever, safe, super docs, we know that this cannot always happen. Patient safety is the most important aspect of our practice and as supervisors we need to make sure that we are aware of how trainees are performing and ensuring that they are supported in their decision making. My concern around forcing docs to go faster (directly or indirectly through incentives) is that it may push them to not seek advice, discuss and basically maintain good clinical supervision. If that happens then safety is the issue. So the last thing we want to do is drive docs down from a position of safe practice to one of faster, but unsafe practice. Fortunately, the slow doc with concerns around competence will probably be known to you through other methods (at least they should be).
So basically, whilst I’d like all our docs to be fast and safe, I’ll settle for safe and we need to be cautious about how we approach increasing speed as I am worried that we could turn a safe plodder into an unsafe sprinter.
4. Should we publish this data locally?
This is a question that really divides opinions between managers, seniors and juniors. As we have discussed there are many variables in the data such as opportunity, holidays, shift patterns etc. so my personal opinion is that publishing this data on a very frequent basis is a really bad idea. It risks equating speed as the sole measure of junior doc performance in the ED and that’s a very bad message to send out. If used as a target in this way it is likely to lead to abnormal behaviours (e.g. preferring to see easy/quick patients).
I do know of departments that keep a weekly league table of patients seen and as we know there are those that offer prizes to ‘fast’ clinicians. I really don’t think this is a good idea to be honest as whilst it might appear to have benefits in making people work faster, there are concerns that this may impact on safe decision making and clinical practice.
However, we are entering a time when physician performance is increasingly being integrated into appraisal and now revalidation in the UK. A component of that for nearly all doctors will be some element of performance data in terms of activity, complication rates and hopefully outcomes as well. With this in mind I do usually publish the data to junior docs at the end of their attachment so that they get a feel (good and bad) for how it feels to work in a healthcare system that is starting to get to grips with measuring ‘performance’ through activity. So, this may seem harsh to some, but it is arguably the future of physician management in the UK.
The data is there so we cannot ignore the fact that docs and managers will look at it. As trainers it is also potentially useful in giving us objective evidence.
So it you are told that one of your juniors is ‘slow’. Here’s some do’s and dont’s
- Check the data
- Look at the case mix
- Look at a sample of medical records
- Do use the same processes for everyone
- Tell people in advance what you do about records and work rates.
- Don’t equate speed with quality.
- Don’t trust your instincts – check the data.
- Don’t pick on individuals just because you or others may not like them
13 thoughts on “Can you get the doctors to work faster!!! St.Emlyn’s”
Loving the categories! I also think that feeding back (not with a public name and shame, although an anonymised version can at least raise the issue of variability) is part of the educational process.
I wonder if the issue becomes somewhat circular – socialites and circulators are perceived as being team players, so get favours done for them by other members of staff, whereas the plodder is the one who always ends up taking his/her own bloods, getting TTOs out of the cupboard him/herself etc.
And a slight challenge – if this information is provided to your trainees, do the consultant body also reflect on their own? I’ve worked with machine guns, circulators, socialites, even one or two holistic consultants (as well as, obviously, all the superstars!) – how much potential do we have to lead by example?
Thanks KC. Glad you like the tongue in cheek categories and glad that you have seen them too (so it’s not just me then).
As for consultants – absolutely. The idea for this really came from the performance data that is already published around consultants in many specialities. In Cardio-Thoracics there is good data that links back to individual surgeons. As you know Mrs C is a corneal surgeon and can quote data on graft survival, complications, cataract success (failure, complications, end visual acuity etc.), over many years. So in many specialities output, efficiency and all sorts of other data is already collected and shared.
So what then for EM? Would you as a consultant be happy to be measured against the numbers of patients you see? Can we differentiate personal efficiency from departmental efficiency ( you allude to the same issue in your response around the circularity attached to some of the spoof personalities)? Would the number of breaches on days when you are nominally on call be an appropriate method?? We could think of others I’m sure.
Clearly not breaches, but there is no doubt that with revalidation looming (very soon for me) evidence to support engagement and performance will be expected if not required.
The concern remains though. As with the juniors we should be very, very cautious about equating speed with quality. At the extremes perhaps there is reason to question and at least consider why, but the numbers question is arguably just a sign and not a diagnosis. For example you can be slap bang in the middle of the pack and still not be working to potential efficiency or competence.
At the risk of sounding like an education minister perhaps we should think of this in terms of helping reach their potential rather than using comparative or absolute outcomes.
It’s a tricky area, but I feel it may be just around the corner for all of us.
Interesting … here in Canada we don’t rely on learners to “move the meat” [as attending is in house 24/7] … with trainees lack of speed usually prompts me to ask is there : 1) knowledge gaps? 2) of lack time management skills? 3) poor clinical decision-making? 4) resident wellness issue?
I don’t care if this resident is not going into EM [but do try and help them because they will be running some other service at some point], BUT very concerned if this would be one of my senior EM residents.
As for consultants – we do publish data with names – it hasn’t led to appreciable differences in work rate. Slow docs continue to be slow. I think that changing the way we pay people may change this [i.e per patient rather than hourly], but it may be a complex issue to fix … as Leadership Henry says “you can’t make good people …. you have to HIRE good people”
thanks for sharing
Nadim – we in the UK are still in the position where in EM juniors provide much of the service commitment. (As an aside, I think this is wrong, both for the patients and the trainees). Our own College President recognised it and called for a change in attitudes recently but I think it will be a long process which will need to run in parallel with improving the image of the specialty, keeping good trainees in the specialty and persuading the rest of the hospital that we are not simply a particularly annoying intern (addressing that will be a chicken and egg…….)
Simon – the issue is always that the facets of the job that are measured (often because they are easily measureable) become the only ones of any importance (4 hour target for individuals, anyone?). Any senior can generate good numbers by:
1. Cherry picking – who wants to see the triaged green Mrs Miggins with her functionally overlaid chronic pain when there’s easy points for Master Noggin with a minor head injury? Mrs Miggins will be great practice for the GP trainee in the department anyway.
2. Dumping – why bother trying to address Mrs Miggins’ underlying issues when you can do some scattergun bloods, find an incidentaloma and dump her on the medics? So the medics will moan, but, hey, they do that whatever!
3. Selective disappearance – if you “help” by grabbing all the easy cards and disappearing into see and treat, nobody can find you to deal with any irritating slow problems.
4. Plain grumpiness – be intimidating enough and the trainees will soon learn not to take up your time asking daft questions to improve patient care – let them chase one of your naive approachable colleagues!
This is also clearly somewhat TIC, but the underlying point is valid – if all we measure is quantity, how long will it take for the stereotypes to become reality. I wonder if we really have a grasp of what constitutes a good emergency physician. We walk a fine line between acceptable care of the patient in front of us now and the opportunity cost to the rest of the waiting room, not helped by the cockpit dynamics of many departments – what is our equivalent of graft survival? And how do we measure it?
Excellent thoughts KC. I think you are getting at the fact that it is easy to measure ‘quantifiable’ elements of our practice, but that’s not quality. I hope that is a fair paraphrasing of your comment?
Sadly many of your TIC comments felt like a mirror on one of my bad days 😉
Your final point is excellent, and a real question for EM.
A reasonable paraphrase yes – we need to look at assessing ourselves somewhat like a clinical trial – we should measure outcomes because they matter, not just because they’re easy to measure.
Great blog with some interesting and familiar themes.
Regarding HST/Consultant numbers. It depends what roles you define for your senior decision makers when they’re on the shop floor and what mechanism you have in place to monitor these roles. For instance if the only activity you count is first doc to click on a patient a senior decision maker doing the advice/senior review role will not be clicking on as many new patients as a colleague allocated to ambulatory care area.
I guess I don’t mind being performance managed provided the measurements reflect all the roles we perform and the methods of collecting it are accurate and reliable. I’m not sure that most EDs currently have systems that could be described as such (that’s based on fairly limited exposure so would be happy to be proved wrong!)
Thanks to all for all your comments: this is a topic near and dear to my heart. On top of the roles SN mentions (which could be quantifiable nowadays thanks to some electronic systems), are other roles that HSTs and Consultants perform that can never be measured like determining whether all members of staff (docs, ENPs, nurses) that should be on shift (especially for a night shift) have turned up, determining whether the various areas in the ED are staffed appropriately for the number of patients currently in that area, receiving handover of issues that happened throughout an earlier shift that may be of concern during the next few hours (e.g. bed occupancy and waiting times), handover of patients in your Short Stay Ward/Observation Unit/ECDU etc.
This is a potential aspect of revalidation and remuneration that we must be REALLY careful with as it is specific to Emergency Medicine (and maybe Acute Medicine) alone, and an area that is often misunderstood by many (even, sadly, within our own Departments) leading to inappropriate and unfair criticism of junior (or senior) doctors alike!
It is A LOT easier to be quick than to be a good doctor; it is even quicker to reduce our role to simply triaging patients to the “appropriate” ward/Service! But what type of doctor would you be if your mother/father/loved one was brought into the department?
98% target? What year are we in 2011…
I have resisted performance managing trainees for several years for these exact reasons. Fully agree – A log book does not measure quality only quantity. I Have always provided feedback to a group as a whole, usefully at a half way point you can quantify how a cohort of a certain grade are performing. You can also look across cohorts in time to see how the work rate averages out. Sadly I can see a declining work rate for training grades at all levels. This however does not explain why. Is the medicine getting harder ?does the process require longer? (Yes if you introduce EPR).
To say one value is correct ignores any other possibilities. Total data allows a department to adequately workforce plan.
Perhaps we should correlate data across multiple units to see what a workforce avg truly is? What we do with the data then…
Thank you for this interesting post. I come from the perspective of a trainee on a consultancy service and I think these types are universal in medicine.
There is another area where this is important for EM management, though. My non-EM colleagues and I in hospitals with residents have noticed (and discussed with friends in other hospitals/services who agree) that we receive a greater than average number of calls for consults at the end of the emergency residents’ shifts (revealed when we confirm the number to call with recommendations and they say “oh, I am going off shift”). Our suspicion is that these are the personality types above who are not as productive and “hold the cards” so to speak so that they look busy managing patients, and are just running out the clock with a full hand. More than with timely consults, these consults often reveal that scans/labs/etc were already done and read quite a bit before the call came in. To be fair, it could also be a benign time-management issue where they realize as the shift ends that they still need to call in the consult.
Frustration builds on the consultant side because 1. these are often calls toward the end of our day as well, which means staying late to finish the consult 2. the delay can complicate patient care and often means we are seeing frustrated patients who think we were the cause of the delay and 3. calls back to the resident for clarification often means getting the on-coming resident who only heard in sign-out that “Service X was consulted and waiting for recs” and doesn’t have information to answer questions or engage in a conversation. I also might add that there is the lost learning opportunity to close the loop on what is happening with your patient if you delay the consult and then don’t hear the consulting service’s opinion.
I have worked in two hospitals where the emergency department was managed by staff (no trainees) and this was never a problem there. This may also be unique to a few hospitals and not a problem in the bigger EM world. If it is, though, perhaps this is another element to include in the metrics of analyzing trainee and staff work-flow.
Thanks for this interesting blog post!
This is a very measured discussion (and quite illuminating). We often don’t incorporate these topics in trainee educational days, preferring to concentrate on clinical topics. The function of ‘number’ in educational feedback is probably its most useful one.
I suspect this issue has the most impact in the UK where the bulk of the service is still delivered by trainees. I resist rushing F2s because as trainees they will have strengths and weaknesses and need to learn to make decisions and deal with the situation as much as anything else. In the ideal world all the patients should be seen by a senior (with a trainee or seen by a trainee who presents the case to a senior, who is also responsible for tasking). After all isn’t that what happens on ‘ward rounds’, the case is presented, the senior confirms and dictates the direction (imparting some education).
Sadly we are just too short of seniors (with huge dependence on ad-hoc locums) and not enough Consultants to fill in the gaps.
Just a few comment on system performance:
Some of our woes are nursing related, and the ‘team-work’ aspects often ignored due to the rather artificial distinction in ED between nurses and doctors. Many departments have no straightforward way to coordinate patient care with nurses and one can waste a lot of time either looking for the ‘right’ nurse or end up having to do many of the tasks by themselves. I guess this is where those that develop a good rapport can be assisted.
The IT systems and resources in departments that I have worked in are wholly inadequate and actually adds to the time it takes to see the patient as often it adds a few more steps. The real role of IT is to make working more efficient by reducing the number of steps needed from picking up a patient through to discharge. I wonder if this is responsible for the drop in the average number of patients seen per doc.
I think one of the major issue with performance data for EPs is that our work rate/ patient numbers is so dependant on other external forces and thecase mix. I dont think the breach numbers in depts accurately assess the work rate or standard of care of your EPs but more often represents down stream in-patient inssues. Similarly as a middle grade I can be sat with the same set of notes trying to write them for 30 minutes and be bombarded with questions one after another with queue forming waiting patiently for me to answer. I think they are interesting but how much weight needs to be puton them? I dont think loads
Pingback: Junge Kollegen zu langsam? Teil 2 | Die Welt der medizinischen Blogs