PHARM quality – how do you know when you’re doing it well?

This post from Dr Alan Garner tackles a core problem for all practitioners who give a damn – how do you know you’re doing it well? A chat worth having and Alan has a pretty good summary of the Carebundle approach. 

How do we measure quality in prehospital and retrieval medicine?  Speed?  Number of procedures performed?  Number of twitter followers?

Seriously though, this is a question that vexed me for many years as a service director and trying to find metrics that measure things that mattered seemed an elusive task.  The major part of the problem stemmed from the heterogeneity of the patient population that we treat.  Even simple (but easily measured and therefore attractive to bean counters) things like timeliness are not straightforward.  Not because they are hard to measure but because sometimes time matters and other times it very clearly does not.  Indeed emphasising it as a measure could lead to perverse outcomes for some patients.

Let me give you a couple of examples to illustrate the problem:

Case 1.  Central abdominal stab wound with hypotension.

There is almost no prehospital intervention that matters in this patient except gasoline and perhaps tranexamic acid.  I don’t think anyone would argue that time is a reasonable quality measure in this patient.

Case 2.  COPD patient in a small hospital an hour flying time from the nearest intensive care unit.

Patient is eventually stabilised on non-invasive ventilation after three hours of effort by the transport team at the referring site. They are then safely transported.  Clearly for this patient time does not matter at all.  Reporting turnaround time at the referring site in this patient may place subtle pressure on the team to intubate the patient early and depart – a move that is very clearly not in the patient’s best interests and would have placed the patient at significantly increased risk of unnecessary morbidity and mortality.

This got me thinking that our measures of quality had to be disease process specific or we were never going to move forward.  Speaking with Erwin Stolpe was the turning point in my thinking.

You Should Really Try to Know Erwin

Many of you will not have heard of Erwin.  Sometimes when I talk to people or read things on social media I get the impression that physician staffed HEMS started in about 2005.  The reality of course is quite different.  Erwin is a trauma surgeon from Munich who began flying as a resident on the Christoph 1 service out of that city in 1968 (yes, not a typo – 1968).

Erwin Stolpe
Here he is, at AirMed 2014 in Rome.

These days he no longer flies but is chair of the ADAC medical committee.  For those unfamiliar with ADAC they run about 35 physician staffed HEMS bases in Germany and also operate several jets for longer range transports.  Their HEMS services alone conduct about 50,000 prehospital cases annually.  The breadth and depth of experience of this organisation is extraordinary and Erwin has been there from the beginning.  You would think there might by a few pearls of wisdom there and you would be right.

The Key Cases

Erwin described to me the “tracer diagnosis” process they use to track the quality of the care that they provide.  Analysis of their prehospital caseload indicated that four diagnoses made up 75% of the cases they attended.  For these four diagnoses they defined the treatments that they expected the teams to achieve (see pages 52 onwards of this presentation by Erwin for more detail).  They used national and international consensus guidelines as a base.  They then began reporting against those criteria and they have also started to publish that performance.

What Erwin was calling “tracer diagnoses” is probably better known to us in the English speaking worlds as a “Carebundle”.  Lots of people will be familiar with the ventilator Carebundle for intubated patients in the intensive care unit.   Adherence to the items in the bundle is associated with lower rates of ventilator associated pneumonia.  In NSW and Queensland, Health Departments have introduced bundles for central line insertion in order to tackle the rates of central line associated bacteraemia.  In this case the bundle applies to a procedure or process rather than a diagnosis.  Is there a place for this kind of methodology in the prehospital and retrieval world to improve quality too?

What are we talking about when it comes to PHARM?

Let’s start by looking at what a Carebundle is.

“A bundle is a structured way of improving the processes of care and patient outcomes: a small, straightforward set of evidence-based practices — generally three to five — that, when performed collectively and reliably, have been proven to improve patient outcomes.”

This definition is taken straight from the Institute for Healthcare Improvement (IHI) website.  There is a bit of controversy regarding whether the items in a Carebundle really need to all be completed for the bundle to be effective in some sort of synergistic way or whether they are in fact just a checklist of items that have been shown to be effective and you get as many done as you can.  I am not aware of any evidence for the synergistic effect multiplier that is implied on IHI website.  I think it is unarguable however that you should try and get as many of the things that are proven to make a difference to that condition completed as possible.  That is certainly the approach that we have taken.

Another quote from the IHI website describes for me what we are trying to achieve by using bundles:

“The power of a bundle comes from the body of science behind it and the method of execution: with complete consistency. It’s not that the changes in a bundle are new; they’re well established best practices, but they’re often not performed uniformly, making treatment unreliable, at times idiosyncratic. A bundle ties the changes together into a package of interventions that people know must be followed for every patient, every single time.”

Using Carebundles in hospitals is clearly not new.  Even in EMS it has been previously described for benchmarking purposes.  The attraction of the methodology for me was that we would know if our care for patients with severe head injury for example was following the best available evidence and we would know what proportion of our patients were receiving that care.  I did not want just some of our patients to get that care, I wanted all of them to get every item of care that we could identify matters for that disease process all of the time.

Making it Match What We Do

For our rapid response service in Sydney we then determined from our medical database the diagnoses that cover 75% of our caseload as ADAC had done.  For us this resulted in the following list:

  • Multiple blunt trauma
  • Isolated severe head injury (GCS<9)
  • Burns (>15% BSA)
  • Penetrating trauma
  • Immersion/drowning
  • Seizures (to which we were often being dispatched as they were mistaken for head injury or had caused a minor traumatic event)
  • ROSC post primary cardiac arrest (similar to seizures – trivial traumatic injury and patient in VF)
  • Traumatic cardiac arrest (for us this is the HOTTT Drill which I have described in a previous post, well podcast but which also includes the HOTTT Drill package to go with it).

We then turned to the evidence based consensus guidelines, Cochrane reviews and good quality RCTs to define the Carebundle items.  This is a sobering process as you realise just how few interventions there are that have good evidence to back them up.  This is particularly true for prehospital care where we are often operating in an evidence free zone.  In many cases we had no choice but to go with the consensus (or best guess as I like to call it).  We decided that we would include intubation for unconscious trauma patients for example despite the evidence not being all that strong and in many cases contradictory.

When we had defined the items for the specific diagnosis we printed them up on cards that team members carry in their pocket.  These serve as a checklist which teams use on site or in transit just to be sure that they have covered all the items.  Below is our isolated severe head injury card – the item I constantly forget is the blood glucose level (BSL).  Highly embarrassing if this is low when you arrive at the trauma centre!  I for one am glad to have the prompt.

BI copy

Some of these items are extrapolated from in-hospital care.  For example having the external auditory meatus (EAM) above the JVP makes sense in terms of managing raised ICP but there is no direct prehospital evidence that shows this changes outcome.  We have also set relatively conservative targets for things like oximetry and blood pressure.  Most of the evidence suggests SpO2 >90% is enough but we felt that desaturation happens very rapidly from this point so we would rather aim a little higher.

Aspirations and Signals

Some of the items we knew from the outset that we would never achieve in all cases.  Scene time of <25mins is the obvious example.  When a patient is trapped this is outside of our control.  We know however that one in five patients with a severe head injury will have a drainable haematoma that is time critical.  We therefore included this item in order to signal to the team that we expect them to treat severe head injury as a time critical disease in the prehospital phase.

Some of the bundles have conditional items as well.  For head injury this is the hypertonic saline which we only expect to be given if there are lateralising signs or neurological deterioration.

When the team returns to base they complete an audit form indicating if the bundle items were achieved and if not, the reason for the variance.  This both reinforces for our personnel the contents of the bundles and also allows us to report on compliance.  Below is an example of our report for severe head injuries showing the reasons of variance in the comments section.

Report copy

You can see that we don’t meet all the targets all the time, and there is usually a good reason when we don’t.  However the Carebundles allow us to be transparent about what we think good care is, and also about how successful we are in achieving it.  We include Carebundle compliance (along with a lot of other stuff) in our external reporting in NSW to the Ministry of Health, NSW Ambulance, The NSW Institute of Trauma and Injury Management and all the trauma centres to which we transport patients.  Transparency is a key component of good governance and this processes helps us to achieve that.

Those People Were Here First

The concept is not new.  I merely walk behind the giants of the industry and follow their lead in this.  It is also worth noting that Russell MacDonald from Ornge in Ontario is leading a similar project with an initial group of 10 “tracer diagnoses” amongst a small international collaboration of critical care transport providers.  It will be interesting to see how closely their bundle items accord with our own.  Aligning our bundle items would allow us to benchmark ourselves against similar organisations in other parts of the world and create opportunities for us to learn from organisations who manage specific conditions better than we do.  In the end this is about maximising the outcomes for our patients and I will gladly accept any help I can get in achieving that.


Here’s the stuff referred to along the way, because the originals remain a vital part of looking at the issue.

J. B. Myers, C. M. Slovis, M. Eckstein et al., “Evidence-based performance measures for emergency medical services systems: a model for expanded EMS benchmarking,” Prehospital Emergency Care, vol. 12, no. 2, pp. 141–151, 2008.

Here’s a link to the English version of the “tracer diagnosis” abstract.

Helm M et al.  [Extended medical quality management exemplified by the tracer diagnosis multiple trauma. Pilot study in the air rescue service] Anaesthesist 2012;61(2):106-115.

(Well, not all of us are clever enough to know German.)

Here’s the direct link to the IHI page.

The image of Erwin Stolpe comes from the Intercongress flickr account and is unaltered under the CC 2.0 licence.


The Dangerous Little Details

A new bit of research is out looking at paediatric intubation in the prehospital and retrieval setting. Picking it up and turning it this way, that way and all around, here’s Dr Andrew Weatherall. 

Advanced prehospital practitioners that I’ve met have some pretty common traits. They are pretty comfortable around things that other people might find chaotic. They often have pretty strong opinions on food and coffee. Not necessarily even on good food either. I’ve been given connoisseur-level education on various take away options. Most importantly, they are appropriately bananas about doing a good job for their patients.

That extends to paediatric patients which is obviously excellent. Except we tend not to do our most excellent work when it comes to kids. The reasons for that could fill many a blog post (and maybe we’ll get back to that another time) but kids tend to get less pain relief when faced with similarly painful situations, less interventions even when they’re indicated and we tend to do those procedural things less well.

In 2011 Bankole et al. compared interventions in kids (defined as < 12 years old) and adults with a head injury and a GCS < 15 in New Jersey (there was 102 patients in the kids group matched to 99 adults with equivalent injuries).  69.2% of the kids had some sort of problem with intubation. That was across failed intubation (29.03% vs 2.27% in adults), tube dislodgement (16.12% vs 2.27%), wrong-sized tube (7.45% vs 0%) and multiple attempts (as in over 3 tries) at intubation (6.45% vs 2.27%). A peripheral IV was there in 85.9% of adults but only 65.7% of kids.

In a paper that also commented on relative intubation rates in advanced EMS vs general EMS in the Netherlands, Gerritse et al also commented on analgesia. In their study 77% of kids who really needed some form of analgesia actually received nothing from the general EMS. No kid under the age of 4 received any form of analgesia from the EMS. Not one.

I’m not quoting those papers to say anything other than good practitioners (I have a predisposition to think most of those working at any level of EMS are people trying to do the best job their system and training allow) find kids extra difficult. This patient group provides an additional challenge on top of the storm you already deal with the scene. Like someone started blasting fairy floss into your eyes in the middle of that storm. OK I’m not sure that was the greatest analogy but it’s happened now so maybe we can just agree to move on while also remembering that when you’re a kid fairy floss is pretty great. Mmmm, fairy floss.

Enter the Swiss, purveyors of good chocolate and cheese with holes, with some interesting work that sheds a little extra light on things that even the most advanced practitioners find challenging about little people and airway management.

Let’s Stop and Check the Scenery

Not the mountains or lakes or Large Hadron Collider scenery, the other scenery.

Appearing in SJTREM, the paper comes from a  look at their database between June 2010 and December 2013. Across their 12 bases and one affiliate base they do around 11000 prehospital or interhospital missions per year with their paramedic-doctor teams. I should point out that these advanced teams really have had good training in airway management and specific paeds time. The study looks at any kid under the age of 17 requiring any airway manipulation (not just intubation or supraglottic airway or tracheostomy but bag-mask ventilation as well).

From their pool of 4505 paediatric patients over the 3.5ish years (which if they’re doing around 11000 jobs per year should be around 11-12% of their total workload) the ended up with 425 kids requiring some sort of airway care (9.4% of the paediatric group). A little over half (225) were prehospital cases. From here on in when we talk about intubation it’ll be about prehospital missions because those moving between buildings were already intubated and ventilated.

So what did these top operators find?

Actually It’s Not About the View

In the 215 patients for whom an attempt at endotracheal intubation was attempted, first-pass success was 95.3%. Now, if you’ve dropped by this blog before you might recall Dr Alan Garner discussing whether this is the most important measure. I think that’s a great post, but I don’t think it is meant to be interpreted as “first pass intubation tells us nothing” (Alan can always correct me).

What this number does say is that the challenges in kids aren’t necessarily about getting a view of the cords that is enough to achieve intubation. Only 10 patients (4.7%) were described as inflicting a difficult airway management scenario on the team. 98.6% eventually ended up with a support snorkel in their trachea.

There were 2 children who could not be intubated and ended up oxygenating very nicely with the aid of a supraglottic airway, while one patient with a known “airway issues syndrome” (Goldenhar’s syndrome) couldn’t be either intubated or ventilated but was already at the end of a prolonged arrest situation.

So for advanced EMS providers, maybe it’s not the getting a view/passing the tube part of the procedure that is really at issue. In our own research that touched on this, the intubation success rate was 98.7% of the paediatric patients were successfully intubated while one patient was managed with a laryngeal mask in the prehospital phase.

This fits with the overall truth of paediatric airways: unanticipated difficult laryngoscopy is less common in kids than adults.

So Where’s the Problem?

The problems with paeds airway intervention here are about the details. You may have noticed that people who do subspecialty work in paeds can be a little bit fanatical about details. There’s a reason for this. A smaller airway is less forgiving of the tube that is the wrong size, be it too big or too small. An endotracheal tube that is 1 cm too far in on your 1 year old is proportionally a lot closer to the carina than when the same situation applies to an adult. Add a little flexion or extension and that whole tube can end up visiting new pockets of the bronchial tree.

This is the part that is really well covered in the Swiss study. In the 82.7% where intubation was noted, 82.5% got an adequately sized tube. It was too shrunken to be appropriate in 2.9% and too gargantuan in 14.6% (in the under 1s that rose to 57.5%). Rates were higher if that tube was placed during a CPR scenario.

The depth? Well, if you went off the formulae often mentioned in dispatches, most insertions were deeper than that. And while I can’t seem to find the bit in the results that clarifies this statement, the authors say in the discussion that “Only the placement of the depth marking of the correct Microcuff ET tube … for age between the vocal cords was accurate for all paediatric patients …” (Not familiar with the markings? You could look at an earlier post on this site, here.)


Details, Details

I think this is the key message of this study. Lots of things might make you sweat about paediatric airways. I suspect that for most practitioners it is the view and “plastic through the cords” components that cause the stress.

That bit is important, of course, and everyone wants to do that bit well. This study supports the argument that advanced practitioners already do that bit really well. Perhaps in thinking keenly about that bit it’s attention to some details, the sort of details that kids are pretty unforgiving about, that gets in the way of safer paeds airway management.

Stavros Markopoulos
Look at this butterfly. Gets fuzzy on the last few details of the right wing and can’t even butterfly properly.

Things to Take Away

Any research only reveals a very particular part of a story. There are questions left unanswered or things that don’t quite apply to your practice. That doesn’t mean we can’t use those results to reflect on things we do when we deliver our variant of advanced care.

So I’d say there are a few key things suggested by this study:

  • If you’ve trained in paediatric airway management, chances are the intubation itself (at least the getting a view and passing the tube bit) will go well.
  • Really well trained people still find the details challenging. The wrong tube size and the wrong depth of insertion matter in these patients.
  • It might be time to review whether those old formulae are the best option.
  • Knowing your equipment (like where the line on the tube goes) is pretty worthwhile.
  • The tube through the cords isn’t where attention to detail stops. That’s not the moment to ease up.

So we can all get out there, push through the fairy floss, be confident that we’ll get those endotracheal tubes in and start remembering the little details that will produce perfection.

No more fuzzy butterflies.


Of course it’s not the fault of the butterfly it’s right wing looks fuzzy. It’s the photographer. Well, actually it’s an amazing photo where the wing is a tiny bit in a different alignment. It’s from flickr Creative Commons via Stavros Markopoulos and is  unaltered.

The source paper link is right here and it’s open access:

Schmidt AR, Ulrich L, Seifert B, Albrecht R ,Spahn DR, Stein P. Ease and difficulty of pre-hospital airway management in 425 paediatric patients treated by a helicopter emergency medical service: a retrospective analysis. Stand J Trauma Resusc Emerg Med. 2016; 24:22. 

I also mentioned a paper we put out there:

Barker CL, Weatherall AD. Prehospital paediatric emergencies treated by an Australian helicopter emergency medical service. 2014; 21:130-5. 

Then there’s the Bankole et al. paper:

Bankole S, Asuncion A, Ross S, et al. First responder performance in pediatric trauma: A comparison with an adult cohort. Pediatr Crit Care Med. 2011;12:e166-70. 

And finally the Gerritse et al. paper which is also open access:

Gerritse BM, Schalkwijk A, Pelzer BJ, Scheffer GJ, Draaisma JM. Advanced medical life support procedures in vitally compromised children by a helicopter emergency medical service. BMC Emerg Med. 2010;10:6.

Addit: After a really helpful comment from Paramedidad the line “In their study 77% of kids who really needed some form of analgesia.” was fixed to read “In their study 77% of kids who really needed some form of analgesia actually received nothing from the general EMS.”