Research That is Positive When It Is Negative

This weeks post is the first in a series touching on some of the challenges when you start researching technology for the prehospital setting (or anywhere really). Dr Andrew Weatherall (@AndyDW_) on why some monitors aren’t the monitors you’re sold. 

I am new to the research game. As is often the case, that brings with it plenty of zeal and some very rapid learning. When we first started talking about the project that’s now my PhD, we set out wondering if we could show something that was both a bit new and a positive thing to add to patient clinical care.

It didn’t take long to realise we’d still be doing something worthwhile if the project didn’t work one little bit.

Yep, if this thing doesn’t work, that would still be fine.

 

Simple Questions

I’m going to assume no one knows anything about this project (seems the most realistic possibility). It’s a project about brains and lights and monitors.

It came out of two separate areas of work. One of these was the prehospital bit of my practice. All too often I’d be at an accident scene, with an unconscious patient and irritated by the big fuzzy mess at the middle of the clinical puzzle.

“How’s the brain?”

Not “how are the peripheral readings of saturation and blood pressure against the normative bell curve?” Not “how are the gross clinical neurological signs that will change mostly when things get really grim?”

“How’s the brain?”

At the same time at the kids’ hospital where I do most of my anaesthesia we were introducing near-infrared spectroscopy tissue oximetry to monitor the brain, particularly in cardiac surgery cases.

The story sounded good. A noninvasive monitor, not relying on pulsatile flow, that provides a measure of oxygen levels in the tissue where you place the probe (referred to as regional oxygen saturation, or tissue saturation or some other variant and turned in to the ideal number on a scale between 0 and 100) and which reacts quickly to any changes. You can test it out by putting a tourniquet on your arm and watching the magic oxygen number dive while you inflate it.

Except of course it’s not really as simple as that. If you ask a rep trying to sell one of these non-invasive reflectance spectroscopy (NIRS) devices, they’ll dazzle you with all sorts of things that are a bit true. They’re more accurate now. They use more wavelengths now. Lower numbers in the brain are associated with things on scans.

But it’s still not that simple. Maybe if I expand on why that is, it will be clearer why I say I would be OK with showing it doesn’t work. And along the way, there’s a few things that are pertinent when considering the claims of any new monitoring systems.

 

A Bit About Tech

Back in 1977, a researcher by the name of Franz Jöbsis described a technique where you could shine light through brain tissue, look at the light that made it out the other side and figure out stuff about the levels of oxygen and metabolism happening deep in that brain tissue. This was the start of tissue spectroscopy.

Now, it’s 38 years later and this technology isn’t standard. We’re still trying to figure out what the hell to do with it. That might just be the first clue that it’s a bit complicated.

Of course the marketing will mention it’s taken a while to figure it out. Sometimes they’ll refer to the clinical monitors of the 1990’s and early 2000’s and mention it’s got better just recently. They don’t really give you the full breadth of all the challenges they’ve dealt with along the way. So why not look at just a few?

  1. Humans Aren’t Much Like Cats

Jöbsis originally tested his technique on cats. And while you might find it hard to convince cat lovers, the brain of a cat isn’t that close to a human’s, at least in size. (As an aside, I’m told by clever bionic eye researchers the cat visual cortex actually has lots of similarities with that of humans – not sure that explains why the aquarium is strangely mesmerising though).

He also described it as a technique where you shone the light all the way across the head and picked up the transmitted light on the other side. But even the most absent-minded of us has quite a bit more cortex to get through than our feline friends and you’d never pick up anything trying that in anything but a neonate.

So the solution in humans has been to send out near-infrared light and then detect the amount that returns to a detector at the skin, on the same side of the head as you initially shone those photons.

When you get handed a brochure by a rep for one of these devices, they’ll show a magical beam of light heading out into the tissues and tracing a graceful arc through the tissues and returning to be picked up. You are given to believe it’s an orderly process, and that every bit of lost light intensity has been absorbed by helpful chromophores. In that case that would be oxy- and deoxyhaemoglobin, cytochromes in the cell and pesky old melanin if you get too much hair in the way.

See? Here's the pretty version that comes with the monitor we're using in the study? [It's the Nonin EQUANOX and we bought it outright.]
See? Here’s the pretty version that comes with the monitor we’re using in the study? [It’s the Nonin EQUANOX and we bought it outright.]
Except that’s the version of the picture where they’ve put Vaseline on the lens. Each one of those photons bounces eratically all over the place. It’s more like a small flying insect with the bug equivalent of ADHD bouncing around the room and eventually finding its way back to the window it flew in.

So when you try to perform the underlying calculations for what that reduction in light intensity you detect means, you need to come up with a very particular means of trying to allow for all that extra distance the photons travel. Then you need to average the different paths of all those photons not just the one photon. Then you need to allow for all the scattering that means some of the light will never come back your way.

That’s some of those decades of development chewed up right there.

  1. Everyone Looks the Same But They Are Different

So that explains the delay then. Well there’s another thing that might make it hard to apply the technology in the prehospital environment. Every machine is different. Yep. If you go between systems, it’s might just be that you’re not comparing apples with apples.

That particular challenge of calculating the distance the light travels? Every manufacturer pretty much has a different method for doing it. And they won’t tell you how they do it (with the notable exception of the team that makes the NIRO device who have their algorithms as open access – and their device weighs 6 kg and is as elegant to carry as a grumpy baby walrus).

So when you read a paper describing the findings with any one device, you can’t be 100% sure it will match another device. This is some of the reason that each company calls their version of the magic oxygen number something slightly different from their competitor (regional saturation, tissue oxygenation index, absolute tissue oxygen saturation just to name a few). It might be similar, but it’s hard to be sure.

Maybe that's harsh. Could a walrus be anything but elegant? [via Allan Hopkins on flickr under CC 2.0]
Maybe that’s harsh. Could a walrus be anything but elegant? [via Allan Hopkins  on flickr without mods under CC 2.0]
  1. When “Absolute” Absolutely Isn’t Absolute

You get your magic number (I’m going to keep calling it regional saturation for simplicity) and it’s somewhere between 60 and 75% in the normal person. The thing is it hasn’t been directly correlated with a gold standard real world measurement that correlates with the same area sampled.

The NIRS oximeter makes assumptions about the proportions of arterial, venous and capillary blood in the tissue that’s there. The regional saturations are validated against an approximation via other measures, like jugular venous saturation or direct tissue oximetry.

On top of that all those “absolute NIRS monitors” that give you a definite number that means something? No. “Absolute” is not a thing.

It’s true the monitors have got much better in responding to changes quickly. And they’ve added more wavelengths and are based on more testing so they are more accurate than monitors from decades past. But they can still have significant variation in their readings (anywhere up to 10% is described).

And they spit out a number, regional saturation, that is actually an attempt to take lots of parameters and spit out a number a clinician can use. How many parameters? Check the photo.

This is from an excellent review by Elwell and Cooper. [Elwell CE, Cooper CE. Making light work: illuminating the future of biomedical optics. Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences. 2011;369:4358-4379.]
This is from an excellent review by Elwell and Cooper. [Elwell CE, Cooper CE. Making light work: illuminating the future of biomedical optics. Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences. 2011;369:4358-4379.]
  1. The Practical bits

And after all that, we reach the practical issues. Will sunlight contaminate the sample? Can it cope with movement? Do you need a baseline measurement first? Does it matter that we can only really sample the forehead in our setting?

All the joy of uncertainty before you even try to start to research.

 

So why bother?

Well the quick answer is that it might be better for patients for clinicians to actually know what is happening to the tissue in the brain. And acknowledging challenges doesn’t mean that it isn’t worth seeing if it’s still useful despite the compromises you have to make to take the basic spectroscopy technique to the clinical environment.

But even if we find it just doesn’t tell us useful stuff, we could at least provide some real world information to counter the glossy advertising brochure.

There are already people saying things like,

“You can pick up haematomas.” (In a device that just tells you if there’s a difference between the two hemispheres.)

“Low regional saturations are associated with worse outcomes.” (But that’s probably been demonstrated more in particular surgical settings and the monitoring hasn’t been shown to improve patient outcomes yet.)

“You can even pick up cytochromes.” (In the research setting in a specially set-up system that are way more accurate than any clinical devices.)

All of those statements are a bit true, but not quite the whole story. The other message I extract from all of this is that all this uncertainty in the detail behind the monitor can’t be unique to NIRS oximetry. I have little doubt it’s similar for most of the newer modalities being pushed by companies. Peripherally derived Hb measurements from your pulse oximeter sound familiar?

After all this it’s still true that if we can study NIRS oximetry in the environment that matters to us we might get an exciting new answer. Or we might not. And sometimes,

“Yeah … nah.”

Is still an answer that’s pretty useful.

 

 

This is the first in a series. The next time around, I’ll chat about the things we’re trying in the design of the study to overcome some of these challenges.

If you made it this far and want to read a bit more about the NIRS project, you can check out the blog I set up ages ago that’s more related to that (though it frequently diverts to other stuff). It’s here

 

One thought on “Research That is Positive When It Is Negative”

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s