A new view on life. Augmenting Reality at St.Emlyn’s.

Unless you’ve been on holiday in, say, Antarctica for the last few weeks (or still haven’t recovered from SMACC), you’ll have probably heard about this small game called Pokémon GO. You may have even been on the receiving end of some of those who have gotten themselves into a spot or two of bother, such as these boys who got stuck in some caves in Wiltshire and had to be rescued. Luckily they were unharmed, but unfortunately there are increasing reports of those who have come to harm whilst playing, such as a man who stepped out into busy traffic in Ireland, two men who fell down a cliff in California, and several people who have been stabbed or shot in various countries (I won’t link these here). It’s possible you may see Pokémon-related injuries such as broken bones, head injuries, or other minor problems in your ED over the coming weeks. Keep an eye out, and make sure you ask patients how their injuries happened. Obviously the most important question is whether or not they caught whatever they were after!

One of the main features of the game is its use of Augmented Reality (AR), a technology where a view of the real world is augmented by computer- (or in this case phone-)generated data. This could be in the form of images, or sounds, or videos, and is different to virtual reality where the world itself is also simulated. AR also generally occurs in real time through overlays which appear on whatever device is being used, and when the device is moved, the images change to reflect this. It has taken on many forms in the last ten years, through utilisation in real-world map overlays, technology such as Google Glass, flight simulation, entertainment, and even medical devices. With the popularity and success of Pokémon GO, we can only expect companies to find more ways to insert Augmented Reality into our everyday lives.

Augmenting Medical Reality

Probably the most well-known application of augmented reality into medical life is the VeinViewer (other vein map overlay devices are available).

These devices use infrared light to detect vein position under the skin, then project an image of this in real time onto the patient’s body. They are designed to make it easier for healthcare professionals to perform cannulation or venepuncture in patients who are usually difficult, and the manufacturers detail that relevant veins even 10mm under the surface of the skin can be detected. The intended outcome is a more pleasant experience for the patient. Studies of the effectiveness of the various devices compared with the more standard device-free methods of cannulation had mixed results in paediatric populations, and a recent study by Aulagnier et al. (2014) found no improvement in cannulation in non-selected adult ED patients. One of the drawbacks of the technology is that whilst the devices surface map the veins, they do not demonstrate depth. In the last month, a group from Beijing have published their research into a 3D reconstructive vein imaging system which may overcome this drawback. It’s clear further development is needed in this area to identify patients groups who may benefit, and to demonstrate this in well-conducted studies.

AR is also being experimented with for use in immersion therapy for overcoming phobias. Patients undergoing the procedure wear goggles which can overlay images of small insects or animals, such as spiders, cockroaches (as in the image below), or rats, in their hands or around their person. This can be used as a means of overcoming phobia on its own, or as an intermediary step to physical exposure. Trials comparing in-vivo exposure versus augmented reality exposure, despite small numbers, show good outcomes and positive experiences from the patients.

Image from Botella et al. (2016) used under Creative Commons Attribution Licence.

Augmenting Education

There are many concepts in skill mastery, one of which is that of deliberate practice. The amount of practice you do certainly helps, but just as important is the quality. Deliberate practice involves repeatedly performing the same skills, using methods aimed at your current level, with the aim of improving performance through immediate feedback. Gaining feedback after each repetition allows small changes to be made. Repetition allows these to be tested and changed again if necessary. Ericsson and colleagues wrote on the subject 23 years ago, and there have been many other papers on the subject since then. Our very own Simon has written a fantastic blog on the topic, in particular relating to his experience of improving musically through deliberate practice. There are many technical skills we perform on a day to day basis, and practising these wisely can make a huge difference. But where does AR fit into this?

Surgical educators are experimenting with augmented reality technology to aid in teaching and learning surgical procedures. When practising laparoscopic surgery, realistic haptic feedback is crucial for learners to progress in improving their skills. Such feedback allows students to engage in deliberate practice independently, and although a surgical tutor in the room is helpful, using technology can provide immediate objective feedback. A quick PubMed search for augmented reality in surgery shows that laparoscopic surgery is not the only focus of research efforts. Real time image overlay could help to integrate what are usually two-dimensional radiological images with the the three-dimensional human form. Groups are also looking into using AR to assist with maxillofacial, orthopaedic, cardiac, and neurosurgery. The main theme appears to be locating lesions, but one interesting study looks into using the technology to aid in K-wire placement in orthopaedic surgery. This has the potential to reduce radiation load, and increase surgical efficiency. Augmented reality could also help with other procedures such as percutaneous interventions and nerve blocks. It will be interesting to see how the technology progresses, as there already seems to be a lot of interest in its integration with medicine and education.

I have also heard of hospitals in the UK using augmented reality as part of the induction process for new doctors. Sadly I can’t find any literature, but the premise was that new starters could go on a self-guided tour. At certain points in the hospital they would use software on their phone to view and listen to videos of staff describing whichever area they were in and showing them around. Overlays on the screen could give them important information, and guide them to the next point on the tour. Probably a much more interactive and engaging way to learn the layout of your workplace than currently happens. There are some free applications out there for you to create your own AR content, and so some of the simpler uses of the technology can be explored by novices, with a bit of searching around.

As for the Emergency Department, vascular access, nerve blockade and fracture manipulation would suggest themselves as ideal applications for AR. You could even wear glasses which recognise patients’ faces, automatically pulling up their name, age, presenting complaint, and most recent observations, overlaying these against the real world in real time. Medical and nursing staff alike could be alerted to urgent jobs, patients with high early warning scores, or simply the next patient to be seen, just by a message in the corner of your eye. It’s sounding futuristic now, but the technology is there (or nearly there), it’s just a case of finding a way to use it that enhances our working lives.

Final Thoughts

Augmented reality is the new kid on the block, and with the many ways it is finding a path into medicine, surgery and education, it is a technology to be embraced and investigated, and can only improve with time. If you’ve had any experience with AR, it would be great to hear about it. If not, I’m sure it will work its way into your medical life in the next decade.

vb

Chris

Further Reading

• How does Augmented Reality work? A HowStuffWorks article.
• Expert Enough blog post on Deliberate Practice
• Simon’s blog on The Pursuit of Mastery through Deliberate Practice

Before you go please don’t forget to…

• Subscribe to the blog (look top right for the link).
• Subscribe to our PODCAST on iTunes.
• Follow us on twitter @stemlyns.
• Like us on Facebook.
• Find out more about the St.Emlyn’s team.