HoloLens-assisted open surgery in Imelda Hospital broadcasted live

Written by Wouter Martens

In cooperation with the medical device company Becton Dickinson (BD), the HoloLens will be used to define the application for training and educational purposes within the medical field. As part of this project, we got introduced to Dr. Tim Tollens, general and abdominal surgeon at the Imelda Hospital in Bonheiden. He quickly noticed the many opportunities the HoloLens and Mixed Reality would give to healthcare workers (doctors, nurses,…) in their daily medical activities. Upon his request and together with BD, we developed a custom HoloLens application to visualize information needed for patient treatment support.

On Thursday February 25th, Dr. Tollens performed a complex open surgery while being assisted by the HoloLens and our custom-made application. The live broadcasting was a registered training program and limited to healthcare professionals all over the world.

Holographic visualization

During surgery, Dr. Tollens often relies on information such as 3D scans, X-rays and even PowerPoint slides containing (when required anonymized) medical information about the patient, his or her journey, videos, … Thanks to the custom app we built, he can now access this data with a simple “swipe” in the air. You could compare this with swiping on your phone without having to hold it while you read or watch a video.

As the holographic window “floats” within Dr. Tollens’ field of view – without blocking his vision – he can consult the data while still having his hands free to proceed the surgery. Besides operating, he can easily consult all the information needed without any risk of contamination through physical components like computer, mouse and/or keyboard.

Image by Imelda Hospital Bonheiden
HoloLens enables a first-person view for spectators

The surgery was broadcasted via the BD-medical training platform herniau.com. The video stream consisted of the footage of two different sources. Firstly, there was a camera crew present in the operation room that filmed the entire surgery from a third person view. Complex surgeries are often broadcasted like this, as it allows doctors from all over the world to follow along and learn new procedures.

What made this broadcast revolutionary – besides the complexity of the procedure itself – is the fact that the HoloLens was also used to film footage from a first-person view. As if the spectators were looking through the eyes of Dr. Tollens. This allowed for a highly detailed view of the patient’s situation. On top of that, the viewers had “front row seats” to follow the operating actions and procedures.

Thanks to the built-in cameras of the smartglasses device and its ability to connect to WiFi, live footage of both the real-life environment and the holographic information projected onto it, were shared. As the spectators were able to see the holographic projections, Dr. Tollens could easily provide extra information, and guide them with the holographic visuals.

Image by Imelda Hospital Bonheiden
Close collaboration with BD

BD is a global medical technology company that is advancing the world of health by improving medical discovery, diagnostics and the delivery of care. They strongly believe in the potential of the HoloLens and introduced us to the Imelda Hospital. Besides this project, together we’re investigating the use of the HoloLens for the following four purposes:

  1. Holographic medical imaging
  2. Training and education
  3. Bringing in remote experts
  4. 3D monitoring of anonymous patient data

Interested in learning more about this procedure, or other applications in the Healthcare industry? Save your spot for our webinar!