We’re bringing the HoloLens 2 to the operating room!

Eric Erdmann, Development Manager, mediCAD

As soon as the anesthesia takes effect, the work begins for the surgeons. Every move must be precise because the shorter the operation, the better it is for the patients. To achieve this, orthopedic procedures, especially the selection and alignment of prostheses, are planned in advance on the PC. The company mediCAD Hectec provides the corresponding software for this. But how helpful would it be if doctors could take the three-dimensional planning into the operating room? The answer: very helpful. That’s why the company has developed the solution mediCAD MR. This allows surgeons to access their own planning with the HoloLens 2 during the operation – and interact with it in a sterile environment.

The Challenge: Bringing the Planning into the Operating Room

The everyday life of a clinic is stressful. For many orthopedic surgeons, the working day often begins at eight in the morning with the first surgery. Then there’s surgery, followed by consultations, rounds, and planning for the next operations. This goes on until the evening – and even though the pace is high, doctors never forget that every person is individual and every operation is different. Therefore, proper planning is essential. “With all the solutions we develop, we are always in constant dialogue with experts from orthopedic surgery,” says Eric Erdmann, Development Manager at mediCAD Hectec. “Our goal is to create solutions that seamlessly integrate into the clinic’s everyday life, so that they can best support the planning and execution of surgeries.”

Preoperatively, the company already offers a way to plan orthopedic procedures in three dimensions with mediCAD 3D. Doctors can align implants and screws even more precisely than with 2D planning tools. However, once they are in the operating room, accessing the planning becomes difficult. “Some actually print out the planning and hang it in the operating room. But the human body is sometimes more complex than what fits on a DIN-A3 sheet,” says David Würdinger, Marketing & Product Manager. “In surgery, operating room monitors are also used. The problem is the time involved: you have to step away from the patient, take off your gloves, operate the PC, look at the planning, but at that moment you don’t have the patient in front of you, so you have to put on new sterile gloves, approach the patient again to continue where you left off. That’s cumbersome.” For this reason, the company mediCAD Hectec wanted to develop a better solution. What they started as a research project in 2018 is now a reality and is called mediCAD MR.

The Solution: HoloLens 2 Makes Intraoperative Planning Usable

MR – the abbreviation stands for Mixed Reality. This requires the right hardware, such as HoloLens 2, to project objects as holograms into the users’ field of view. Objects, such as the planning of an operation, for example. “After completing the surgical planning in mediCAD 3D, you press a button and the data is prepared for use on the HoloLens 2,” says David Würdinger. “With our assistant app for mediCAD MR on the PC, you can additionally enrich the data, for example with X-rays or with instructions for a new implant that you use for the surgery.” Once the data is prepared, a QR code is generated – viewed through the HoloLens 2, the planning images open up in the field of view of the surgeons.

“Those who have worn the HoloLens 2 in the operating room see it as a true game-changer. They have the planning images in a 1:1 scale, exactly as the patient lies before them, three-dimensionally in sight,” says Eric Erdmann, from conversations with doctors who already use the solution. “They can zoom into tissue and bone structures, view each layer individually, and rotate the view around any axis.” All of this happens sterilely, because the operation is controlled by voice and gesture. “We plan our solutions so that they can be seamlessly integrated into the workflow of clinics. With mediCAD MR and HoloLens 2, we have once again succeeded in doing so,” summarizes David Würdinger.

A new feature that will be available soon is remote access. This allows people with the corresponding access to follow the images streamed from the operating room live and in first-person view by the HoloLens 2. It is also possible to communicate with the operating doctors. “This opens up completely new possibilities,” says David Würdinger. Other specialists can provide advice during complicated surgeries. When using new implants, experts from the implant manufacturers are often present. “Currently, they have to travel extra for five or ten minutes, in which they are needed for the surgery. Remote access greatly simplifies this workflow,” says David Würdinger.

The solution also offers great potential for training future surgeons. Especially special and therefore rare surgeries can be recorded with mediCAD MR. Students can then discuss the operation step by step with their teachers afterwards.

Currently, the solution is still running on-premises in clinics. However, the goal is to leverage the full potential of the Azure cloud platform in the future. “Data protection, security, and reliability are of the utmost importance in clinics,” says Eric Erdmann. “Microsoft meets these requirements for us, even in the cloud, for example with services like Azure Confidential Computing for data protection. We have a huge cloud toolbox here that we can use to further develop the solution. We will use it in the future.”

“We plan our solutions so that they can be seamlessly integrated into the workflow of clinics. With mediCAD MR and HoloLens 2, we have once again succeeded in doing so.”

David Würdinger, Marketing & Product Manager, mediCAD

BACK TO THE OVERVIEW