Peoria, IL,
20:10 PM

OSF HealthCare using virtual reality to view heart disease

Pediatric cardiologist transforming medical imaging

What if medical imaging and visualizing human anatomy were as easy, immersive, and intuitive as playing a video game? The latest project taken on by the Advanced Imaging and Modeling (AIM) program at Jump Simulation & Education Center, an OSF HealthCare facility, is working to achieve just that. AIM has nearly perfected the refined, manual process of converting traditional medical images (MRI and CT scans) into digital formats, allowing doctors and surgeons to view three-dimensional anatomic models for medical decision making, pre-surgical planning, and patient education.

In fact, hospitals from coast to coast are sending Jump Sim medical images of congenital heart defects and other abnormal conditions for 3D printing. Jump provides this service for free when patient specific medical decision making is involved in hopes of giving physicians and surgeons improved understanding of each patient’s clinical scenario. The ultimate goal is to utilize intuitive 3D tools to improve the clinical outcomes and standardize interpretations of anatomical images across all clinical specialties.

Now, AIM is taking digital formats of medical scans and plugging them into evolving visual technologies eliminating the need for a physical model and shrinking the time it takes to view a complete 3D image. It's the belief of Dr. Matthew Bramlet, Director of the Advanced Imaging and Modeling program at Jump, that virtual and augmented reality will revolutionize how radiologists and clinicians look at anatomy.

Physical 3D Models Vs. Digital Models

Just as there was not an existing process for converting medical images into a digital format, there was also not software to translate imaging data to be compatible with virtual environments like augmented and virtual realities. Engineers at Jump created a prototype program to view 3D images of hearts and other parts of the human anatomy for the HTC Vive, a virtual reality headset.

Dr. Bramlet and the engineers on the AIM team found there are only so many ways 3D printed heart models can be viewed. They believe augmented and virtual realities allow for more flexibility to immerse oneself into the entire image and expand viewing capabilities. The next step was giving surgeons the opportunity to view anatomy using the Vive and collecting their feedback. 

“As one who does not have a background in video games and this type of technology, the HTC was fairly easy to grasp,” said Dr. Karl Welke, a pediatric congenital heart surgeon with Children’s Hospital of Illinois. “It puts you into a more intimate relationship with the heart and you can manipulate it in ways that you can’t do with a typical medical scan. As a surgeon, I can understand what I’m going to see in the operating room in much greater detail than I could before.”

Other surgeons found there to be educational value in using the Vive to view human anatomy. “I could have surgical residents view virtual models of cancer and ask them to practice removing tumors before going into surgery,” said Dr. Richard Anderson, thoracic and cardiac surgeon. “This would give residents more experience before operating on a real patient and prevent errors in the future.”

The Future of Medical Imaging

Dr. Bramlet and the AIM team see a future state where clinicians are truly immersed inside the virtual reality environment, and interact with it in ways they were unable to before. They also see the potential for bringing people together inside virtual and augmented realities for group learning and comprehension.


I’ve been doing pediatric cardiac congenital MRI for nearly nine years, and I strongly believe that viewing medical images using immersive, visual technologies is not a fad. It’s the direction of the future. We are putting tools in clinicians’ hands that they didn’t realize they needed and helping them fulfill their potential in medical decision making with higher efficiency and quality so that we can improve health care.
Dr. Matthew Bramlet, Director of Advanced Imaging and Modeling at Jump Simulation & Education Center.