Imagine being a heart surgeon preparing for a complex case. Traditionally, you rely on stacks of 2D images from CT scans to build a mental picture of your patient’s anatomy. But the heart isn’t static – it twists and pumps blood in a rhythm unique to each person. Now, thanks to a breakthrough at OSF Innovation, surgeons can “see” their patient’s heart beating in 3D before ever stepping into the operating room.

This innovation is called the 4D Heart. Unlike standard 3D models, which capture a single snapshot in time, the 4D Heart recreates every phase of a patient’s cardiac cycle, allowing doctors to watch their heart beat virtually. And it isn’t just a generic animation. Rather, it is the creation of a digital twin.

“This is taking a CT scan of a patient, a retrospectively gated CT scan, and converting a patient’s own heart – in each phase of that cardiac cycle – and making it beat in a virtual environment,” says Matthew Bramlet, MD, who directs the Advanced Image and Modeling (AIM) Lab at OSF Innovation. The lab is focused on improving the understanding of complex anatomy for surgeons and diagnosticians. Dr. Bramlet also specializes in congenital cardiac MRI for OSF HealthCare Children’s Hospital of Illinois and is an associate professor of Clinical Pediatrics at the University of Illinois College of Medicine in Peoria and the University of Illinois Grainger College of Engineering in Urbana-Champaign.

Dr. Bramlet’s efforts began in 2018 with manually segmenting parts of the heart so it could be viewed in virtual reality. It continued with an infusion of grant money two years ago. Using an Innovation for Health (IFH) grant with Bradley University in Peoria, Illinois, a team used machine learning to convert standard medical images into a sequence of 3D models that, when played sequentially, create a 3D beating heart – the 4D heart. The initial process took an entire summer.

That made the technology nearly impossible to use in real world clinical care. But in late 2024, researchers achieved a major breakthrough. With the help of engineers at the University of Illinois Urbana-Champaign, AIM lab engineers trained neural networks on hundreds of high-quality CT scans and automated the entire process. Now, what once took months can be done with a single click.

“Have you ever seen a flip book where you see a drawing move? That’s what we’re doing with these 3D models in a digital, virtual space.” Dr. Bramlet explains, “We’re just turning them off and on in rapid succession until we have that beating heart.”

For surgeons, the benefits are game-changing. They can see not only the anatomy but also how it changes with every beat. This is especially critical for conditions such as hypertrophic obstructive cardiomyopathy, where the problem lies in how the heart contracts. Even more surprising, the AI began rendering valves — structures notoriously difficult to see on CT scans.

“Now we can see that valve which we really couldn’t see before. So, repeat valve procedures are very valuable for this. TAVR or any valve in the cath lab, we’re finding good use with these as well as the hypertrophic cardiomyopathy cases.”

Hospitals across the country are taking notice, sending scans to OSF and receiving beating heart files in return. Some don’t even have the technology to view them yet. Dr. Bramlet says OSF Innovation is pushing the boundaries so far forward, some other health systems will need to get on board with the technology to be able to visualize the data being sent to them.

For now, the ability to view a 4D Heart is already changing how surgeons think about the body’s most vital organ. In some cases, the new view is confirming a surgery plan. In other cases, including a most recent pediatric heart case from another hospital, it changed the approach. Dr. Bramlet says as surgeons interact with the models, they are amazed.

He shares, “I see those lightbulbs go on and there’s nothing like being able to observe and interact with these 3D models in a format, and especially a 4D model, in a way they’ve never been able to see them before, and it will become the standard of care, I have no doubt.”

The ultimate vision is bold: a patient enters the CT or MRI scanner, and their digital twin heart comes out the other side – ready for analysis, surgical planning and even analysis of AI-driven measurements like ejection fraction or blood flow strain to help with diagnosis and a treatment plan.

How long until that happens? Hard to say because as Dr. Bramlet points out, “seeing is believing,” and it might take time for health systems to invest in the technology to make the vision become reality. Government reimbursement for the care will also drive adoption. For now, the work is being covered through research grants, most notably; the Jump ARCHES endowment through the Health Care Engineering Systems Center at the University of Illinois Urbana Champaign and OSF HealthCare.

Dr. Mathew Bramlet

B-roll-4D Beating Heart

A group of about 40 participants in a recent training session at the OSF HealthCare Athletic Training Center at Illinois State University braved high humidity in the domed field to get a refresher on emergency procedures for medical emergencies during high school practices or sporting events.

As part of a collaboration, athletic trainers from ISU and OSF HealthCare gave high school trainers and master’s degree students from ISU’s Athletic Training program a refresher on critical skills. The timing was planned to be aligned with the start of practices and the Illinois High School Athletic Association (IHSA) sports season on August 11.

Joe Whitson, former athletic trainer and OSF HealthCare manager of the office that provides physician athletic trainers, says they simulated some of the most common conditions trainers see on the field.

“We had various stations going on, whether it was splinting and spine boarding, treatment of the spine-injured athlete. Dr. Rai, one of our sports medicine physicians, was here as well to help supervise. He was here giving pointers, walking around the stations, also working with our athletic trainers, giving them pointers and tips on skills that they can utilize out in the clinical field.”

OSF provides athletic trainers to seven local high schools, which were represented in the training. Whitson says the training helped reinforce how to identify symptoms of various conditions.

“Hitting tips on hydration, hitting tips on how to recognize the difference between sickle cell anemia, heat stroke and rhabdomyolysis. Unfortunately, with the increase in high school participation, high school athletes participating in sports, obviously, the incidence of these types of episodes are increasing as well.”

Rhabdomyolysis is a serious condition where damaged muscle tissue breaks down, releasing its contents into the bloodstream.

Whitson says every year there are headlines of student athletes who have issues and are admitted to the hospital due to some of these conditions, so it was important to go over related symptoms and emergency procedures, not only to know how to respond, but also how to prevent medical emergencies.

High school athletes want to perform at their highest level, and in many cases, they’re pushing for a college scholarship, so Whitson says it’s important for coaches to have support from medically trained professionals.

“That’s really a benefit that we've been able to provide here at OSF within the community, to be able to have those athletic trainers as resources, so that the coaches at all of our high schools have a resource to be able to lean on that’s clinically and medically trained to be able to handle those types of situations.”

The training emphasized communication as participants switched off playing roles, serving either as the lead practitioner, a medical assistant or the athlete. The training even highlighted infant CPR in case of a situation that might occur in stands.

Whitson stresses the IHSA has put safety standards into place to prevent injuries and medical emergencies and he says athletic trainers provide education around those protocols.

“The athletic directors obviously are in charge and kind of house enforcement of some of those things for student athlete safety. And that's the great thing where these athletic trainers from OSF, are able to come in and be able to educate the athletic directors, the student athletes, the coaches, and give those speaking points that are critical to be able to take care of them.”

ISU’s athletic training department has been hosting emergency training events for several years to make sure its staff is ready to handle emergencies that arise. This year, under a new collaboration with OSF HealthCare, it was the first time it was open to other community members.

Joe Whitson

B-roll for OSF-ISU train the trainers for medical emergencies

Sixteen research projects are sharing more than $1.6 million in funding through the Jump ARCHES research and development program. Projects funded focused on novel automation in health data and patient experience, precision medicine, digital tools to enhance patient engagement in health, wellness and treatment, plus improved treatment and diagnosis of neurologic disorders. The latest request for proposals also encouraged supporting underserved populations.

The Jump ARCHES program is a collaboration between OSF HealthCare, the University of Illinois Urbana-Champaign (UIUC), and the University of Illinois College of Medicine Peoria (UICOMP). The funding supports research involving clinicians, engineers and social scientists to rapidly develop technologies, devices, and treatment approaches that improve outcomes and reduce costs.

Spring 2023 Project Awards

STREAM-ED: Simulation to Refine, Enhance and Adapt Management of Emergency
William Bond, MD, OSF HealthCare
Hyojung Kang, PhD, University of Illinois Urbana-Champaign

This study aims to develop practical models combining machine learning, discrete event simulation, and optimization techniques to improve emergency department (ED) resource utilization and address ED overcrowding, which is exacerbated by the COVID-19 pandemic and staffing shortages.

Prototype: Intelligent Regulatory Change Management System
Scott Lowry, MHA, CHC, CCEP, OSF HealthCare
ChengXiang Zhai, PhD, University of Illinois Urbana-Champaign

This study proposes an Intelligent Regulatory Change Management (IRCM) System that uses natural language processing and artificial intelligence to track and evaluate public policy actions governing OSF HealthCare. This will enable compliance professionals to identify critical changes and determine appropriate courses of action, reducing manual review and improving quality, safety, privacy risk management and efficiency.

Machine Learning of Standardized DICOM Metadata from Imaging Datasets
Matthew Bramlet, MD, OSF HealthCare
Brad Sutton, PhD, University of Illinois Urbana-Champaign

This project aims to develop a machine learning-based algorithm that can categorize image parameters directly from signal intensity variations of 2D medical images to enable efficient pipelines for medical image segmentation. The proposed algorithm is expected to estimate patient and image-acquisition information by utilizing machine learning methods in situations where the DICOM header fields are incomplete or unreliable, ultimately allowing for automated characterization of unknown 3D DICOM imaging datasets.

Machine-Guided Staging of Neuroblastic Tumors of Patient Specific 3D Models
Daniel Robertson, MD, OSF HealthCare
Brad Sutton, PhD, University of Illinois Urbana-Champaign

The OSF HealthCare Children’s Hospital of Illinois is using segmentation services to create 3D models of neuroblastic tumors for pre-surgical planning. The hospital aims to transition from 2D imaging to 3D modeling to increase the reproducibility of staging analysis, establish a new standard for segmented models of neuroblastic tumors and develop machine-guided tools that can improve upon and automate current recommended image-defined risk factors staging.

Toward Machine-Learned Aortic Arch Measured Diameters
Matthew Bramlet, MD, OSF HealthCare
Brad Sutton, PhD, University of Illinois Urbana-Champaign

The original project aims to automate the segmentation and clinical measurement of aortic arch diameters from MRI imaging. The researchers leading this project have successfully completed several steps, including de-identification and curation of datasets, manual segmentation and the development of a novel method for automatically analyzing each aortic arch with promising results, indicating correlation between the automated and clinically derived measurements.

A Field Experiment to Evaluate the Efficacy of Convenient Health Kiosks
Ann Willemsen-Dunlap, CRNA, PhD, OSF HealthCare
Ujjal Mukherjee, PhD, University of Illinois Urbana-Champaign

This proposal outlines a field experiment to evaluate the efficacy of health kiosks supported by community health workers (CHWs) in delivering first line preventive health screenings to rural and underserved communities. The project is intended to lead to large-scale development and deployment of health kiosks with the goal of positively impacting social determinants of health and long-term health status of those served.

Contextualizing Nursing Needs for Development of Retention-Support App
Sheryl Emmerling, PhD, Rn, NEA-BC, OSF HealthCare
Ann-Perry Witmer, PhD, University of Illinois Urbana-Champaign

The goal of this project is to address the high turnover rate of new nurses by providing a digital app that offers personalized nursing support. The Contextual Engineering (CE) paradigm will be used to assess the needs and values of first-year nurses, including those who have left their positions, to inform the development of the app in the first phase of the project, with the goal of stabilizing the nursing staff, improving the quality of service and reducing operating costs.

Community Health Café: Engaging Digital Innovation and Community-Based Resources to Enhance Health Equities in Underserved Communities
Scott Barrows, MA, OSF HealthCare
Joe Bradley, PhD, MA, University of Illinois Urbana-Champaign

The purpose of the community health café is to provide digital access to health and health care resources, including links for assistance with the social determinants of health, health education and connections to public health in underserved communities. The eventual goal is a Medicaid telemedicine option with OSF OnCall. This proposal aims to address critical needs of underserved residents in vulnerable communities and is crucial for their health.

AI-Powered Brain Tumor Segmentation
Matthew Bramlet, MD, OSF HealthCare
Zhi-Pei Liang, PhD, University of Illinois Urbana-Champaign

This project aims to enhance the detection and monitoring of brain diseases. Phase 1 of the project focuses on accurate delineation and segmentation of brain tumors using a combination of structural and molecular multimodal brain imaging data and deep learning. The proposed work includes developing brain atlases for AI-powered brain image analysis, computational tools for automated tumor detection and segmentation and evaluating potential clinical applications.

Optimizing Pharmacologic Management of Behaviors in Patients with Autism
Adam Cross, MD, FAAP, OSF HealthCare
Ravishankar Iyer, PhD, University of Illinois Urbana-Champaign

This proposal aims to provide physicians with a machine learning model that assists in selecting appropriate medication and dosage strategies for patients with Autism Spectrum Disorder (ASD). By incorporating patient history, genetic information and clinician notes, the model will dynamically adapt the treatment protocol as the patient progresses, ensuring optimal choices for improved behavioral symptoms with a high degree of confidence.

Predicting Medication Non-Adherence in Type 2 Diabetes
Mary Stapel, MD, OSF HealthCare
Hyojung Kang, PhD, University of Illinois Urbana-Champaign

Medication adherence is crucial for managing diabetes, but disparities exist, particularly among racial/ethnic minorities and those with lower socioeconomic status. This proposal aims to use data-driven models to identify high-risk individuals and areas for non-adherence to diabetes medication, develop and validate prediction models and implement and evaluate them in clinical practice.

Knowledge Graph Construction with Large Language Models to Predict DKA Occurrence and Severity
Adam Cross, MD, FAAP, OSF HealthCare
Jimeng Sun, PhD, University of Illinois Urbana-Champaign

Diabetic ketoacidosis (DKA) hospitalizes over 50,000 American children annually, with underprivileged and underserved children at higher risk. This proposal aims to develop a predictive model using patient-specific knowledge graphs generated from clinical data extracted through name entity recognition and language modeling. Clinicians can use the model to identify high-risk diabetic patients and prevent DKA.