A new generation of 3D Virtual Reality (VR) imaging technology at OSF HealthCare is giving surgeons a clearer view of cancer than ever before – and in some cases, completely changing surgical plans. For OSF HealthCare thoracic surgeon Richard Anderson, MD, that level of insight recently made the difference between removing part of a patient’s chest wall and preserving it entirely.

Dr. Anderson, chair and Norman Estes Professor of Surgery at University of Illinois College of Medicine Peoria, says the leap from standard imaging to 3D modeling is transformative for surgeons.

A better view

“So the real difference between a 2D and 3D is, just imagine looking at something on a piece of paper with two different pictures, looking at it from side to side, as opposed to having a model in your hand and being able to take that model and move it all around in all different directions, upside down, inside out, even open up things within the model and look at it. It’s really the difference between 2D and 3D.” Dr. Anderson adds, “It’s really quite impressive.”

The technology allows his team to isolate tumors, peel away layers of anatomy, and examine the intricate relationships among bones, vessels, and muscle before they ever enter the operating room. That clarity is especially crucial in complex thoracic oncology, where anatomy is delicate, variable and easily injured.

For Dr. Anderson and others who use 3D imaging, it increases the surgeon’s confidence.

“In the more complex oncology cases, it can make a big difference. It can help the surgeon really identify what structures they have to worry about during the operation, what structures they might have to sacrifice, before they actually do the operation and gives them a little bit more level of confidence going into the operation than probably they would have before.”

A tumor in a rare location

That level of detail proved critical in a recent complex thoracic oncology case. Thoracic surgery presents unique challenges because of the delicate and complex anatomy of the chest.

“The arteries aren't made the same, and so they're easier to tear. They're much more delicate. Lung tissue is sometimes like tissue paper, too. It’s easy to injure and tear. You have got to be very careful when you do the dissections in thoracic surgery. That's what makes it so challenging.”

Dr. Anderson treated Jake Wilson, a 69-year-old retired farmer and former prison guard from Central Illinois. Wilson’s case was challenging. He was diagnosed with a rare form of lung cancer and in August of 2025 began a series of chemotherapy and radiation to shrink the aggressively growing tumor.

Still, as surgery approached, Dr. Anderson was challenged with removing a nearly six‑centimeter tumor (about the size of a small egg) situated high in the chest, closely hugging the first three ribs. Standard 2D scans suggested the cancer had invaded the bone – a scenario that would typically require removing three ribs to ensure the cancerous tumor was completely removed.

Seeing is believing

But when Wilson became the first adult patient with lung cancer to have his case analyzed using advanced 3D modeling and virtual reality, the picture changed dramatically. The technology revealed that his ribs were not invaded at all. The tumor, though large and aggressive, was confined to muscle.

Sister M. Pieta Keller, Connor Davey, and Reid Jockisch, innovation engineers at Jump Simulation worked to segment the tumor for 3D viewing in virtual reality. Sister M. Pieta recently showed Wilson the 3D rendering. It was something he’ll never forget as she displayed his chest wall anatomy in vivid color and manipulated the images to reveal why Dr. Anderson could spare all three ribs.

“That was one of my biggest fears was losing my ribs,” Wilson explained. “Just the concept of that,” he said as his voice trailed off. Sister M. Pieta explained, “And your recovery and your pain would have been a lot higher,” to which Wilson responded, “I’m sure.” Sister M. Pieta added, “So this was quite the gift.”

After years of lung scans due to his history of smoking, seeing the vivid digital model after his successful surgery made the size and seriousness of the tumor undeniably real. Having only retired from farming six months earlier, Wilson admitted that going into the surgery, he was really scared.

“That put the fear of God into me, needless to say, because I’m an active person. I’m a farmer. I thought, ‘I could see being debilitated for the rest of my life.’”

A recovery that surprised everyone

The new imaging technology altered not only the surgical approach, but Wilson’s experience afterward. Instead of the severe pain typically expected after chest wall removal, his discomfort was minimal; it was largely limited to the area where an intercostal muscle was removed.

Wilson required only mild pain medication and never needed supplemental oxygen after losing half of his lung, something his care team described as highly unusual.

Even more astonishing, follow‑up imaging showed his remaining lung tissue had expanded far faster than expected. His post‑surgical biopsies revealed a remarkable result: the tumor was completely non‑viable (dead) at the time of removal.

A message about screening

Today, Wilson is cancer‑free and sharing his story. He credits many people along the way, including his primary care doctor, who five years ago, when Wilson was still smoking, encouraged him to get an annual CT screening to detect lung cancer.

“I thought, ‘I’m the one that’s stupid enough to do this to myself I ought to at least be smart enough to use a little preventive maintenance.’” He added, “And thank God I did because I might already be gone.”

But above all, Wilson returns to the strength of his faith which he credits with leading him to Dr. Anderson and his care team at OSF HealthCare Cancer Institute in Peoria and OSF Cancer Center in Bloomington. He is sharing his story to show the power of prayer and faith, and to instill hope.

“I know for a fact that the most important thing a person can have in these ordeals is hope because without hope you give up. And this is one time you can’t afford to give up because if you give up, you’re done,” Wilson declared.

For Dr. Anderson, the case represents the promise of 3D technology not just to guide surgeons, but to change outcomes. He calls that, “A good day at work.” And for Wilson, it represents something even bigger: a second chance he intends to use to encourage others to seek preventive screenings, face fear with hope and share gratitude for every day.

Video clips with Dr. Richard Anderson

Video clips with oncology patient Jake Wilson

B-roll of 3D imaging and Dr. Anderson

Imagine being a heart surgeon preparing for a complex case. Traditionally, you rely on stacks of 2D images from CT scans to build a mental picture of your patient’s anatomy. But the heart isn’t static – it twists and pumps blood in a rhythm unique to each person. Now, thanks to a breakthrough at OSF Innovation, surgeons can “see” their patient’s heart beating in 3D before ever stepping into the operating room.

This innovation is called the 4D Heart. Unlike standard 3D models, which capture a single snapshot in time, the 4D Heart recreates every phase of a patient’s cardiac cycle, allowing doctors to watch their heart beat virtually. And it isn’t just a generic animation. Rather, it is the creation of a digital twin.

“This is taking a CT scan of a patient, a retrospectively gated CT scan, and converting a patient’s own heart – in each phase of that cardiac cycle – and making it beat in a virtual environment,” says Matthew Bramlet, MD, who directs the Advanced Image and Modeling (AIM) Lab at OSF Innovation. The lab is focused on improving the understanding of complex anatomy for surgeons and diagnosticians. Dr. Bramlet also specializes in congenital cardiac MRI for OSF HealthCare Children’s Hospital of Illinois and is an associate professor of Clinical Pediatrics at the University of Illinois College of Medicine in Peoria and the University of Illinois Grainger College of Engineering in Urbana-Champaign.

Dr. Bramlet’s efforts began in 2018 with manually segmenting parts of the heart so it could be viewed in virtual reality. It continued with an infusion of grant money two years ago. Using an Innovation for Health (IFH) grant with Bradley University in Peoria, Illinois, a team used machine learning to convert standard medical images into a sequence of 3D models that, when played sequentially, create a 3D beating heart – the 4D heart. The initial process took an entire summer.

That made the technology nearly impossible to use in real world clinical care. But in late 2024, researchers achieved a major breakthrough. With the help of engineers at the University of Illinois Urbana-Champaign, AIM lab engineers trained neural networks on hundreds of high-quality CT scans and automated the entire process. Now, what once took months can be done with a single click.

“Have you ever seen a flip book where you see a drawing move? That’s what we’re doing with these 3D models in a digital, virtual space.” Dr. Bramlet explains, “We’re just turning them off and on in rapid succession until we have that beating heart.”

For surgeons, the benefits are game-changing. They can see not only the anatomy but also how it changes with every beat. This is especially critical for conditions such as hypertrophic obstructive cardiomyopathy, where the problem lies in how the heart contracts. Even more surprising, the AI began rendering valves — structures notoriously difficult to see on CT scans.

“Now we can see that valve which we really couldn’t see before. So, repeat valve procedures are very valuable for this. TAVR or any valve in the cath lab, we’re finding good use with these as well as the hypertrophic cardiomyopathy cases.”

Hospitals across the country are taking notice, sending scans to OSF and receiving beating heart files in return. Some don’t even have the technology to view them yet. Dr. Bramlet says OSF Innovation is pushing the boundaries so far forward, some other health systems will need to get on board with the technology to be able to visualize the data being sent to them.

For now, the ability to view a 4D Heart is already changing how surgeons think about the body’s most vital organ. In some cases, the new view is confirming a surgery plan. In other cases, including a most recent pediatric heart case from another hospital, it changed the approach. Dr. Bramlet says as surgeons interact with the models, they are amazed.

He shares, “I see those lightbulbs go on and there’s nothing like being able to observe and interact with these 3D models in a format, and especially a 4D model, in a way they’ve never been able to see them before, and it will become the standard of care, I have no doubt.”

The ultimate vision is bold: a patient enters the CT or MRI scanner, and their digital twin heart comes out the other side – ready for analysis, surgical planning and even analysis of AI-driven measurements like ejection fraction or blood flow strain to help with diagnosis and a treatment plan.

How long until that happens? Hard to say because as Dr. Bramlet points out, “seeing is believing,” and it might take time for health systems to invest in the technology to make the vision become reality. Government reimbursement for the care will also drive adoption. For now, the work is being covered through research grants, most notably; the Jump ARCHES endowment through the Health Care Engineering Systems Center at the University of Illinois Urbana Champaign and OSF HealthCare.

Dr. Mathew Bramlet

B-roll-4D Beating Heart

As we begin a new year at OSF HealthCare, we’re reflecting on 2024 and the major strides in digital transformation to make access to care and health information seamless for our patients. At OSF, we continue to collaborate with our academic and business partners to leverage collective expertise and to responsibly use the power of artificial intelligence, always taking care to use guiding principles of what is fair, appropriate, valid, effective and safe (FAVES).

The efforts are highlighted here in our Top 10 Digital Health and Innovation stories of 2024 in no particular order.

1. OSF Unveils new 55+ primary care service in south suburban Chicago: Called OSF Complete Care 55+, this is a unique collaboration between OSF HealthCare Little Company of Mary Medical Center and OSF OnCall, the digital arm of the health care Ministry. When launched in early 2025, this primary care program will incorporate in-person visits, plus a mix of virtual care, digital health monitoring tools and 24/7 access to a care team focusing on those with chronic conditions.

2. Building an AI prepared workforce: OSF HealthCare leaders took just six weeks to develop mandatory training about generative AI for all 24,000 OSF Mission Partners (employees) because tools were already publicly available. Melissa Knuth, Vice President of Planning, outlines that to rise to the challenge, she worked with OSF Innovation to use mixed media to keep the content interesting and tapped generative AI tools to assist with the content creation.

3. OSF Digital Hospital patients in Rockford area say it’s the best medicine: OSF OnCall expands its digital hospital program to Rockford and receives rave reviews from some of its first patients. In addition, a well-known farm broadcaster shares her story about receiving acute-level care from the comfort of her own home and urges others who are eligible to consider the option.

4. OSF Innovation sets the table for solution to prevent cancelled surgeries: Two OSF HealthCare nurses turned to the OSF Innovation Studio team to help improve their Device Table software that serves as a database for implanted medical devices – offering information needed for surgical teams to reference for safety in surgery planning. We also reported on the creation of a spin-off company, OpenSurg, which will license and promote widespread sales of Device Table and related support features.

5. OSF HealthCare experts: ‘Establish guiding principles when developing digital care strategy’: OSF OnCall was a spotlight vendor and presenter at this year’s American Telemedicine Association’s annual Nexus conference in Phoenix, Arizona. OSF OnCall Senior Vice President Jennifer Junis advised during her presentation that health systems use guiding principles when developing their approach to digital health and to involve clinicians and patients when piloting new programs and digital tools. OSF HealthCare has also become a founding member of an ATA Digital Center of Excellence.

6. Mapping the future for early diagnosis of rare diseases: A Jump ARCHES grant-funded project with the University of Illinois Urbana-Champaign is using generative AI and knowledge graphs that map symptoms to detect rare diseases. Adam Cross, MD, who leads the OSF Children's Innovation Lab in Peoria, Illinois, wants to create a point-of-care tool for primary care providers to help patients get an early diagnosis of a rare disease.

7. Health economist: Connectivity and trust can expand telehealth: OSF OnCall hosted its first-ever digital health symposium which drew nearly 150 clinicians and vendors. Keynote speaker health economist and advisor Jane Sarasohn-Kahn stressed evidence-based digital technologies can improve health outcomes, but health systems need to build trust and expand community partnerships to increase engagements.

8. Fast-Tracking Hope: Rapid Intake at the OSF Cancer Institute: OSF HealthCare began 2024 with the opening of its new comprehensive Cancer Institute, offering advanced radiation therapy such as brachytherapy and proton beam therapy.  It also launched a new rapid intake system, developed by OSF OnCall and OSF Innovation to offer a single point of contact through OSF-HOPE (844- 673-4467) so cancer patients can be immediately connected to treatment and support services.

9. Piloting a smarter colonoscopy: OSF HealthCare St. Joseph Medical Center piloted artificial intelligence-backed colonoscopies, using a company called GI Genius, to improve detection of precancerous polyps. During the pilot, the adenoma detection rate (ADR) averages increased among all patients involved to as high as 12%, while individual physician ADR increased as high as 32.8%. The technology is now being implemented across the OSF Ministry.

10. Testing the path to better concussion diagnosis: An OSF Innovation lab-designed FlightPath app is now being tested by athletes at Illinois State, Bradley and Illinois Wesleyan Universities. The app only takes about two minutes to collect more than a million data points as a person tries to catch a hummingbird within a 3D space on a screen. The clinical trial could last two years. 

In the past decade, breakthrough treatments for cancer have emerged requiring a personalized approach.  The new OSF HealthCare Cancer Institute on the campus of OSF HealthCare Saint Francis Medical Center in Peoria, Illinois, will offer access to a variety of treatments under one roof when it welcomes its first patients next month.

A lesser discussed treatment that will be available is brachytherapy or implant radiation. Unlike external beam radiation, this treatment is delivered from inside the tumor and can deliver more intense doses of radiation at the tumor site. Brachytherapy is most commonly used to treat cancers of the female pelvis, prostate, breast, eye and skin. 

James McGee, MD, radiation oncologist at OSF HealthCare Saint Francis Medical Center has sub-specialized in brachytherapy. It is the most targeted radiation treatment type and preserves healthy tissue. 

“Placing a radiation source directly into the patient’s tumor allows for many advantages. In this way, the radiation is not going through normal tissues and has much less normal tissue effect.  At the same time, it creates the need to be very individualized and very creative.” 

The radiation is given with high dose delivery rates usually with a very small radiation source in a cable moving through an applicator or even a series of needles to the core of the malignancy. The devices are called temporary implants that are removed after the dose is delivered. Brachytherapy implants may be short-term (temporary) or long-lasting (permanent). The result is a higher cure rate for certain cancers than with external beam radiation alone says Dr. McGee.  

The OSF Cancer Institute will have a state-of-the-art brachytherapy surgical suite, fitted with a GE CT scanner on rails, specifically designed for the space. It will allow patients to stay under anesthesia during CT-guided applicator or implant insertion. A new MRI scanner is located directly outside the suite, creating efficient transfer for the patient, when MRI imaging will help, improving their overall experience. 

Better preparation through 3D models 

Dr. McGee sits in front of a 3D model made for a woman who had cervical cancer. It was developed using translucent silicone molding and graphite material to represent tumor growth. He explains that because of the need for precision application of the radioactive material, OSF HealthCare enlisted engineers at Jump Trading Simulation & Education Center to create patient-specific models for complex cancer cases. 

“We've actually been taking MRI scans from patients and in preparation for their treatment and we've created models that represent their normal anatomy and the anatomy of their cancer. Usually this is after initial treatment with chemotherapy radiation, probably for about five weeks.”

The use of brachytherapy reduces recovery time and side effects, and is often a shorter, more cost-effective treatment than traditional external beam radiation. There are many types of brachytherapy.  These can be categorized by the duration of the dose delivery - high dose rate, low dose rate, and permanent. They are also categorized by the technique used such as interstitial (in between cells of cancerous tissue), intracavity and surface application. 

 Dr. McGee looks forward to being able to enhance brachytherapy procedures by having the operating room, imaging, and treatment delivery system integrated in the same space at the OSF Cancer Institute. 

Video Clips with Dr. James McGee courtesy of WTVP-PBS

B-roll of Brachytherapy-Courtesy WTVP-PBS