OSF HealthCare Newsroom

Mapping the future for early diagnosis of rare diseases

Tue, 20 Aug 2024 13:19:57 -0500

Rare diseases might seem individually uncommon but collectively 1 in 10 people across the globe have a rare disease according to the World Health Organization. Undiagnosed rare diseases pose a significant challenge in health care, often leading to delayed treatment and poorer patient outcomes.

When specialists or primary care providers are confounded by a patient’s symptoms, it can send them back to their medical books. But, Adam Cross, MD, a pediatric hospitalist and clinical informaticist who leads the Children’s Innovation Lab at the Jump Trading Simulation & Education Center, is collaborating on research to better help primary care providers make a rare disease diagnosis quickly. The goal is to create an AI-backed, point-of-care support tool that can prompt testing for confirmation for the most relevant diagnoses.

Dr. Cross says typically genetic testing is the only way to confirm a rare disease. That testing is expensive and not readily available in primary care settings, particularly in rural communities. Patients have long waits and often must travel in hopes of getting a diagnosis.

A Jump ARCHES grant is funding a collaborative effort with the University of Illinois Urbana-Champaign (UIUC), with co-lead researcher Jimeng Sun, PhD, to use generative AI and medical knowledge graphs to improve diagnosis of rare diseases.

“We are trying to create a model that will take the notes that are already being written by primary care physicians and look for unique sets of signs and symptoms that might suggest a patient has an undiagnosed rare disease.” Dr. Cross adds, “That approach can empower the physician with information regarding those diseases and prompt further testing if they choose to do so.”

Dr. Cross believes rare diseases, despite their complexity, leave identifiable patterns within electronic medical records. The project introduces Automated Rare Disease Mining (AutoRD) to help unearth patterns when analyzed using machine learning models and mapped using knowledge graphs. He believes the Auto RD method can potentially enable the early detection of these diseases. These patterns could be drawn from clinical signs and symptoms and medical history, in addition to demographic features such as race/ethnicity, age and geographic location.

Researchers are making progress

In one year, researchers have been able to develop robust machine learning models to extract signs and symptoms of rare diseases from notes within electronic medical records that have been stripped of personal, identifying information. They’ve also been constructing comprehensive knowledge graphs to visually map the relationships between various medical conditions and symptoms of rare diseases from existing resources.

“We did this so that we could give people a better visual example of just how complex these diseases are and how they're all connected,” Dr. Cross observes while viewing an early version of a knowledge graph created by researchers that looks like a celestial constellation. “It's kind of beautiful when you see the simplicity and the complexity of all the different symptoms, but the simplicity of the structure. It gives you a sense of just how incredible these new technologies really are in terms of helping us find patterns in the chaos.”

The effort involves sophisticated machine learning approaches that have changed with lightning speed, even since Dr. Cross and his UIUC colleagues began their work.

“Even in that year, there have been more advanced and more powerful methods that have come out that we've since adopted, and we continue to improve our own methods based on what's being developed globally.”

The research is leveraging the abilities of a supercomputer at UIUC which has a federal health privacy compliant section for secure data storage and processing of massive amounts of health care data. The next focus is to work on the predictive ability of the models. Dr. Cross is convinced that dynamically linking patient-specific data with the broader context of medical knowledge on rare diseases can uncover patterns and correlations that might otherwise remain hidden.

As the approach is refined and improved, he also doesn’t rule out the possibility of discovering new rare diseases.

“As we go along, it is certainly possible with the technology that we've created, maybe we'll start to see patterns of signs and symptoms that don't have a known diagnosis. Or maybe there are subtypes of diseases that are out there that really seem to have their own unique progression, and we could also maybe discover those in subsequent phases of the project.”

Dr. Cross, who also serves as an assistant professor at the University of Illinois College of Medicine in Peoria (UICOMP) and is an adjunct professor at UIUC, says he and his fellow researchers have already published one scientific paper, and another has been submitted, showing the results of their research so far.

Video clips with Dr. Adam Cross

B-roll of knowledge graphs for rare disease diagnosis

Emergency department overcrowding: Can AI, predictive modeling and simulation fix it?

Fri, 30 Jun 2023 11:50:20 -0500

It seems almost everyone has a story about how long they had to wait to receive care in an emergency department due to overcrowding. In fact, research shows patient congestion is one of the main factors threatening efficiency, safety and quality of care.

The pressure is even greater following the COVID-19 pandemic, as more people feel comfortable returning to hospital emergency departments at the same time many health systems are dealing with a nursing shortage. William Bond, MD, is an emergency department (ED) physician at OSF HealthCare Saint Francis Medical Center in Peoria, Illinois. He also directs simulation research at Jump Simulation, a collaborative effort between OSF and the University of Illinois College of Medicine Peoria (UICOMP).

Dr. Bond and fellow researchers, including co-lead Hyojung Kang, PhD, a visiting assistant professor at the University of Illinois Urbana-Champaign (UIUC), will use a nearly $100,000 Jump ARCHES grant to develop innovative models aimed at reducing ED wait times.

“To acknowledge that suffering (in the waiting room) to use compassion, which is part of us at OSF HealthCare, and to address those needs as quickly as we can; to acknowledge that timeliness is part of the quality of care and we really want to have as timely of care as we can for our emergency department patients."

Improving time to treatment

The project is called: STREAM-ED: Simulation to Refine, Enhance and Adapt Management of Emergency. Dr. Bond explains his team is creating models to predict short-term, mid-range and long-term demand using historic data in de-identified electronic medical records (EMR). The goal is to combine machine learning prediction, discrete event simulation (a method to test processes and interventions ideally prior to intervention) and optimization techniques to determine best possible operational changes in emergency department management.

“We can say that based on past and current data inputs, here's where we think we'll be in the next 12, 24, to 72 hours in the emergency department … and of course, the further out you go in time, just as with weather forecasting, the more the uncertainty grows.”

Assistant professor Kang says the EMR information leveraged by researchers to create predictive models includes chief complaints, acuity levels, whether a patient was discharged, and timestamps collected throughout the patient’s time in the emergency department. They’ll also use data about physical resources and providers, including nurses and technicians who deliver assessments or care in different pods within the emergency department.

Kang specializes in discrete simulation, which provides a layered analysis of non-linear relationships among factors such as patient flow, availability of resources and operational policies that influence where patients are placed and for how long. The process provides a more comprehensive understanding of the way the system performs.

Dr. Bond says it also offers a way of testing interventions and timing without having to do it in real life.

Dr. Bond adds, "Instead, we may find that staffing an area with a more balanced team is the thing to do, staffing the team earlier in the day or later in the day. These types of things may make significant changes in our ability to care for patients.”

Running those scenarios will help identify high-reward interventions that can make the biggest impact with the fewest resources to increase efficiencies that can also help providers from feeling burnt out.

There have been studies that use forecasting and modeling approaches in the past, but assistant professor Kang says their practical application and integration into real-world operations have been limited. The project should result in helping decision-makers understand feasible actions they can take to improve emergency department flow.

“Our research team aims to empower ED leaders with the necessary, data-informed tools to navigate the complexities of resource allocation, making a tangible difference in the daily functioning of the ED.”

Dr. Bond says time-to-treatment will be a key metric for success because it is such a critical indicator of patient and medical provider satisfaction.

Interview Clips

Saint Francis Medical Center emergency department B-roll

Common medical statistics often wrong or misleading

Thu, 17 Nov 2022 16:17:06 -0600

A simple, yet revolutionary new statistical technique enables better assessment and implementation of many tests and predictive models, leading to greater patient benefits. Faulty assumptions in some widely used statistics can lead to flawed predictive model implementations that impact patient care. This can be corrected with a novel, utility-based approach (“u-metrics”), according to a study in the IEEE Journal of Biomedical and Health Informatics, co-authored by Dr. Jonathan Handler, senior fellow for Innovation at OSF Healthcare.

What’s wrong with the classic statistics?

Predictors of whether something will or will not happen in the future are used to facilitate care. Classic statistics to assess these predictors and guide their implementations include sensitivity, specificity, and positive and negative predictive values. These are based only on counts of how often the predictor was right or wrong. The article notes that these classic statistics make assumptions that don’t apply to many (probably most) real-world scenarios. Statistics based on faulty assumptions might suggest that a predictor will yield great benefit to patients even though the real-world performance will prove disappointing or even harmful. The result? Too often, busy health care workers must suffer through frequent false or useless alarms that they soon learn to ignore (“alert fatigue”).

For example, a prediction system might incorrectly trigger an alarm, claiming that a patient who is healthy has a dangerous infection. It might also correctly trigger an alarm for a patient with a dangerous infection even though the team is already addressing the issue. In each case, the alarm adds no value and distracts the care team away from other important work. Worse, in the case of a correct but unhelpful and distracting alarm, classic statistics inappropriately “take credit” for a correct prediction even though the alarm created more harm than benefit. This is because classic statistics assume that correct predictions are always helpful and every correct prediction is equally helpful, even though, as the authors note, those assumptions are commonly not the case.

A new and better approach

To address these challenges, the authors created u-metrics, an intuitive and comprehensive solution that does not rely on assumptions that rarely apply in the real world. Unlike classic statistics, it does not take a one-size-fits-all approach. Instead, it assigns to each prediction only the credit it deserves, and categorizes each prediction based on the benefit or harm created rather than its correctness.

“There has been some limited acknowledgement that the assumptions of classic count-based statistics commonly do not apply and a few partial fixes have been proposed. However, to our knowledge, this is the first comprehensive evaluation of the assumptions required by these classic statistics, and more importantly, the first comprehensive fix to the problem.” Dr. Handler added, “We believe that using u-metrics to guide the development, selection, and implementation of these types of predictors will benefit patients, providers, and health systems.”

In addition to clinical predictors, u-metrics can be used to assess any system that provides yes or no responses, from weather alerts to stock market predictions.

How can u-metrics help address alert fatigue?

By better informing the selection of predictors and their implementations, u-metrics might reduce the likelihood that a health system will choose and operationalize a predictor that fires too many useless alerts. The paper also describes “snoozing,” an implementation technique to dramatically reduce false and nuisance alerts in many cases. Snoozing is when an alarm is automatically or manually silenced for a period of time after it fires. Although manual snoozing of sensor alarms is common in ICUs, its use has not been well studied for predictive alarms. The paper notes this might be due to the inability of classic metrics to correctly assess the impact of snoozing. The u-metrics solution correctly assesses the impact of snoozing because it will neither reward nor penalize the system for suppressing alerts that were correct but would create distraction or harm if fired. The paper also describes a method to identify optimal snooze times. Properly applied, snoozing could reduce alert fatigue and increase the likelihood that clinicians will respond to true alarms.

Looking for more?

Dr. Handler provides an accessible explanation of u-metrics in his blog: Shocking new discovery: recall and sensitivity are not the same! as well as an explanation of the techniques to reduce false positives in his blog: Avoiding false alerts: snoozing ≠ laziness.

Healthleaders Media recently hosted a panel discussion about artificial intelligence featuring Dr. Handler. Here’s a link to the webina r. Dr. Handler discusses the importance of using better metrics when developing predictive models at (36:34) minutes into the webinar.

Dr. Handler leads OSF HealthCare’s Clinical Intelligence lab, with a mission to help patients get correct diagnoses and optimal therapy as quickly as possible. Visit osfhealthcare.org/innovation to learn more about how OSF HealthCare is working to transform health care.

Video clips with Dr. Jonathan Handler, senior fellow for Innovation at OSF HealthCare (from Healthleaders Media webinar on Medical Use of Artificial Intelligence)