AI in Healthcare: Balancing Innovation with Safety and Precision

At the University of Illinois Hospital, a patient misspelled a medication name while testing an AI tool, leading to incorrect side effect information when a nurse failed to double-check. This error nearly halted the project due to safety concerns. As healthcare seeks safe AI implementation, there is growing interest in tools that automate administrative tasks to reduce provider burnout and minimize risks. With a projected shortage of over 100,000 healthcare workers by 2028, there is strong pressure to adopt AI. However, experts emphasize careful implementation, noting concerns about accuracy, bias, and security.  

How AI could address a basic gap between supply and demand:

AI could help tackle a major issue in healthcare: the increasing number of older patients with complex, chronic conditions and the shortage of providers available to care for them, an expert explained during a panel discussion.  

As older Americans require more care, millennials, the largest U.S. generation, expect a more on-demand, consumer-friendly experience. However, with limited healthcare providers, meeting this demand is challenging. Training new doctors takes years, and having fewer doctors leads to more burnout, delayed care, and higher costs. 

AI can support clinicians in their workflows, potentially improving patient care. An algorithm developed by researchers at a major medical group, for example, saves around 500 lives each year by identifying patients at risk of sudden health declines.  

The technology could help reduce burnout and improve retention among clinicians by decreasing the time they spend on administrative tasks like note-taking. Providers have often said they spend hours on electronic records, which can negatively affect patient care. 

A vascular surgeon nearly left medicine due to the shift to electronic health records, finding it hard to collect information while listening to patients and missing non-verbal cues while focusing on a laptop. However, ambient documentation, where AI records conversations and drafts notes, has transformed the experience. 

 Investors, health systems focus on administrative burden:

Due to heavy administrative workloads and concerns about errors or bias in clinical decision-making models, AI products that address administrative issues are top priorities for adoption. Some investors are also focused on automating these operational tasks.  

The focus is on back-office automation to reduce workforce burdens, rather than clinical AI. Administrative AI is attracting more venture capital, with companies raising $2.4 billion in 2024 compared to $1.8 billion for clinical AI. This shift is likely due to fewer regulatory hurdles for decision support tools.   

There are significant opportunities to use large language models for drafting tasks, such as notes and nurse handoffs. Providers will review the AI-generated outputs before finalizing them, making quality evaluation easier.   

However, it’s important to carefully assess administrative tools using local patient data and ensure data protection and safety. Products like ambient scribes and revenue cycle management tools are considered safer options since they do not involve clinical decision-making.  

While AI has the potential to reduce provider burnout, health systems face challenges in getting providers on board, launching pilots, establishing governance, and rolling out products. For example, a health system initially targeted clinicians who struggled with documentation but received little interest. They then invited anyone interested to join the pilot, which resulted in a quick response.  

It’s essential to choose the right participants for testing, as some may not understand the requirements. Health systems should also clarify their goals for adopting AI, whether that’s financial savings, time efficiency, or improved patient satisfaction. Additionally, cybersecurity remains crucial, and organizations need to consider data protection, access, and agreements with developers to safeguard patient information effectively.