The healthcare industry is embracing large language models (LLMs), such as ChatGPT and GPT-4, with much more enthusiasm than expected. This is despite LLMs being relatively new and lacking safety frameworks to ensure their ethical use. While the FDA has released over 500 AI-powered medical devices, experts are calling for a safety framework specific to AI’s evolving impact on healthcare. In the meantime, healthcare organizations are developing their own guardrails to ensure they are using LLMs ethically and efficiently.
The University of Miami Health System and Phoenix Children’s Hospital are two such organizations that recognize the potential risks of LLMs and aim to develop responsible use guidelines. These guidelines involve educating staff on data immortality as it pertains to LLMs, ensuring people know that using any form of proprietary data to train LLMs is off-limits. Another crucial guardrail is staying away from use cases that involve diagnoses or clinical patient interactions to avoid privacy and reliability risks.
Experts recommend starting with piloting LLMs in nonclinical settings and focusing efforts on hospital operations. For example, health systems could implement LLMs to assist with appointment scheduling or answer patients’ questions about their bills, as these are relatively safe use cases that don’t involve protected health information.
While LLMs purpose-built for healthcare will inevitably make their way into the market, experts agree that healthcare organizations must wait for a regulatory framework before using the AI models that involve clinical patient interactions. Until then, healthcare organizations can refer to their internally created guidelines to ensure ethical use of LLMs.
Overall, healthcare organizations must balance innovation and risk when it comes to implementing LLMs. While LLMs carry some risks, experts believe their benefits outweigh their risks and are critical to alleviating healthcare’s inefficiencies.
