Successful Patient Incident Investigations of Intelligent and Artificial Intelligence Systems

Investigating patient harm or death involving intelligent or AI-based devices and systems requires specialized approaches. These technologies form a complex System of Systems, where devices, software and firmware interact in dynamic and often unpredictable ways, influenced by configurations, updates and application settings. Semi- and fully autonomous systems further complicate causality assessments. Effective investigation demands rapid, skilled system analysis and documentation to determine root causes and implement timely remediation. A key challenge is preserving the “as-was” state of all involved technologies and the incident environment before potential reconfiguration or automatic updates occur. Investigators must act fast to document and secure components—such as smart infusion pumps, order entry systems, and integrated EHRs—since these systems are highly interdependent and subject to asynchronous changes by vendors, users, clinical engineers, biomedical technicians, and IT professionals. Once root causes are fully understood, necessary remediation will likely require well-documented technical interventions and both clinical and technical specialist training. This session outlines essential steps and success factors for conducting thorough, timely forensic investigations and remediation in today’s AI-driven healthcare environment.