Regulating ICU AI: From Narrow Tools to Generalist Systems
Original Title: The regulation of artificial intelligence in intensive care units: from narrow tools to generalist systems
Journal: NPJ digital medicine
DOI: 10.1038/s41746-026-02535-3
Overview
Intensive care units represent highly data-intensive environments in healthcare, requiring continuous monitoring and rapid decision-making. While artificial intelligence has been explored for decades, its formal regulation as a medical device began in 1995. By May 2025, the number of approved artificial intelligence-enabled medical devices reached 1,016 in the United States. Many of these tools are designed for narrow, single-task applications such as interpreting radiological images or predicting sepsis. The emergence of generative artificial intelligence and large language models marks a shift toward generalist systems capable of managing multiple tasks. Currently, only two large-language-model-based medical devices have received approval globally, reflecting a gap between research and clinical implementation. This paper examines the regulatory challenges as these systems move from isolated tools toward integrated, autonomous entities.
Novelty
This research introduces a five-paradigm framework to categorize artificial intelligence applications based on their scope and scale of operation. The paradigms range from single-patient, narrow-task tools to unit-level orchestrating systems that manage multiple patients and devices simultaneously. A significant contribution is the distinction between device-centric regulation and the need for a system-of-systems approach. While paradigms one and two align with existing frameworks like the European Medical Device Regulation, paradigms three through five introduce complexities such as non-determinism and adaptive behavior that current laws do not fully address. The authors identify the concept of agentic oversight, where one autonomous system supervises another, as a potential solution for maintaining safety in highly dynamic environments. This framework provides a structured method for regulators to assess the increasing complexity and autonomy of clinical systems.
Potential Clinical / Research Applications
The proposed framework has direct applications in clinical governance and future research design. In clinical settings, the fifth paradigm of orchestrating artificial intelligence could automate administrative tasks such as generating discharge summaries or coordinating bed assignments across a unit. This allows healthcare professionals to focus on direct patient care. Research-wise, the framework identifies a path for testing generalist medical artificial intelligence through regulatory sandboxes, which provide controlled environments for evaluating adaptive models. Specifically, agentic systems could be utilized to manage complex ventilator settings or fluid resuscitation protocols by integrating data from multiple bedside monitors. By applying the staged evaluation pathway suggested by the authors, hospitals can move from offline benchmarking to supervised deployment, ensuring that autonomous systems remain within safe boundaries of authority while improving overall unit efficiency and patient outcomes.