EKFZ Proposes AI Agent Reforms to Enable Safe Healthcare Innovation

Researchers at the Else Krner Fresenius Center (EKFZ) for Digital Health at TU Dresden propose regulatory adaptations to address challenges posed by increasingly autonomous AI agents in healthcare, detailed in a Nature Medicine publication. Current regulations, designed for static medical devices, struggle to accommodate AI systems capable of complex, independent workflows. The EKFZ proposes immediate enforcement discretion, medium-term voluntary alternative pathways supplementing existing approvals, and long-term regulation mirroring medical professional standards, requiring demonstrated safe performance. Established in 2019 with approximately 40 million euros in funding, the EKFZ argues substantial regulatory reform is necessary for meaningful AI agent implementation.

Evolving AI and Regulatory Discrepancies

Current medical device regulations were designed for static technologies with human oversight and do not account for post-market evolution, creating challenges for the implementation of autonomous AI agents which demonstrate adaptability and a broader scope. These agents, comprised of interconnected components and controlled by large language models, represent a fundamental shift in how AI can be implemented in medicine, necessitating a re-evaluation of existing regulatory paradigms.

Researchers at the Else Krner Fresenius Center (EKFZ) propose immediate adaptations to address these regulatory barriers, including extending enforcement discretion policies and applying non-medical device classifications to systems falling outside traditional regulation while still serving a medical purpose. Medium-term solutions involve developing voluntary alternative pathways (VAPs) and adaptive regulatory frameworks, shifting from static pre-market approval to dynamic oversight informed by real-world performance data and stakeholder collaboration. Devices demonstrating misconduct could be transferred to established regulatory pathways.

Long-term solutions, according to the research, could involve regulating AI agents similarly to medical professionals, requiring structured training processes and granting autonomy only after demonstrating safe and effective performance. While regulatory sandboxes offer some flexibility for testing, the researchers note they are not scalable for widespread deployment due to resource demands.

Meaningful implementation of autonomous AI agents in healthcare will likely remain impossible in the medium term without substantial regulatory reform, with VAPs and adaptive pathways highlighted as the most effective strategies. Collaborative efforts between regulators, healthcare providers, and technology developers are crucial to create frameworks that meet the unique characteristics of AI agents while ensuring patient safety and enabling innovation within the field of AI healthcare regulation.

Proposed Regulatory Adaptations

The researchers propose several solutions to overcome regulatory barriers, including extending enforcement discretion policies – acknowledging a product qualifies as a medical device but temporarily forgoing certain requirements – or applying a non-medical device classification to systems serving a medical purpose but falling outside traditional regulation. Medium-term solutions involve developing voluntary alternative pathways (VAPs) and adaptive regulatory frameworks that supplement existing approval processes, shifting from static pre-market approval to dynamic oversight using real-world performance data, stakeholder collaboration, and iterative updates. Misconduct could lead to devices being transferred to established regulatory pathways.

Long-term solutions could involve regulating AI agents similarly to medical professionals, requiring structured training processes and granting autonomy only after demonstrating safe and effective performance. While regulatory sandboxes offer some flexibility for testing, the researchers note they are not scalable for widespread deployment due to resource demands.

Realizing the full potential of AI agents in healthcare requires bold and forward-thinking reforms, and regulators must prepare now to ensure patient safety and provide clear requirements to enable safe innovation. The research was published in Nature Medicine (Freyer et al., 2025).

Future Frameworks and Collaborative Requirements

The researchers propose several solutions to overcome regulatory barriers, including extending enforcement discretion policies – acknowledging a product qualifies as a medical device but temporarily forgoing certain requirements – or applying a non-medical device classification to systems serving a medical purpose but falling outside traditional regulation. Medium-term solutions involve developing voluntary alternative pathways (VAPs) and adaptive regulatory frameworks that supplement existing approval processes, shifting from static pre-market approval to dynamic oversight using real-world performance data, stakeholder collaboration, and iterative updates. Misconduct could lead to devices being transferred to established regulatory pathways.

Long-term solutions could involve regulating AI agents similarly to medical professionals, requiring structured training processes and granting autonomy only after demonstrating safe and effective performance. While regulatory sandboxes offer some flexibility for testing, the researchers note they are not scalable for widespread deployment due to resource demands.

Meaningful implementation of autonomous AI agents in healthcare will likely remain impossible in the medium term without substantial regulatory reform, with VAPs and adaptive pathways highlighted as the most effective strategies. Collaborative efforts between regulators, healthcare providers, and technology developers are crucial to create frameworks that meet the unique characteristics of AI agents while ensuring patient safety and enabling innovation within the field of AI healthcare regulation.

The Else Krner Fresenius Center (EKFZ) for Digital Health at TU Dresden and University Hospital Carl Gustav Carus Dresden was established in 2019, receiving approximately 40 million euros in funding over ten years to focus on innovative, medical, and digital technologies at the patient interface, aiming to fully exploit the potential of digitalization in medicine. The research was published in Nature Medicine (Freyer et al., 2025).