Dana-Farber Cancer Institute in Boston has implemented the use of GPT-4, an artificial intelligence system, across its entire workforce. The system is used to streamline work and conduct research, but not in direct clinical care. The deployment, led by Renato Umeton, Ph.D., is the first of its kind in an academic medical center or hospital. The AI system, GPT4DFCI, is private, secure, and HIPAA-compliant. It is primarily used for operations and research tasks, with the main concern being inaccurate or false output. The deployment and its challenges were detailed in a case study published in NEJM AI.
GPT-4 Deployment at Dana-Farber Cancer Institute: An Overview
The Dana-Farber Cancer Institute in Boston has implemented the use of GPT-4, a language model developed by OpenAI, across its entire institution over a period of six months. This deployment, which involved training its 12,500-strong workforce, is aimed at streamlining work and facilitating research. However, the use of GPT-4 is not extended to direct clinical care. While several hospitals are conducting pilot studies on the clinical application of generative AI, Dana-Farber is the first to deploy a language model for general use by the workforce of an academic medical center or hospital.
The deployment process and its associated challenges were documented in a case study by Renato Umeton, Ph.D., and his colleagues. The paper, titled “GPT-4 in a Cancer Center — Institute-Wide Deployment Challenges and Lessons Learned,” was published in NEJM AI on March 28, 2024. It details the use of GPT-4 across various business areas, including basic research, clinical research, and operations.
GPT4DFCI: Design and Implementation
The GPT-4 model, referred to as GPT4DFCI, was designed with a user interface similar to ChatGPT. It is private, secure, and compliant with the Health Insurance Portability and Accountability Act (HIPAA). The model can handle up to 265 text pages per session.
The introduction of GPT4DFCI to the workforce was carefully managed. Initially, it was rolled out to a small group of advanced users, then gradually extended to more employees via a ticketing system, starting with a large computational research department and an operations department. The model was socialized on Microsoft Teams Channels and email broadcasts, with training materials that included a glossary of AI terms, practical suggestions for better prompt engineering, usage guidelines, and best practices.
Applications and Concerns of GPT4DFCI
GPT4DFCI is primarily used to increase efficiency in operational, research, and administrative tasks. It is not used in clinical care outside of controlled studies. During the initial rollout, the primary work areas reported were Operations (75%), Research (28%), and Clinical Care (15%). The most reported primary uses of AI were “Extracting or searching for information in notes, reports, or other documents” and “Answering general knowledge questions,” each of which was reported by more than 50% of respondents.
However, the deployment of GPT4DFCI was not without concerns. The top concern, reported by 65% of respondents, was “Inaccurate or false output,” followed by “Ethics and/or compliance with policies,” at 51%. In addition to ethical considerations, cost is a potential concern for widespread research use. A single query of maximum input and output length in GPT-4 costs about $1, which is expected to become a significant consideration when enabling higher-throughput research use cases.
The Future of GPT4DFCI
The Dana-Farber Cancer Institute’s deployment of GPT-4 represents a significant step in the integration of AI in healthcare institutions. The lessons learned from this deployment, as documented in the case study, provide valuable insights for other institutions considering similar initiatives. However, the concerns raised during the deployment highlight the need for ongoing evaluation and refinement of AI applications in healthcare, particularly in terms of accuracy, ethics, and cost.
The source code, infrastructure-as-code, and training material for GPT4DFCI is available at https://github.com/Dana-Farber-AIOS/GPT4DFCI, providing a valuable resource for other institutions interested in implementing similar AI solutions.
External Link: Click Here For More