How UCSF is Using AI to Speed Up Investigational Drug Trials

Pharmacists from the University of California, San Francisco, shared their best practices and lessons learned from building an internal, bespoke artificial intelligence (AI) tool to streamline investigational drug services (IDS) during a session presentation at the American Society of Health-System Pharmacists (ASHP) 2025 Midyear Clinical Meeting and Exhibition.

Craig Michael, PharmD, a data science pharmacist, led the development of the AI system called Foghorn alongside Lisa Janssen Carlson, PharmD, BCOP, DPLA, in her then-capacity as IDS manager at UCSF Health. She has since transitioned to a new role at Gilead.

The Complicated Path to Patient Enrollment

Carlson said the idea to build an AI system to use in drug trials was driven by logistical limitations on the ground. "I had very limited resources in IDS," she said. "We had 800 trials. We served an entire enterprise, pediatrics and adults, and I wanted a way to do this a little bit more quickly.”

Carlson used some of her time to elucidate the differences between standard medications and investigational drugs. Every investigational drug comes with a protocol—Carlson cited one protocol that was 1600 pages long—that must be strictly observed. They also often have investigator brochures and pharmacy manuals, and often numerous amendments. The onus is on the site participating in the trial to ensure all health care providers are familiar with all these documents; this can be particularly challenging for amendments, which may come at any time during the trial's run. "You can actually have amendments on a monthly basis, especially for phase 1 and pediatric trials," she said.

Every trial requires detailed review and file and system building and validation, after which practitioners and pharmacists go through a detailed initial training and then ongoing training throughout the trial's run. "It's not just usually plug-and-play," Carlson said. The process is also highly subjective to human error. "If I'm tired one day, there is a good chance I missed something," she said.

IDS is also "highly, highly regulated," Carlson said, by the FDA, the Joint Commission, the Board of Pharmacy, and internal oversight bodies within the hospital system.

Before a patient can be enrolled in a drug trial, the site must collect a detailed medical history in the electronic medical record (EMR) and then translate that information into the format the trial requires, all of which can be time-consuming and costly.

The time from when a new protocol request is received at USCF to when they are ready to actually launch a trial takes about 32 working days, or 8 weeks, Carlson said, but this work is rarely done continuously; rather, it has to be balanced with other job functions. Delays at any step in the process are common, and the consequences are real. If enrollment is slow at one site, sponsors may fill trial slots elsewhere or add new sites instead. "There's a chance you've done all this work, and some other site did it faster, and ... we have nobody enrolled in the trial," Carlson said.

Using AI to Minimize Delays

After Carlson outlined the challenges her team faced on the ground at UCSF Health, she and Michael detailed how they developed Foghorn to address those challenges. "This is not about replacing pharmacists,” Carlson said. “This is about pulling information together so we don’t miss something, and so we can spend more time on patient care, counseling, and clinical decision-making."

They built Foghorn to be a retrieval-augmented generation (RAG) platform rather than only relying on a large language model (LLM), the type of AI commonly used today (Gemini, ChatGPT, etc.). LLMs are powerful generators of information, but they can make information up (called "hallucinating") to fit whichever prompt is entered by the user, regardless of whether the information is true or not, or even if the information exists at all.

RAG models generate the same finished product as LLMs (readable information that answers the prompt), but their parameters for answering that information are limited to relevant information. In Foghorn, this relevant information is a specific clinical trial's documents. Foghorn uses these documents to answer a predefined set of questions, then generates medication record templates for use in each trial at UCSF. Those records include vital information for the pharmacy team, such as default dose, administration route, frequency, stability, and more. This limitation reduces the risk of hallucinations and protects the integrity of the trial process.

The system can also answer user prompts about a given protocol and can do so more quickly compared to the time it would take an individual to find the answer manually. As an added layer of assurance, Foghorn also cites the specific page number(s) in the protocol that it uses to extract the answers it gives.

Michael said that because Foghorn was built and is hosted internally at UCSF, it offers a higher level of confidentiality than using a publicly accessible AI tool. "These study materials often contain confidential, proprietary information," he said. "So when you're using these public models, you cannot guarantee the protection of that confidentiality." This also applies to patient records, where feeding them into a public AI program could violate HIPAA rules.

Performance, Cost, and Limitations

Michael said text chunking was essential to Foghorn's usefulness. Chunking is a process used by most LLMs to break down large bodies of text into smaller sections ("chunks") to make it easier for the tools to read. "If you don't do this processing and splitting in a meaningful, thoughtful manner, it's hard to guarantee success when you're asking questions and expecting the right answer," Michael said.

Michael said there are many different approaches to chunking, ranging from semantic (grouping text by theme) to simple length (such as grouping text every 500 words), but said his team uses a hybrid approach; semantic and thematic chunking, though, usually offer better downstream results.

Performance testing showed that Foghorn's runtime depended heavily on the length of the source documents and on the computing resources available. Michael gave the example of using a standard laptop to ask Foghorn to process a 300-page protocol. Foghorn completed the request in about 120 minutes. When the same request was made on a virtual 24-gigabyte graphics processing unit (GPU), Foghorn shaved about 10 minutes off the time to complete the request. The cost to run the program is 1 cent per 10 pages (Michael did not address the cost of actually building Foghorn).

Results, Michael said, are "extremely accurate, ... but not perfect." The program relies heavily on the clarity and quality of the protocol and related study documents to find its answers, which Michael said can "differ wildly."

He cited one time where the team asked Foghorn a question about dosing for one particular protocol. "On one page, [the protocol] said the dose was between 4 and 5 mg per kilogram. On an entirely different page, it said the dose should be 5 mg per kilogram. The problem with that was, nowhere in the protocols did it provide guidance on how you choose 4 or how you choose 5. ... [T]here's nothing AI can do to fill that gap." So when a user prompts Foghorn asking for the dosage amount, "it may pick 4, it may pick 5," Michael said.

The Future of AI in Pharmacy Practice

Michael and Carlson emphasized that though Foghorn was built specifically for IDS, the approach could be applied across pharmacy practice. Potential future uses include protocol amendment comparisons, concomitant medication reviews, inclusion and exclusion screening via the EMR, medication reconciliation, policy harmonization, and guideline summarization.

Most of all, in Carlson's words, "This is not intended to replace a pharmacist or a technician. The goal of this is to pull the information so I don't miss something, [or maybe] it opens me up to do other things." Many of the tasks of a pharmacist or technician can and should only be done by a human being, such as counseling patients. "This is really intended to augment and help us, to support us as we continued on this journey. Technology is coming. There's no way to get around it, but how do we use it to help us and to better serve our patients?"

REFERENCES

Michael C, Carlson LJ. "Let's Get This Trial Started: An AI-Powered Approach to Streamlining Investigational Drug Services (IDS)." Presented at: American Society of Health-System Pharmacists Midyear 2025 Clinical Meeting and Exposition; December 7-10, 2025; Las Vegas, Nevada.