1.65 Million Clinical Trial Records Build New Benchmark for AI Training

Researchers have developed a large-scale, structured clinical trial database called TrialPanorama, containing 1.65 million records from 15 global sources, along with over 9,000 systematic review papers. This dataset aims to address critical gaps in the development of specialized AI agents for clinical research, providing a robust foundation for training, evaluating, and deploying AI systems in the pharmaceutical domain. The project was led by Zifeng Wang, a Ph.D. graduate from the University of Illinois Urbana-Champaign and co-founder of Keiji.AI. Over the years, Wang has worked on several AI-driven initiatives in clinical trials, including TrialMind, LEADS, TrialGPT, DSWizard, and InformGen—tools designed to assist with systematic reviews, patient recruitment, data analysis, and document generation. However, through hands-on collaboration with clinicians, pharmaceutical researchers, and industry partners, Wang realized that despite growing interest in AI for clinical trials, significant challenges remained. Two major barriers emerged: first, the lack of high-quality, structured data tailored for AI training and evaluation; second, the absence of standardized benchmarks that reflect real-world clinical trial workflows. While commercial data providers like Citeline offer comprehensive trial data, they are prohibitively expensive—often costing millions annually for large pharmaceutical firms. Public databases like ClinicalTrials.gov cover only trials registered in the U.S., leaving many international and published studies unaccounted for. To overcome these limitations, Wang and his team built TrialPanorama—a meticulously curated, globally representative database that captures essential trial elements such as study design, interventions, indications, biomarkers, and outcome measures. The data is aligned with authoritative medical ontologies like DrugBank and MedDRA, ensuring consistency, interoperability, and scalability. Complementing the database, the researchers introduced a comprehensive evaluation benchmark with eight tasks across two categories: systematic review (e.g., study retrieval, screening, evidence summarization) and trial design (e.g., defining inclusion/exclusion criteria, selecting endpoints, estimating sample size, and assessing feasibility). Experiments on five leading large language models revealed that while general-purpose models show some zero-shot capability, their performance remains insufficient for high-stakes clinical trial applications requiring precision, reliability, and regulatory compliance. TrialPanorama is not only a data resource but also a platform for advancing AI in drug development. It enables knowledge discovery—for example, identifying past treatments and ongoing research for a specific indication—supporting smarter, more informed trial planning. The database serves as a high-quality training corpus for domain-specific models and provides a standardized framework for assessing AI performance. For the emerging field of vertical AI agents, TrialPanorama offers a structured, standardized data source that can be integrated via Model Context Protocol (MCP) servers, enabling rapid deployment and seamless interaction with AI systems. This integration is key to building trustworthy, domain-aware agents capable of handling complex, regulated clinical workflows. Wang reflects that his journey into AI for healthcare began with optimism about quick wins—using large models and agents to automate trial tasks. But real-world engagement revealed deeper challenges: many AI solutions failed to meet clinical or regulatory standards, and tools often couldn’t integrate into existing workflows. These experiences taught him the importance of listening to end users, understanding their processes, and redefining problems from a practical, rather than technical, perspective. This shift in mindset—between research and real-world impact—inspired Wang to co-found Keiji.AI with his Ph.D. advisor, Jimeng Sun, and a team from UIUC. The company focuses on translating academic research into practical tools for the pharmaceutical industry. Its flagship platform, TrialMind, integrates AI agents with the TrialPanorama database to accelerate trial design, patient cohort identification, data analysis, and recruitment. Keiji.AI has already partnered with major pharmaceutical companies such as Takeda, AbbVie, and Regeneron, as well as real-world data providers like Medidata and Guardant Health, and CROs including IQVIA. The company is currently in a growth phase, actively raising funds and expanding its team. With TrialPanorama as a cornerstone, the team is now developing specialized AI agents tailored specifically for clinical trial workflows—aiming to deliver not just technological innovation, but tangible value to drug developers, clinicians, and patients.