Learning the Language of Patients: Hoifung Poon (Microsoft Research)

2025-26 Allen School Distinguished Lecture Series
Title: Learning the Language of Patients: Multimodal Generative AI for Precision Health
Speaker: Hoifung Poon (Microsoft Research)
Date: Thursday, November 13, 2025

Abstract: The dream of precision health is to develop a data-driven, continuous learning system where new health information is instantly incorporated to optimize care delivery and accelerate biomedical discovery. The confluence of technological advances and social policies has led to rapid digitization of multimodal, longitudinal patient journeys, such as electronic health records (EHRs), imaging, and multiomics. Our overarching research agenda lies in advancing multimodal generative AI for precision health, where we harness real-world data to pretrain powerful multimodal patient embedding, which can serve as digital twins for patients. This enables us to synthesize multimodal, longitudinal information for millions of cancer patients, and apply the population-scale real-world evidence to advancing precision oncology in deep partnerships with real-world stakeholders such as large health systems and life sciences companies.

Bio: Hoifung Poon is the General Manager of Real-World Evidence at Microsoft Research and an affiliated faculty at the University of Washington Medical School. He leads biomedical AI research and incubation, with the overarching goal of structuring medical data to optimize delivery and accelerate discovery for precision health. His team and collaborators are among the first to explore large language models (LLMs) and multimodal generative AI in health applications, producing popular open-source foundation models such as PubMedBERT, BioGPT, BiomedCLIP, LLaVA-Med, BiomedParse, with tens of millions of downloads. His latest publication in Nature features GigaPath, the first whole-slide digital pathology foundation model pretrained on over one billion pathology image tiles. He has led successful research partnerships with large health providers and life science companies, creating AI systems in daily use for applications such as molecular tumor board and clinical trial matching. His prior work has been recognized with Best Paper Awards from premier AI venues such as NAACL, EMNLP, and UAI, and he was named the "Technology Champion" by the Puget Sound Business Journal in the 2024 Health Care Leadership Awards. He received his PhD in Computer Science and Engineering from the University of Washington, specializing in machine learning and NLP.

This video is closed captioned.