Join us Friday, April 10 for a hands-on short course hosted by Xiaowu Dai, PhD, an assistant professor of Statistics and Data Science at the University of California, Los Angeles (UCLA).
Dr. Dai’s research focuses on the intersection of economics and machine learning, integrating game theory with online learning and developing predictive models for economic applications and large language model contexts. Dr. Dai is also interested in statistical machine learning, particularly dynamical models, multimodal learning, and kernel-based methods, with applications in neuroimaging, diabetes, and kidney exchange.
TITLE:
Game-Theoretic Perspectives and Uncertainty Quantification of Large Language ModelsLarge Language Models (LLMs) are prone to inconsistencies and hallucinations. I will review training-free, game-theoretic frameworks for aligning LLMs, and then introduce the Peer Elicitation Games (PEG) mechanism, which involves a generator and multiple discriminators instantiated from distinct base models. The discriminators interact in a peer evaluation setting, where rewards are computed using a determinant-based mutual information score that provably incentivizes truthful reporting without requiring ground-truth labels. I will discuss the connection between this game-theoretic analysis and online learning with no-regret guarantees, as well as last-iterate convergence to a truthful Nash equilibrium, ensuring that the policies used by agents converge to stable and truthful behavior over time. I will also discuss topics related to uncertainty quantification for machine learning models and LLMs, including prediction-powered conditional inference, retrieval-augmented inference, and in-context learning.