{ "147770-21901951": { "datetime_modified":"20260415T132944", "datetime_start":"20260428T113000", "datetime_end":"20260428T130000", "has_end_time":1, "date_start":"2026-04-28", "date_end":"2026-04-28", "time_start":"11:30:00", "time_end":"13:00:00", "time_zone":"America\/Detroit", "event_title":"Topics on the Generalization and Learnability of Modern Machine Learning", "occurrence_title":"", "combined_title":"Topics on the Generalization and Learnability of Modern Machine Learning: Jake Trauger", "event_subtitle":"Jake Trauger", "event_type":"Lecture \/ Discussion", "event_type_id":"13", "description":"Learning theory is a subfield of machine learning research where we analyze the theoretical properties of machine learning problems and algorithms through mathematics. This thesis is a culmination of three standalone works in the field of learning theory.\n\nFirst, we analyze the generalization bounds of the Transformer architecture to show that it does not depend on maximum sequence length. To do this, we analyze the Rademacher Complexity of the architecture and create novel covering number bounds on linear functions that do not depend on the amount of samples. We also run a simulation and show the results support our theoretical findings.\n\nIn the next chapter, we analyze a quirk seen in the training of modern large language models. Most of these models are trained to only predict the next token of the output; however, the output of the model is a sequence of tokens. We study this mismatch in training optimization and output through the lens of the surrogate loss consistency framework. We analyze different ways of decoding these next-token predictors to see when we achieve asymptotic consistency for two use cases when encoded as loss functions.\n\nIn the final work of this thesis, the theoretical learnability of multiclass forgiving 0-1 loss functions is studied through the PAC-Learnability framework. We show a generalization the Natarajan dimension [Natarajan, 1989] characterizes the learnability of many instantiations of learning problems that use forgiving 0-1 loss functions. We also show how this setting can be used to model other known settings in the literature.", "occurrence_notes":null, "guid":"147770-21901951@events.umich.edu", "permalink":"http:\/\/events.umich.edu\/event\/147770", "building_id":"1000167", "building_name":"West Hall", "campus_maps_id":"163", "room":"335", "location_name":"West Hall", "has_livestream":0, "cost":"", "tags":["Dissertation"], "website":"", "sponsors":[ { "group_name":"Department of Statistics Dissertation Defenses", "group_id":"4045", "website":"" } ], "image_url":"", "styled_images":{ "event_thumb":"", "event_large":"", "event_large_2x":"", "event_large_lightbox":"", "group_thumb":"", "group_thumb_square":"", "group_large":"", "group_large_lightbox":"", "event_large_crop":"", "event_list":"", "event_list_2x":"", "event_grid":"", "event_grid_2x":"", "event_feature_large":"", "event_feature_thumb":"" }, "occurrence_count":1, "first_occurrence":21901951 } , "147383-21900951": { "datetime_modified":"20260403T113109", "datetime_start":"20260429T090000", "datetime_end":"20260429T110000", "has_end_time":1, "date_start":"2026-04-29", "date_end":"2026-04-29", "time_start":"09:00:00", "time_end":"11:00:00", "time_zone":"America\/Detroit", "event_title":"Principled Evaluation of Large Language Models: A Statistical Perspective", "occurrence_title":"", "combined_title":"Principled Evaluation of Large Language Models: A Statistical Perspective: Felipe Maia Polo", "event_subtitle":"Felipe Maia Polo", "event_type":"Lecture \/ Discussion", "event_type_id":"13", "description":"The rapid progress of large language models has outpaced the development of principled methodologies for their evaluation. This dissertation draws on ideas from psychometrics and statistics to build rigorous, efficient, and interpretable evaluation frameworks for modern AI systems. In this talk, I focus on three contributions that address complementary challenges in LLM evaluation.\n\nFirst, I present PromptEval, a method that confronts the problem of prompt sensitivity \u2014 the phenomenon whereby minor rephrasing of benchmark questions can substantially alter measured model performance. By combining Item Response Theory with matrix completion, PromptEval efficiently approximates the full distribution of model performance across hundreds of prompt variations while requiring less than 5% of the total evaluations, replacing arbitrary single-prompt assessments with statistically robust characterizations of model behavior.\n\nSecond, I introduce skill-based scaling laws that model LLM performance through latent capabilities such as reasoning and instruction-following. Inspired by factor analysis, this approach exploits the correlation structure among benchmark tasks to produce scaling predictions that are both more accurate and more interpretable than existing laws, which typically focus on aggregate validation loss and fail to generalize across model families.\n\nThird, I present Bridge, a unified statistical framework that explicitly connects LLM-as-a-Judge evaluations to human assessments. Bridge models the systematic discrepancies between human and LLM judgments through a latent preference score and a linear transformation of divergence-capturing covariates, enabling principled recalibration of automated scores and formal statistical testing for human\u2013LLM gaps.\n\nTogether, these contributions advance a vision of AI evaluation as a scientific discipline in its own right \u2014 one that demands the same statistical care we expect from the systems being evaluated.", "occurrence_notes":null, "guid":"147383-21900951@events.umich.edu", "permalink":"http:\/\/events.umich.edu\/event\/147383", "building_id":"1000167", "building_name":"West Hall", "campus_maps_id":"163", "room":"470", "location_name":"West Hall", "has_livestream":0, "cost":"", "tags":["Dissertation"], "website":"", "sponsors":[ { "group_name":"Department of Statistics Dissertation Defenses", "group_id":"4045", "website":"" } ], "image_url":"", "styled_images":{ "event_thumb":"", "event_large":"", "event_large_2x":"", "event_large_lightbox":"", "group_thumb":"", "group_thumb_square":"", "group_large":"", "group_large_lightbox":"", "event_large_crop":"", "event_list":"", "event_list_2x":"", "event_grid":"", "event_grid_2x":"", "event_feature_large":"", "event_feature_thumb":"" }, "occurrence_count":1, "first_occurrence":21900951 } }