Abstract: In this talk I address two questions—how well a classifier can distinguish between subspaces, and how hard it is to describe a Bayes classifier to another party—in terms of the information-theoretic content of the signals and their classifiers. First, I present limits on subspace classifier performance in the asymptotic regimes of high signal dimension and high SNR, respectively, by bounding the capacity and diversity order of the associated sensing channels. These limits relate the number of information bits supported by the sensing channel to the reliability of classification, and lead to expressions for the number of distinguishable subspace classes and the classification error rate in terms of the signal geometry and tensor structure. Second, I present bounds on the rate-distortion function of a parametric Bayes classifier. This captures the number of information bits needed to “explain” a classifier to another party up to a small Lp loss. These bounds are expressed in terms of the data statistics and the model complexity of the parametric family under consideration. In addition to applications to distributed learning, the rate-distortion analysis provides lower bounds, which are tight in some cases, on the sample complexity of supervised learning with respect to the Bayes Lp risk. This is joint work with Miguel Rodrigues at University College London, Ahmad Beirami at Harvard, Robert Calderbank at Duke, and Ishan Jindal at Wayne State University.

Bio: Dr. Matthew Nokleby received the B.S. and M.S. degrees from Brigham Young University, Provo, UT, in 2006 and 2008, respectively, and the Ph.D. degree from Rice University, Houston, TX, in 2012, all in electrical engineering. From 2013-2015 he was a post-doctoral research associate at the Information Initiative @ Duke University. In 2015 he joined Wayne State University, where he is an assistant professor of Electrical and Computer Engineering. His research interests include distributed signal processing, wireless communications, information theory, and machine learning.