Abstract
Despite recognizing that Large Language Models (LLMs) can generate inaccurate or unacceptable responses, universities are increasingly making such models available to their students. Existing university policies defer the responsibility of checking for the correctness and appropriateness of LLM responses to students and assume that they will have the required knowledge and skills to do so on their own. This lecture teaches attendees how to critically engage with LLMs to understand their strengths and weaknesses. As part of the lecture, the attendees engage in a hands-on activity to evaluate an LLM provided by the university; first by themselves, and then with a scaffolded design probe that guides them through an end-user auditing exercise.

Biography
Nikola Banovic, Ph.D., is an Associate Professor of Computer Science and Engineering and an Associate Director of Michigan Institute for Data and AI in Society at the University of Michigan, Ann Arbor. His research broadly focuses on Human-AI Interaction, Explainable AI (XAI), and Responsible AI. In particular, he has taken a keen interest in developing methods to explain the decisions of Artificial Intelligence (AI) systems to end-users without computer science backgrounds. Nikola’s research focuses on using explanations to raise end-user AI literacy, which in turn could help them detect and counter untrustworthy AI. Before joining the University of Michigan, Nikola received his Ph.D. degree from the Human-Computer Interaction Institute (HCII) at Carnegie Mellon University, and his B.Sc. and M.Sc. degrees from the University of Toronto. Nikola’s work has been recognized with an NSF CAREER award, and best paper and honorable mention awards at premier HCI conferences.