Research from Purdue University presents a novel approach for deploying Connected and Automated Vehicles (CAVs) in challenging Midwest winter environments through the integration of Vision Language Models (VLMs). The system addresses the critical safety and mobility challenges posed by winter conditions through a VLM capable of interpreting complex visual scenes and natural language instructions to generate personalized control strategies. The framework incorporates a Retrieval-Augmented Generation (RAG) memory module that enables continuous adaptation to individual preferences through human feedback. Real-world experiments conducted on professional test tracks demonstrate that our approach reduces takeover rates by up to 76.9% compared to baseline systems, particularly when processing indirect user instructions. The system successfully interprets diverse winter driving scenarios, maintaining high safety standards while adapting control parameters to both environmental conditions and user preferences. A comprehensive benchmark dataset, focused specifically on winter driving conditions, provides a foundation for future CAV development in extreme weather. This work represents a significant step in autonomous vehicle technology for winter operations and builds a framework for safer, more reliable transportation systems in extreme weather conditions in the Midwest.
---
About the speaker: Dr. Ziran Wang is a Tenure-Track Assistant Professor in the Lyles School of Civil Engineering at Purdue University. Prior to joining Purdue, Dr. Wang worked for Toyota R&D in Silicon Valley as a Principal Researcher of Digital Twin. He also serves as Founding Chair of the IEEE Technical Committee on Internet of Things in Intelligent Transportation Systems, and Associate Editor of four academic journals. His achievements were demonstrated at the Consumer Electronics Show (CES) in Las Vegas, and acknowledged by five best paper awards, the First Prize in IEEE Shape the Future of ITS Competition, and the U.S. Department of Transportation Dissertation Award. Dr. Wang is an author of 60+ refereed publications and 50+ patent applications, and his research interests are digital twins, autonomous driving, and embodied AI. He received the Ph.D. degree in Mechanical Engineering from the University of California, Riverside.