How does a listener benefit from observing a talker visually? This classic problem of perception in practical work situations was introduced in a study by Sumby & Pollack in 1954. Their question remains vital. Their account calibrated a perceiver’s ability to take advantage at every degree of intelligibility of an opportunity to view the talker. Today’s accounts often treat a perceiver as a deaf viewer bound to a blind listener, puzzling through the differing phonetic impressions which result from such different sensory opportunities. Other accounts suppose that the incommensurate dimensions of visual and auditory sensation are recoded into a hypothetically common amodal metric to permit phonetic analysis. Almost every account derives its evidence from small inventories of test items presented under minimal uncertainty, and consider the role of phoneme incidence to guide perceptual analysis. I will discuss new evidence from our laboratory of intelligible audiovisual presentation of speech, using auditory and visual components which are unintelligible presented individually. Perceivers reported their impressions of the utterances in open set transcription under minimally constrained conditions of uncertainty. Such phenomena are consistent with primitive intersensory combination on which perceptual analysis of audiovisual speech might truly depend.

About the speaker:
Robert E. Remez is Professor of Psychology at Barnard College, Columbia University, where he has taught since 1980. A native New Yorker, Robert was a Predoctoral Research Trainee at Haskins Laboratories in New Haven, and received the doctorate in 1978 from the University of Connecticut. He received the Bachelor of Arts degree in 1971 from Brandeis University. At Barnard, he has held an Ann Whitney Olin Chair, has been Chair of the Departments of Psychology and Sociology, and is Chair of the Columbia University Seminar on Language & Cognition. He is co-editor of both the first and second edition of the Handbook of Speech Perception, and was Associate Editor of the journals Perception & Psychophysics and the Journal of Experimental Psychology: Human Perception and Performance. In service to the research community, he was a member of the Committee of Visitors for the Social, Behavioral and Economic Sciences of the National Science Foundation, and was a sitting member of the Communication Sciences Study Section of the Division of Research Grants and the Language and Communication Study Section of the Center for Scientific Review of the National Institutes of Health. He has been elected a Fellow of the Acoustical Society of America, the Association for Psychological Science, the American Psychological Association, the American Association for the Advancement of Science, and the Psychonomic Society. The Speech Perception Lab at Barnard has been supported by grants from the National Institute of Mental Health, the National Institute of Neurological Disorders and Stroke, the National Science Foundation, and a longstanding grant from the National Institute on Deafness and Other Communication Disorders.