Chair: Nima Fazeli

Note: Halloween costumes for audience members allowed (if not encouraged!)

Abstract:
Safe operation in unstructured environments, like homes or offices, is imperative for robust robotic systems. At the core of unstructured environments is uncertainty. Failing to consider uncertainty can lead to downstream task failures with costly and even dangerous consequences.

There are many types of uncertainty that need to be considered for real-world systems in unstructured environments. In this dissertation, I focus on uncertainty of in-hand object pose for objects grasped by a robotic system. If not addressed, this type of uncertainty can lead to potential failures including grasp instability and unexpected or unreliable contacts made with the environment (including other robots, objects, or humans). These types of failures significantly impact task performance for applications including robotic tool use, assembly, object hand-over, and human-robot interaction.

This dissertation makes several contributions to in-hand object pose estimation using proprioceptive and tactile feedback. Firstly, I provide a proprioceptive method to estimate contact locations and in-hand object poses simultaneously in a bimanual system from single contact-rich interactions. Then, I build upon this approach to connect several sequential actions in order to more precisely estimate in-hand object poses. Next, I incorporate robot proprioception uncertainty with this proprioceptive in-hand object pose estimation framework to adapt to real-world challenges with configuration-dependent robot proprioception uncertainty. Finally, I introduce the GelSlim 4.0 sensor, a powerful visuotactile sensor that enables key tactile sensing capabilities like depth estimation and shear field estimation. With accessibility as an explicit design goal, this sensor is accompanied by open-source manufacturing documentation that we user-tested with 17 novice users. In summary, this dissertation contributes algorithms and hardware for robotic manipulation that leverage proprioceptive and tactile feedback to address uncertainty in in-hand object poses.