Research

Competency-Aware Navigation

Competency-Aware Planning for Probabilistically Safe Navigation Under Perception Uncertainty

Perception-based navigation systems are useful for unmanned ground vehicle (UGV) navigation in complex terrains, where traditional depth-based navigation schemes are insufficient. However, these data-driven methods are highly dependent on their training data and can fail in surprising and dramatic ways with little warning. To ensure the safety of the vehicle and the surrounding environment, it is imperative that the navigation system is able to recognize the predictive uncertainty of the perception model and respond safely and effectively in the face of uncertainty. In an effort to enable safe navigation under perception uncertainty, we develop a probabilistic and reconstruction-based competency estimation (PaRCE) method to estimate the model's level of familiarity with an input image as a whole and with specific regions in the image. We find that the overall competency score can correctly predict correctly classified, misclassified, and out-of-distribution (OOD) samples. We also confirm that the regional competency maps can accurately distinguish between familiar and unfamiliar regions across images. We then use this competency information to develop a planning and control scheme that enables effective navigation while maintaining a low probability of error. We find that the competency-aware scheme greatly reduces the number of collisions with unfamiliar obstacles, compared to a baseline controller with no competency awareness. Furthermore, the regional competency information is very valuable in enabling efficient navigation.
Related Paper: "Competency-Aware Planning for Probabilistically Safe Navigation Under Perception Uncertainty"

Competency Counterfactuals

Explaining Low Perception Model Competency with High-Competency Counterfactuals

There exist many methods to explain how an image classification model generates its decision, but very little work has explored methods to explain why a classifier might lack confidence in its prediction. As there are various reasons the classifier might lose confidence, it would be valuable for this model to not only indicate its level of uncertainty but also explain why it is uncertain. Counterfactual images have been used to visualize changes that could be made to an image to generate a different classification decision. In this work, we explore the use of counterfactuals to offer an explanation for low model competency--a generalized form of predictive uncertainty that measures confidence. Toward this end, we develop five novel methods to generate high-competency counterfactual images, namely Image Gradient Descent (IGD), Feature Gradient Descent (FGD), Autoencoder Reconstruction (Reco), Latent Gradient Descent (LGD), and Latent Nearest Neighbors (LNN). We evaluate these methods across two unique datasets containing images with six known causes for low model competency and find Reco, LGD, and LNN to be the most promising methods for counterfactual generation. We further evaluate how these three methods can be utilized by pre-trained Multimodal Large Language Models (MLLMs) to generate language explanations for low model competency. We find that the inclusion of a counterfactual image in the language model query greatly increases the ability of the model to generate an accurate explanation for the cause of low model competency, thus demonstrating the utility of counterfactual images.
Related Paper: "Explaining Low Perception Model Competency with High-Competency Counterfactuals"
While deep neural network (DNN)-based perception models are useful for many applications, these models are black boxes and their outputs are not yet well understood. To confidently enable a real-world, decision-making system to utilize such a perception model without human intervention, we must enable the system to reason about the perception model's level of competency and respond appropriately when the model is incompetent. In order for the system to make an intelligent decision about the appropriate action when the model is incompetent, it would be useful for the system to understand why the model is incompetent. We explore five novel methods for identifying regions in the input image contributing to low model competency, which we refer to as image cropping, segment masking, pixel perturbation, competency gradients, and reconstruction loss. We assess the ability of these five methods to identify unfamiliar objects, recognize regions associated with unseen classes, and identify unexplored areas in an environment. We find that the competency gradients and reconstruction loss methods show great promise in identifying regions associated with low model competency, particularly when aspects of the image that are unfamiliar to the perception model are causing this reduction in competency.
Related Paper: "Understanding the Dependence of Perception Model Competency on Regions in an Image"
While convolutional neural networks (CNNs) are extremely popular and effective for image classification, we must understand the level of confidence in their predictions before using them to make decisions. We leverage probability theory and the generative capabilities of autoencoders to develop a probabilistic and reconstruction-based competency estimation (PaRCE) method. Through this approach, we obtain a human-interpretable score that captures multiple facets of predictive uncertainty arising in CNN models. We compare our method to existing approaches for uncertainty quantification and out-of-distribution (OOD) detection and find that our method can best distinguish between correctly classified, misclassified, and OOD samples with anomalous regions, as well as between samples with visual image modifications resulting in high, medium, and low prediction accuracy. We then describe how to extend our approach for anomaly localization tasks and demonstrate the ability of our approach to distinguish between regions in an image that are familiar to the perception model from those that are unfamiliar. We find that our method generates interpretable scores that most reliably capture a holistic notion of perception model confidence.
Related Paper: "PaRCE: Probabilistic and Reconstruction-based Competency Estimation for CNN-based Image Classification"
Reinforcement learning (RL) methods for social robot navigation show great success navigating robots through large crowds of people, but the performance of these learning-based methods tends to degrade in particularly challenging or unfamiliar situations due to the models' dependency on representative training data. To ensure human safety and comfort, it is critical that these algorithms handle uncommon cases appropriately, but the low frequency and wide diversity of such situations present a significant challenge for these data-driven methods. To overcome this challenge, we propose modifications to the learning process that encourage these RL policies to maintain additional caution in unfamiliar situations. Specifically, we (1) modify the training process to systematically introduce deviations into a pedestrian model, (2) update the value network to estimate and utilize pedestrian-unpredictability features, and (3) implement a reward function to learn an effective response to pedestrian unpredictability. These modifications allow for similar navigation times and path lengths, while significantly reducing the number of collisions and the proportion of time spent in the pedestrians' personal space.
Related Paper: "Stranger Danger! Identifying and Avoiding Unpredictable Pedestrians in RL-based Social Robot Navigation"
Unmanned ground vehicles (UGVs) that autonomously maneuver over off-road terrain are susceptible to a loss of stability through untripped rollovers. Without human supervision and intervention, untripped rollovers can damage the UGV and render it unusable. We create a runtime monitor that can provide protection against rollovers that is independent of the type of high-level autonomy strategy (path planning, navigation, etc.) used to command the platform. In particular, we present an implementation of a predictive system monitor for untripped rollover protection in a skid-steer robotic platform. The system monitor sits between the UGV’s autonomy stack and the platform, and it ensures that the platform is not at risk of rollover by intercepting mobility commands sent by the autonomy stack, predicting platform stability, and adjusting the mobility commands to avoid potential rollovers.
Related Paper: "A Runtime Monitor for Platform Protection Against Skid-Steer Untripped Rollovers"
Machine learning techniques have become an effective way for intelligent systems to analyze data and generate a model of their environment with minimal human intervention. An important challenge in developing assistive robots is the design of socially compliant robot navigation policies that enable safe and comfortable movement around people. Previous works demonstrate that deep reinforcement learning (DRL) methods are effective in developing such navigation strategies. However, existing DRL policies are designed for simple, open-space environments, and through our experiments, we find that such policies do not generalize well to more realistic environments containing walls and other stationary objects. We present a modular approach to social navigation in complex environments, in which we combine a DRL policy with a global path planner and a deterministic safety controller. By designing each component of the modular architecture to handle different, and potentially conflicting, navigation objectives, we divide the indoor navigation problem into logically distinct and manageable steps. This allows us to extend the applicability of existing DRL policies to complex indoor spaces.
Related Paper: "A Modular Framework for Socially Compliant Robot Navigation in Complex Indoor Environments"
Human-aware robot navigation promises a range of applications in which mobile robots bring versatile assistance to people in common human environments. While prior research has mostly focused on modeling pedestrians as independent, intentional individuals, people move in groups; consequently, it is imperative for mobile robots to respect human groups when navigating around people. This project explores learning group-aware navigation policies based on dynamic group formation using deep reinforcement learning. Through simulation experiments, we show that group-aware policies, compared to baseline policies that neglect human groups, achieve greater robot navigation performance (e.g., fewer collisions), minimize violation of social norms and discomfort, and reduce the robot's movement impact on pedestrians. We also provide hardware demonstrations on a physical robot to present the applicability of our group-aware policy to real-world robotic systems. Our results contribute to the development of social navigation and the integration of mobile robots into human environments.
Related Paper: "Learning a Group-Aware Policy for Robot Navigation"
Machine learning techniques have become an effective way for intelligent systems to analyze data and generate a model of their environment with minimal human intervention. However, unlike methods that rely on robust control theory, machine learning techniques are limited in their ability to guarantee safety, which limits their applicability to safety-critical systems. This project aims to combine machine learning techniques with control theory to learn safety constraints without overly constraining the freedom of the system. In order to ensure safety, Hamilton-Jacobi-Isaacs (HJI) reachability methods will determine the safe region within the state space where the system can operate. This safe region will be updated as the system learns more about its environment, allowing it to become more or less conservative when appropriate. The system can then move freely according to any planning algorithm when well within the safe region, but when the system approaches an unsafe region, a safety override is performed to ensure that the system statays within the safe region.
Related Paper: "Efficient Safe Learning for Robotic Systems in Unstructured Environments"
As robotic systems enter our workplaces, roads, and homes, it is becoming increasingly important to ensure that robots are able to work safely and effectively with and around humans. Understanding the ways in which humans move in their environment and communicating this understanding to robotic systems can allow robots to operate safely and productively around humans. This project uses empirical data to gain understanding of the regularities in human movement and to find motion laws that reflect the movement of humans in cluttered environments. The data collected through this project supports the idea that human walking patterns exhibit a power law relationship between the speed and curvature of human feet, regardless of the trajectory of motion. This project considers both a single human walking in set patterns and around obstacles and two humans walking in a shared, and perhaps cluttered, space. With a better understanding of these walking patterns, planning algorithms can leverage the steering laws used by humans to better control the behavior of robots that operate in the same environment as people.
Related Paper: "Application of Control Theory Principles to Human Movement to Ensure the Safe Operation of Robots"
Robotic-Assisted Surgery (RAS) improves upon traditional minimally invasive (MIS) and open surgical techniques by maintaining the benefits of MIS while also providing surgeons with a wider range of motion, increased depth perception, and control for tremors. However, an inherent limitation of the technology is that surgeons performing RAS must rely solely on visual feedback and lose the sense of touch, creating a steep learning curve for the technique. To address this, we proposed a proof-of-concept addition to RAS systems that relays the firmness of soft tissue to surgeons. We constructed a probe containing a force-sensitive resistor (FSR) to collect information on silicone samples of known varying firmness that mimic soft tissue. From the FSR, currents were generated and amplified into a solenoid actuator. By pressing on the actuator, the user feels a force corresponding to the firmness of the silicone. Preliminary testing of the integrated feedback system indicated that users were able to successfully distinguish between varying silicone firmnesses.
Related Paper: "Investigating a Cooperative System of Sensing and Transmitting Haptic Feedback of Soft Tissue for Robotic Surgical Applications"