Modeling and Influencing Human Attentiveness in Autonomy-to-Human Perception Hand-offs

Yash Vardhan Pant, Balasaravanan Thoravi Kumaravel, Ameesh Shah, Erin Kraemer, Marcell Vazquez-Chanlatte, Kshitij Kulkarni, Bjoern Hartmann, and Sanjit A. Seshia. Modeling and Influencing Human Attentiveness in Autonomy-to-Human Perception Hand-offs. In 23rd IEEE International Conference on Intelligent Transportation Systems (ITSC), September 2022.

Download

[pdf] 

Abstract

It is not uncommon for autonomous systems (e.g., self-driving cars) to require the timely intervention of a human operator to ensure safe operation. It is important to design these systems such that the human is brought into the decision-making loop in a manner that enables them to make a timely and correct decision. In this paper, we consider one such application, which we refer to as the perception hand-off problem, which brings the driver into the loop when the perception module of an Autonomous Vehicle (AV) is uncertain about the environment. We formalize the perception hand-off problem by designing a Partially Observable Markov Decision Process (POMDP) model. This model captures the latent cognitive state (attention) of the driver which can be influenced through a proposed query-based active information gathering (AIG) system for a Human-Machine Interface (HMI). We design a web-based human study to identify the model parameters, and demonstrate the impact of the proposed HMI system. Results from this study show that the state of attentiveness does indeed impact the human performance, and our proposed active information gathering (AIG) actions, i.e., queries to the human driver, result in 7% faster responses from the human. Simulations with the identified POMDP model show that a learnt policy for deploying the AIG actions improves the percentage of correct responses from the human in the perception hand-off by around 5.4%, outperforming other baselines while also using fewer of these actions

BibTeX

@inproceedings{pant-itsc22,
 author    = {Yash Vardhan Pant and Balasaravanan Thoravi Kumaravel and Ameesh Shah and Erin Kraemer and Marcell Vazquez-Chanlatte and Kshitij Kulkarni and Bjoern Hartmann and Sanjit A. Seshia},
  title     = {Modeling and Influencing Human Attentiveness in Autonomy-to-Human Perception Hand-offs},
  booktitle = {23rd {IEEE} International Conference on Intelligent Transportation Systems (ITSC)}, 
  month = sep,
  year = {2022},
  abstract = {It is not uncommon for autonomous systems (e.g., self-driving cars) to require the timely intervention of a human operator to ensure safe operation. It is important to design these systems such that the human is brought into the decision-making loop in a manner that enables them to make a timely and correct decision. In this paper, we consider one such application, which we refer to as the perception hand-off problem, which brings the driver into the loop when the perception module of an Autonomous Vehicle (AV) is uncertain about the environment. We formalize the perception hand-off problem by designing a Partially Observable Markov Decision Process (POMDP) model. This model captures the latent cognitive state (attention) of the driver which can be influenced through a proposed query-based active information gathering (AIG) system for a Human-Machine Interface (HMI). We design a web-based human study to identify the model parameters, and demonstrate the impact of the proposed HMI system.  Results from this study show that the state of attentiveness does indeed impact the human performance, and our proposed active information gathering (AIG) actions, i.e., queries to the human driver, result in 7% faster responses from the human. Simulations with the identified POMDP model show that a learnt policy for deploying the AIG actions improves the percentage of correct responses from the human in the perception hand-off by around 5.4%, outperforming other baselines while also using fewer of these actions},
}