Scenic: A Language for Scenario Specification and Scene Generation

Daniel J. Fremont, Tommaso Dreossi, Shromona Ghosh, Xiangyu Yue, Alberto L. Sangiovanni-Vincentelli, and Sanjit A. Seshia. Scenic: A Language for Scenario Specification and Scene Generation. In Proceedings of the 40th annual ACM SIGPLAN conference on Programming Language Design and Implementation (PLDI), June 2019.

Download

[pdf] 

Abstract

We propose a new probabilistic programming language for the design and analysis of perception systems, especially those based on machine learning. Specifically, we consider the problems of training a perception system to handle rare events, testing its performance under different conditions, and debugging failures. We show how a probabilistic programming language can help address these problems by specifying distributions encoding interesting types of inputs and sampling these to generate specialized training and test sets. More generally, such languages can be used for cyberphysical systems and robotics to write environment models, an essential prerequisite to any formal analysis. In this paper, we focus on systems like autonomous cars whose environment is a scene, a configuration of physical objects. We design a domain-specific language, Scenic, for describing scenarios that are distributions over scenes. As a probabilistic programming language, Scenic allows assigning distributions to features of the scene, as well as declaratively imposing hard and soft constraints over the scene. We develop specialized techniques for sampling from the resulting distribution, taking advantage of the structure provided by Scenic’s domainspecific syntax. Finally, we apply Scenic in a case study on a convolutional neural network designed to detect cars in road images, improving its performance beyond that achieved by state-of-the-art synthetic data generation methods.

BibTeX

@inproceedings{fremont-pldi19,
  author = {Daniel J. Fremont and Tommaso Dreossi and Shromona Ghosh and Xiangyu Yue and Alberto L. Sangiovanni-Vincentelli and Sanjit A. Seshia},
  title     = {Scenic: A Language for Scenario Specification and Scene Generation},
  booktitle = {Proceedings of the 40th annual ACM SIGPLAN conference on Programming Language Design and Implementation (PLDI)},
  Year = {2019},
  Month = {June},
  abstract = {We propose a new probabilistic programming language for  
the design and analysis of perception systems, especially  
those based on machine learning. Specifically, we consider  
the problems of training a perception system to handle rare  
events, testing its performance under different conditions,  
and debugging failures. We show how a probabilistic programming language can help address these problems by  
specifying distributions encoding interesting types of inputs  
and sampling these to generate specialized training and test  
sets. More generally, such languages can be used for cyberphysical systems and robotics to write environment models,  
an essential prerequisite to any formal analysis. In this paper,  
we focus on systems like autonomous cars whose environment is a scene, a configuration of physical objects. We design  
a domain-specific language, Scenic, for describing scenarios  
that are distributions over scenes. As a probabilistic programming language, Scenic allows assigning distributions to features of the scene, as well as declaratively imposing hard and  
soft constraints over the scene. We develop specialized techniques for sampling from the resulting distribution, taking  
advantage of the structure provided by Scenic’s domainspecific syntax. Finally, we apply Scenic in a case study on a  
convolutional neural network designed to detect cars in road  
images, improving its performance beyond that achieved by  
state-of-the-art synthetic data generation methods.},
}