Software-Defined Buildings Research Group
Sam Kumar, Michael P Andersen, David E. Culler
University of California, Berkeley
Mr. Plotter, the Multiresolution Plotter, is a tool for visualizing timeseries data stored in the Berkeley Tree Database (BTrDB). It scales to massive data streams containing millions of data points, while allowing users to interactively pan and zoom through the data. Mr. Plotter retrieves and plots only statistical aggregates of the data (min, mean, max, count), eliminating bottlenecks in fetching data over the network and rendering the plot. Furthermore, BTrDB can compute these aggregates in sub-linear time using scalable precomputation techniques, alleviating data processing bottlenecks on the server. To convey enough meaningful information without plotting too many aggregates, Mr. Plotter automatically adjusts the resolution as the user zooms, so that the time range over which each aggregate is computed corresponds roughly to one pixel column in the resulting graph. The result is that users can interactively visualize timeseries data using an application that automatically queries new data and intelligently adjusts the plot as users freely zoom and scroll.
Although Mr. Plotter plots statistical aggregates instead of raw data points, the min and max together provide a silhouette of the data's full range, and is in many cases more useful than plotting the data points individually. See the demo below for an example. Figure 2 in the report provides a comparison to a plot of raw data.
Here is a demonstation video of Mr. Plotter and BTrDB. Thanks to Michael Andersen for making the video.
If the above video does not display for some reason, try clicking here.
I originally developed Mr. Plotter for the Micro-Synchrophasors (µPMU) Project during the summer of 2014. The very first version was a web application that used Meteor for its backend. Along with the first version of BTrDB (called QUASAR at the time), it served as a proof-of-concept of what would become the Smart Grid Store system.
Since then, I have rewritten Mr. Plotter's backend in Go and made performance and usability improvements to its frontend. It has been used extensively as part of the µPMU project. Unlike existing tools, which either filtered data at the source or provided non-interactive "click-and-wait" visualizations, Mr. Plotter allows users to fluidly explore the full data set despite the size and density of synchrophasor data. This was especially refreshing to power operators and utility engineers, one of whom described his experience using Mr. Plotter as "being a kid in a candy shop."
Power Standards Lab (PSL) has an industrial offering of micro-synchrophasors and software tools to store and process their data. Their product includes Mr. Plotter, among other software that came out of the µPMU project.
Mr. Plotter has also been integrated into Smart Grid Store, an open-source distribution of software created for the µPMU project.
PingThings provides a platform for smart grid data analytics targeted at utility companies. PingThings is including software written as part of the µPMU project as part of their analytics platform. Mr. Plotter influenced the design of a data visualization tool that PingThings is working on.
The XBOS Project, which provides an extensible operating system for smart buildings, also uses Mr. Plotter. In fact, you can see real building data, collected as part of XBOS, in Mr. Plotter here. Most data streams are confidential and require a registered user account to access; however some data streams are public (as of June 2018).
The most popular version of Mr. Plotter is a web application that renders the plot as an SVG. This is the version demonstrated in the above video. It is briefly described in the DISTIL paper.
The source code for Mr. Plotter can be found here. The backend is written in Go and can be built using the standard "go" utility (v1.7 and later). The binary, "mr-plotter", is configured via a configuration file, "plotter.ini", as well as an etcd database for compatibility with the rest of the BTrDB ecosystem. If you run Mr. Plotter using this method, you will have to run BTrDB separately and configure Mr. Plotter to connect to it.
Mr. Plotter is also integrated into Smart Grid Store (SGS), a distribution of the entire BTrDB ecosystem geared toward smart grid power data. Unlike the previous method, which requires BTrDB and etcd to be run separately, SGS bundles together all of the necessary infrastructure. In particular, if you are looking for a fast way to get Mr. Plotter up and running to try it out, the development installation of SGS is a good choice.
During the summer of 2016, I wrote another version of Mr. Plotter for power users who need to plot tens of data streams simultaneously. For increased performance to support this use case, this version of Mr. Plotter is a desktop/mobile application, written in C++ using the Qt framework, that renders the plot using OpenGL. The Mr. Plotter report focuses on this version.
The source code for this version of Mr. Plotter can be found here. On its own, this version of Mr. Plotter is a QML plugin, rather than a standalone application. In can be embedded into a larger application and controlled programmatically. If you are looking to try out Mr. Plotter, a standalone application can be found here. It uses the QML plugin for Mr. Plotter, and provides a user interface so it can be used as a standalone application.
The source code linked above can be built using Qt Creator. The build has been tested on Windows, Mac, Ubuntu, and Android. If you have a Ubuntu machine and would like to try out this version of Mr. Plotter, you can find a standalone distribution of the Mr. Plotter desktop application here, tested on Ubuntu 16.04. Extract the archive (run "tar zxvf mr-plotter-linux.tar.gz"); this will create a directory called "mr-plotter". Enter the directory with "cd mr-plotter" and then run "./run-mr-plotter-static.sh". Mr. Plotter will open in a new window. Note that you will need to run BTrDB separately and connect to it using the GUI in the Mr. Plotter window. The simplest way to run BTrDB is to install Smart Grid Store, as explained above.