U.C. Berkeley CS267 Home Page

Applications of Parallel Computers

Spring 2016

T Th 11:00-12:30, 306 Soda Hall

Instructor:

  • Jim Demmel
  • Offices:
    564 Soda Hall ("Virginia", in ASPIRE Lab), (510)643-5386
    831 Evans Hall
  • Office Hours: M 9-10, T 9-10 and F 1:30-2:30, in 564 Soda Hall
  • (send email)
  • Teaching Assistants:

  • Orianna DeMasi
  • Office Hours: T 1-3pm, in 580 Soda
  • (send email)
  • Marquita Ellis
  • Office Hours: W 11:30-1:30pm, in 580 Soda (except Feb 3 and Mar 23, see Announcements below)
  • (send email)
  • Administrative Assistants:

  • Tammy Chouteau
  • Office: 565 Soda Hall
  • Phone: (510)643-4816
  • (send email)
  • Roxana Infante
  • Office: 563 Soda Hall
  • Phone: (510)643-1455
  • (send email)
  • Syllabus and Motivation

    CS267 was originally designed to teach students how to program parallel computers to efficiently solve challenging problems in science and engineering, where very fast computers are required either to perform complex simulations or to analyze enormous datasets. CS267 is intended to be useful for students from many departments and with different backgrounds, although we will assume reasonable programming skills in a conventional (non-parallel) language, as well as enough mathematical skills to understand the problems and algorithmic solutions presented. CS267 satisfies part of the course requirements for the Designated Emphasis ("graduate minor") in Computational Science and Engineering.

    While this general outline remains, a large change in the computing world started in the mid 2000's: not only are the fastest computers parallel, but nearly all computers are becoming parallel, because the physics of semiconductor manufacturing will no longer let conventional sequential processors get faster year after year, as they have for so long (roughly doubling in speed every 18 months for many years). So all programs that need to run faster will have to become parallel programs. (It is considered very unlikely that compilers will be able to automatically find enough parallelism in most sequential programs to solve this problem.) For background on this trend toward parallelism, click here.

    This is a huge change not just for science and engineering but the entire computing industry, which has depended on selling new computers by running their users' programs faster without the users having to reprogram them. Large research activities to address this issue are underway at many computer companies and universities, including Berkeley's ASPIRE project, and its predecessor the ParLab. A summary of the ParLab's research agenda, accomplishments, and remaining challenges may be found here.

    While the ultimate solutions to the parallel programming problem are far from determined, students in CS267 will get the skills to use some of the best existing parallel programming tools, and be exposed to a number of open research questions.

  • Tentative Detailed Syllabus
  • Grading

    There will be several programming assignments to acquaint students with basic issues in memory locality and parallelism needed for high performance. Most of the grade will be based on a final project (in which students are encouraged to work in small interdisciplinary teams), which could involve parallelizing an interesting application, or developing or evaluating a novel parallel computing tool. Students are expected to have identified a likely project by mid semester, so that they can begin working on it. We will provide many suggestions of possible projects as the class proceeds.

    Asking Questions

    Outside of lecture, you are welcome to bring your questions to office hours (posted at the top of this page). If you cannot physically attend office hours, you may contact the instructor team via the instructor email. For students registered for the course at UC Berkeley, we encourage you to post your questions to the CS267 Piazza page (you need to sign up first). If you send a question to the instructor email, we may answer your question on Piazza if we think it might help others in the class. Students registered for version of the course being offered by XSEDE should sign up on Moodle for their on-line questions. During lecture students who are viewing remotely can send questions via either email or chat to instructor email, which the teaching assistants will be monitoring during lecture. Depending on the question, the teaching assistants will either answer by email, or ask the instructor to answer during the lecture. You will also submit homeworks via the instructor email; please check with assignment-specific submission instructions first.

    Class Projects

    You are welcome to suggest your own class project, but you may also look at the following sites for ideas:

  • the ParLab webpage,
  • the ASPIRE webpage,
  • the BEBOP webpage,
  • the Computational Research Division and NERSC webpages at LBL,
  • class posters and their brief oral presentations from CS267 in Spring 2009.
  • class posters from CS267 in Spring 2010
  • Brief oral poster presentations from CS267 in Spring 2012
  • Brief oral poster presentations from CS267 in Spring 2013
  • Brief oral poster presentations from CS267 in Spring 2014
  • Brief oral poster presentations from CS267 in Spring 2015
  • Announcements

  • (Apr 29) As a reminder, final project writeups are due Monday May 9 at midnight (see Mar 15 lecture notes on Class Project Suggestions for details).
  • (Apr 27) Please see Piazza and your email for details about the poster session on May 5, 8-11am in the Wozniak Lounge, Soda Hall. In particular, you need to mail your (1 to 3) presentation slides to cs267.spr16@gmail.com by midnight Tuesday so the GSIs can assemble them into one large file for all of you to present from on May 5. Coffee will also be served!
  • (Mar 24) Please send us your 1 page (or less) project proposals this week, so we can give you feedback. Please send them to cs267.spr16@gmail.com and demmel@berkeley.edu.
  • (Mar 22) GSI Marquita Ellis is changing her office hours, just this week (Spring Break), to Friday 11am-1pm in the 5th floor graduate student lounge, Soda Hall
  • (Feb 17) The project poster session will be Thursday of RRR week, May 5, from 8-11am in the Wozniak Lounge, Soda Hall
  • (Feb 9) GSI Orianna DeMasi is changing her office hours, just this week, to Tuesday, Feb 9, 6-7pm in 580 Soda Hall, and Thursday, Feb 11, 10-11am in 580 Soda Hall
  • (Feb 1) GSI Marquita Ellis is changing her office hours, just this week, to Friday, Feb 5, 1:30-3:30pm in 580 Soda Hall.
  • (Jan 19) UC Berkeley students should fill out the following on-line forms.
  • Please create an XSEDE User Portal account and let us know of your account usernames here by Jan 29, 2016.
  • Please complete the following class survey.
  • Fill out the following form to allow us to create a NERSC account for you.
  • Please read the NERSC Computer Use Policy Form so that you can sign a form saying that you agree to abide by the rules state there.
  • Please sign up for Piazza, which we will use for on-line Q&A.
  • (Jan 19) Homework Assignment 0 has been posted here, due Jan 29 by midnight for UC Berkeley students.
  • (Jan 19) For students who want to try some on-line self-paced courses to improve basic programming skills, click here. You can use this material without having to register. In particular, courses like CS 9C (for programming in C) might be useful.
  • (Jan 19) This course satisfies part of the course requirements for the Designated Emphasis ("graduate minor") in Computational Science and Engineering.
  • Class Resources and Homework Assignments.

  • This will include, among other things, class handouts, homework assignments, the class roster, information about class accounts, pointers to documentation for machines and software tools we will use, reports and books on supercomputing, pointers to old CS267 class webpages (including old class projects), and pointers to other useful websites.
  • Lecture Notes and Video

  • Live video streaming of the lectures may be seen here here.
  • Archived video of the lectures may be seen here here. The final video shows student introducing posters about their final projects.
  • To ask questions during live lectures, you can email them to instructor email, which the teaching assistants will be monitoring during lecture. Depending on the question, the teaching assistants will either answer by email, or ask the instructor to answer during the lecture.
  • The class web page from the 1996 offering has detailed, textbook-style notes available on-line which are up-to-date in their presentations of some parallel algorithms. The slides to be posted during this semester will contain a number of more recently invented algorithms as well.

  • Lectures from Spr 2016 will be posted here.
  • Jan 19, Lecture 1, Introduction, in ppt and pdf
  • Jan 21, Lecture 2, Single Processor Machines: Memory Hierarchies and Processor Features, in ppt and pdf (updated Jan 25, Feb 12)
  • Jan 26, finish Lecture 2, then start Lecture 3: Parallel Machines and Programming Models, in ppt and pdf
  • Jan 28, finish Lecture 3 (updated Jan 28) then start Lecture 4: Sources of Parallelism and Locality in Simulation (Part 1), in ppt and pdf (updated Feb 1)
  • Feb 2, finish Lecture 4 (updated Feb 1) then start Lecture 5: Sources of Parallelism and Locality in Simulation (Part 2), in ppt and pdf
  • Feb 4, finish Lecture 5, then start Lecture 6: Shared Memory Programming: Threads and OpenMP, in ppt and pdf, and then Tricks with Trees, in ppt and pdf
  • Feb 9, finish Lecture 6 on Tricks with Trees, then start Lecture 7: Distributed Memory Machines and Programming, in ppt and pdf
  • Feb 11, Lecture 8: UPC and UPC++: Partitioned Global Address Space Languages, by Kathy Yelick, in pptx and pdf
  • Feb 16, Lecture 9: Cloud Computing and Big Data Processing, by Shivaram Vankataraman, in pdf
  • Feb 18, Lecture 10: NERSC, Cori, Knights Landing, and Other Matters, by Jack Deslippe, in pdf
  • Feb 23, Lecture 11, An Introduction to CUDA/OpenCL and Graphics Processors (GPUs), by Forrest Iandola, in pptx and pdf
  • Feb 25, Lecture 12, Dense Linear Algebra (Part 1), in ppt and pdf
  • Mar 1, Lecture 13, Dense Linear Algebra (Part 2), in ppt and pdf
  • Mar 3, Lecture 14, Graph Partitioning, in ppt and pdf
  • Mar 8, finish Lecture 14 on Graph Partitioning, then start Lecture 15, Automatic Performance Tuning and Sparse-Matrix-Vector-Multiplication, in ppt and pdf
  • Mar 10, finish Lecture 15, Automatic Performance Tuning and Sparse-Matrix-Vector-Multiplication, in ppt and pdf (updated Mar 10)
  • Mar 15, Lecture 17, Homework #3 Presentation, in pptx and pdf, then Class Project Suggestions, in pptx and pdf then Structured Grids, in ppt and pdf.
  • Mar 17, Lecture 18, Parallel Graph Algorithms, by Aydin Buluc, in pptx and pdf
  • Mar 29, Lecture 19, Architecting Parallel Software with Patterns, by Kurt Keutzer, in pptx and pdf
  • Mar 31, Lecture 20, Fast Fourier Transform, in ppt and pdf
  • Apr 5, Lecture 21, Climate Modeling, by Michael Wehner
  • Slides in .pptx (518 MB)
  • Movie of Simulated Atmospheric Water Vapor Content in .mov (56MB)
  • Movie of Simulated Global Warming in .mov (138MB)
  • Apr 7, Lecture 22, Scientific Software Ecosystems, by Mike Heroux, in pptx and pdf
  • Apr 12, Lecture 23, Dynamic Load Balancing, in ppt and pdf
  • Apr 14, Lecture 24, Accelerated Materials Design through High-throughput First-Principles Calculations and Data Mining, by Kristin Persson, in pptx and pdf
  • Apr 19, Lecture 25, Hierarchical Methods for the N-Body Problem, in pptx and pdf
  • Apr 21, Complete Lecture 25, Hierarchical Methods for the N-Body Problem, in pptx and pdf (updated April 21),
    then Communication Lower Bounds and Optimal Algorithms
  • Apr 26, Lecture 27: Big Bang, Big Data, Big Iron: High Performance Computing and the Cosmic Microwave Background Data Analysis, by Julian Borrill, in pdf
  • Apr 28, Lecture 28: Big Data and Exascale: A Tale of Two Ecosystems, by Kathy Yelick, in pptx and pdf
  • May 5, Project Poster Presentations, in pptx, pdf, and youtube
  • Sharks and Fish

  • "Sharks and Fish" are a collection of simplified simulation programs that illustrate a number of common parallel programming techniques in various programming languages (some current ones, and some old ones no longer in use).
  • Basic problem description, and (partial) code from 1999 class, written in Matlab, CMMD, CMF, Split-C, Sun Threads, and pSather, available here.