U.C. Berkeley CS267 Home Page
Applications of Parallel Computers
Spring 2015
T Th 9:30-11:00, 306 Soda Hall
Instructor:
Teaching Assistants:
Evangelos Georganas
Office: 5th Floor Soda Hall (ASPIRE Lab)
Office Hours: T/Th 1-2pm, in 283E Soda Hall (updated Jan 19)
(send email)
Forrest Iandola
Office: 5th Floor Soda Hall (ASPIRE Lab)
Office Hours: F 3-5pm, in 580 Soda (updated Jan 20)
(send email)
Penporn Koanantakool
Office: 5th Floor Soda Hall (ASPIRE Lab)
Office Hours: T/Th 4-5pm, in 580 Soda Hall (updated Jan 19)
(send email)
Administrative Assistants:
Tammy Johnson
Office: 565 Soda Hall
Phone: (510)643-4816
(send email)
Roxana Infante
Office: 563 Soda Hall
Phone: (510)643-1455
(send email)
Syllabus and Motivation
CS267 was originally designed to teach students how to
program parallel computers to efficiently solve
challenging problems in science and engineering,
where very fast computers are required
either to perform complex simulations or to analyze enormous datasets.
CS267 is intended to be useful for students from many departments
and with different backgrounds, although we will assume reasonable
programming skills in a conventional (non-parallel) language,
as well as enough mathematical skills to understand the
problems and algorithmic solutions presented.
CS267 satisfies part of the course requirements for the
Designated Emphasis ("graduate minor") in
Computational Science and Engineering.
While this general outline remains, a large change
in the computing world started in the mid 2000's:
not only are the fastest computers
parallel, but nearly all computers are becoming parallel,
because the physics of semiconductor manufacturing
will no longer let conventional sequential processors
get faster year after year, as they have for so long
(roughly doubling in speed every 18 months for
many years). So all programs that need to
run faster will have to become parallel programs.
(It is considered very unlikely that compilers will be
able to automatically find enough parallelism in most
sequential programs to solve this problem.)
For background on this trend toward parallelism, click
here.
This is a huge change not just for science
and engineering but the entire computing industry,
which has depended on selling new computers by running
their users' programs faster without the users
having to reprogram them. Large research activities
to address this issue are underway at many computer
companies and universities, including
Berkeley's
ASPIRE project,
and its predecessor the
ParLab.
A summary of the ParLab's research agenda, accomplishments,
and remaining challenges may be found
here.
While the ultimate solutions to the parallel programming
problem are far from determined, students in CS267 will
get the skills to use some of the best existing parallel programming
tools, and be exposed to a number of open research questions.
Tentative Detailed Syllabus
Grading
There will be several programming assignments to acquaint students
with basic issues in memory locality and parallelism needed for
high performance. (Note: plan to redesign the third programming assignment
to solve a different problem, a graph problem arising in genomics.)
Most of the grade will be based on a final project
(in which students are encouraged to work in small interdisciplinary teams),
which could involve parallelizing an interesting application, or
developing or evaluating a novel parallel computing tool. Students
are expected to have identified a likely project by mid semester,
so that they can begin working on it. We will provide many suggestions
of possible projects as the class proceeds.
Asking Questions
Outside of lecture, you are welcome to bring your questions to office hours
(posted at the top of this page). If you cannot physically attend office hours,
you may contact the instructor team via the
instructor email.
We encourage you to post your questions to the
CS267 Piazza page
(you need to sign up first).
If you send a question to the instructor email, we may answer your question
on Piazza if we think it might help others in the class.
During lecture, remote students can also email their questions to
instructor email,
which the teaching assistants will be monitoring during lecture.
Depending on the question, the teaching assistants will either
answer by email, or ask the instructor to answer during the lecture.
You will also submit homeworks via the
instructor email;
please check with assignment-specific submission instructions first.
Class Projects
You are welcome to suggest your own class project, but you may also look at
the following sites for ideas:
the ParLab webpage,
the ASPIRE webpage,
the BEBOP webpage,
the Computational Research Division and
NERSC webpages at
LBL,
class posters and their
brief oral presentations from
CS267 in Spring 2009.
class posters from
CS267 in Spring 2010
Brief oral poster presentations
from CS267 in Spring 2012
Brief oral poster presentations
from CS267 in Spring 2013
Brief oral poster presentations
from CS267 in Spring 2014
Announcements
(Apr 17) Poster presentations of final projects (possibly including recording a short video presentation)
will occur in the morning of May 7 (Thursday of RRR week) in the Wozniak Lounge.
Final report writeups are due Monday May 11 at midnight (11:59pm).
For more details, see Class Project Suggestions,
in pptx and
pdf
(updated May 3, 6:45am).
(Apr 2) Prof. Demmel will hold extra office hours on Th, Apr 2, 2-3pm;
Fr, Apr 3, 1-2pm; M, Apr 6, 11-12am
(Feb 3) Results of Homework 0 are now posted near the bottom of the
Class Resources Web Page.
(Feb 3) Homework 1, including assigned student teams, is now posted
near the bottom of the
Class Resources Web Page.
(Jan 26) Please create an XSEDE User Portal account and let us know of your account usernames here by Jan 30, 2015.
(Jan 18) For students who want to try some on-line self-paced
courses to improve basic programming skills, click
here.
You can use this material without having to register.
In particular, courses like CS 9C (for programming in C) might be useful.
(Jan 18) Please complete the following
class survey.
(Jan 18) Homework Assignment 0 has been posted
here,
due Jan 30 by midnight.
(Jan 18) Fill out the following form
to allow us to create a NERSC account for you.
(Jan 18) Please read the
NERSC Computer Use Policy Form so that you can sign
a form saying that you agree to abide by the rules state there.
(Jan 18) This course satisfies part of the course requirements
for the Designated Emphasis ("graduate minor") in
Computational Science and Engineering.
This will include, among other things,
class handouts, homework assignments,
the class roster, information about class accounts, pointers
to documentation for machines and software tools we will use,
reports and books on supercomputing,
pointers to old CS267 class webpages (including old class projects),
and pointers to other useful websites.
Lecture Notes and Video
Live video streaming of the lectures may be seen here
here.
Archived video of the lectures may be seen here
here.
The final video shows student introducing posters about their final projects.
To ask questions during live lectures, you can email them to
instructor email,
which the teaching assistants will be monitoring during lecture.
Depending on the question, the teaching assistants will either
answer by email, or ask the instructor to answer during the lecture.
The class web page from the
1996 offering
has detailed, textbook-style notes available on-line which are
up-to-date in their presentations of some parallel algorithms.
The slides to be posted during this semester will contain a number of
more recently invented algorithms as well.
Lectures from Spr 2015 will be posted here.
Lecture 1, Jan 20, Introduction,
in ppt
and pdf
Lecture 2, Jan 22, Single Processor Machines: Memory Hierarchies and Processor Features
in ppt
and pdf
Lecture 3: Jan 27, Complete Lecture 2 (updated Jan 27, 5:36am),
begin Parallel Machines and Programming Models
in ppt
and pdf
Lecture 4: Jan 29, Complete Lecture 3 (updated Jan 29, 8:40am),
begin Sources of Parallelism and Locality in Simulation (Part 1)
in ppt
and pdf
Lecture 5: Feb 3, Complete Lecture 4 (updated Feb 3, 5:50am),
then Sources of Parallelism and Locality in Simulation (Part 2)
in ppt
and pdf
Lecture 6: Feb 5, Complete Lecture 5, then
Shared Memory Programming with Threads and OpenMP,
in ppt
and pdf, and
Tricks with Trees,
in ppt
and pdf.
Lecture 7: Feb 10, Complete Lecture 6, then
Distributed Memory Machines and Programming,
in ppt
and pdf.
Lecture 8: Feb 12, Complete Lecture 7, then
Partitioned Global Address Space Programming with Unified Parallel C (UPC) and
UPC++,
in pptx
and pdf.
by Kathy Yelick.
Lecture 9: Feb 17, Debugging and Optimization Tools, by
Richard Gerber,
in pptx
and pdf; and
Performance Debugging Techniques for HPC Applications, by
David Skinner,
in pptx
and pdf.
Lecture 10, Feb 19, Cloud Computing and Big Data Processing, by
Shivaram Venkataraman,
in pdf.
Lecture 11: Feb 24, An Introduction to CUDA/OpenCL and
Graphics Processors (GPUs), by
Forrest Iandola,
in pptx
and pdf.
Lecture 12: Feb 26, Dense Linear Algebra (Part 1),
in ppt
and pdf.
Lecture 13: Mar 3, Complete Lecture 12, then
Dense Linear Algebra (Part 2): Communication Avoiding Algorithms,
by Laura Grigori,
in ppt
and pdf.
Lecture 14: Mar 5, Complete Lecture 13, then
Graph Partitioning,
by Laura Grigori,
in ppt
and pdf.
Lecture 15: Mar 10, Complete Lecture 14, then
Sparse Linear Solvers,
by Laura Grigori,
in pdf.
Lecture 16: Mar 12, Complete Lecture 15,
updated (Mar 12),
then Sparse Iterative Solvers,
in pdf,
by Laura Grigori.
Lecture 17: Mar 17, Homework #3 Presentation,
in pptx and
pdf,
then Class Project Suggestions,
in pptx and
pdf
(updated May 3, 6:45am),
then Structured Grids,
in ppt and
pdf.
Lecture 18: Mar 19, Parallel Graph Algorithms,
in pptx and
pdf,
by Aydin Buluc
Lecture 19: Mar 31, Architecting Parallel Software with Patterns,
in pptx and
pdf,
by Kurt Keutzer
Lecture 20: Apr 2, Frameworks in Complex Multiphysics HPC Applications,
in pptx and
pdf,
by John Shalf
Lecture 21: Apr 7, Hierarchical Methods for the N-body problem,
in pptx
and pdf.
Lecture 22: Apr 8, complete Hierarchical Methods for the N-body problem,
(updated Apr 9, 7:15am)
Lecture 23: Apr 14, Fast Fourier Transform,
in ppt
and pdf.
Lecture 24: Apr 16, Big Bang, Big Data, Big Iron: High Performance Computing
and the Cosmic Microwave Background, by
Julian Borrill,
in pptx
and pdf
Lecture 25: Apr 21, Dynamic Load Balancing,
in ppt
and pdf.
Lecture 26: Apr 23, Modeling and Predicting Climate Change, by
Michael Wehner,
in ppt
and pdf.
Video of NASA Projections of Temperature and Precipitation in the 21st Century
Video of Preliminary CAM5 hi-resolution simulations
Video of fvCAM5.1 Simulated Atmospheric River
Lecture 27, Apr 28, Accelerated Materials Design through High-throughput
First-Principles Calculations and Data Mining, by
Kristin Persson,
in pptx
and pdf
Lecture 28, Big Data, Big Iron and The Future of HPC,
by Kathy Yelick.
in pptx
and pdf
Sharks and Fish
"Sharks and Fish" are a collection of simplified simulation programs
that illustrate a number of common parallel programming techniques
in various programming languages (some current ones, and some
old ones no longer in use).
Basic problem description, and (partial) code from 1999 class,
written in Matlab, CMMD, CMF, Split-C, Sun Threads, and pSather,
available
here.