Applications of Parallel Computers

A Collaborative Online Course

Between 2013 to 2018, the XSEDE project has sponsored collaborative courses in parallel computing taught by the faculty at the University of California, Berkeley.  Applications of Parallel Computers has been offered as an online, blended learning course.  Collaborating institutions create their own, local course number so their students can receive university credit.  The lectures recorded by the lead instructors at University of California, Berkeley are used by all participants, often in a “flipped” classroom mode.  Students take quizzes focused on the lectures, complete a series of programming assignments, and complete a final project developed with their local instructors.  Local instructors use class time to discuss the course materials and work with their students on programming assignments and the final project.

For the past two years, Spring 2017 and 2018, the course was offered using this same model with the additional idea of assessing whether this model of shared, collaborative courses has the potential for expanding the availability of specialized courses in computational science.  This site summarizes that experience.

 

NSF logoThe Extreme Science and Engineering Discovery Environment (XSEDE) is supported by National Science Foundation grant number ACI-1053575. The APC workshop for 2017 and 2018 was funded under National Science Foundation grant number ACI 1650912. For additional information regarding this workshop, please contact Steven Gordon, sgordon@osc.edu.

Fields of Science: 

The Collaborative Model

Collaborators and Their Responsibilities

The collaborative course model involves the cooperation of a lead instructional institution, a project coordinator, and multiple participating universities.  The lead institution for the APC course is the University of California at Berkeley.  Under the auspices of the XSEDE project, they prepared the instructional materials based on their local course CS267.  The Ohio Supercomputer Center served as the project coordinator, facilitating the participation of the collaborating universities.  Each participating university, in turn, took responsibility for their own students with the support of a shared teaching assistant at Berkeley and the OSC staff.

 


Benefits of the Collaborative Model

Structuring the course in this way provided several benefits to the participating institutions.  The nature of their experience varied based on the status of HPC and parallel computing courses within their curriculum.  Based on conversations with the participating faculty these are the variety of benefits that they derived from their participation:

 

  • One institution is in the process of starting a minor program in computational science.  The course helped them to accelerate their implementation of the program on their campus.  They also indicated that they would be willing to participate in a wider ranging collaborative course program offering multiple courses.
  • Several institutions indicated that the collaborative model allowed them to offer parallel computing and HPC experience for their students for the first time in several years.
  • Several institutions offered a course of this nature for the first time, allowing faculty to learn about the breadth of parallel computing techniques and providing a first experience with HPC for their students.
  • Several institutions were able to offer the course to their students in small numbers that would otherwise not meet institutional enrollment requirements for course offerings.
  • All participants were able to use the XSEDE infrastructure which allowed instruction to include a variety of different parallel computing techiques associated with different combinations of modern HPC hardware including multi- and many-core processing.

Lead Instructors

The lead instructors at Berkeley provided all of the instructional materials used in the course.  This included videos of all lectures, quizzes relating to each of the lectures, and several programming assignments.  The recorded videos allowed each of the participating institutions to work through the course on their own academic schedule.  The lectures can either be watched independently by the students or jointly in the classroom.  Class time can then be used to discuss the lecture material and/or augment it with related discussions.

The quizzes are provided online as a way to gauge whether the remote students are keeping up with the class and to assess their comprehension of the lecture materials.  The quiz grades can be used as part of the final grading system at each participating institution.

The computer exercises focus on several different strategies for optimizing parallel computing code, using a range of programming options and algorithms.  An autograder was created for each exercise.  The autograders run the student's codes and provide a score based on the best possible optimization of each program.  Students can use that score to gauge the efficiency of their own code and instructors can use it as one way of gauging the mastery of the programming topics as part of the grading system.   The teaching assistant at Berkeley and the project coordinator track course and coding questions by the students and faculty through discussion forums on the course management system.  Instructors are also given access to the optimized code so that they can better advise their own students about programming strategies.

The lecture and assignment outline for the most recent offering of the course can be found here.  An earlier version of the lectures and quizzes are available through the XSEDE training website.  The first written assignment is to describe an existing parallel application (Homework 0).  All programming assignments are completed on XSEDE resources based on a classroom allocation that serves all course participants. The first computer exercise is an optimization of a matrix multiplication on a single processor.  The second assignment is to optimize a particle simulation.  Part 1 is done using multiple processors and part 2 using GPU's.  The third assignment uses the UPC language to optimize a graph algorithm to solve a de Novo genome assembly problem.  Students also complete an independent individual or group final project under the direction of their local instructors.  Examples of past projects are provided by Berkeley.

Course Outline and Schedule

 

Topic

 

Start Date

Completion Date

Introduction

 

1/16/2018

1/16/2018

Single Processor Machines: Memory Hierarchies and Processor Features

 

1/18/2018

1/23/2018

Homework 0 – Describe a Parallel Application

 

1/18/2018

1/29/2018

Parallel Machines and Programming Models

 

1/23/2018

1/25/2018

Sources of Parallelism and Locality in Simulation - Part 1

 

1/25/2018

1/30/2018

Sources of Parallelism and Locality in Simulation - Part 2

 

1/30/2018

2/1/2018

Shared Memory Programming: Threads and OpenMP, and Tricks with Trees

 

2/1/2018

2/6/2018

Programming Homework 1 - Optimize Matrix Multiplication

 

1/26/2018

2/9/2018

Distributed Memory Machines and Programming

 

2/6/2018

2/8/2018

Partitioned Global Address Space Programming with Unified Parallel C (UPC) and UPC++, by Kathy Yelick

 

2/8/2018

2/13/2018

ICloud Computing and Big Data Processing, by Shivaram Venkataraman

 

2/13/2018

2/13/2018

NERSC, Cori, Knights Landing and Other matters by Jack Deslippe

 

2/15/2018

2/15/2018

Programming Homework 2 (Part 1) Parallelizing a Particle Simulation

 

2/9/2018

3/2/2018

An Introduction to CUDA/OpenCL and Graphics Processors (GPUs), by Forrest Iandola

 

2/20/2018

2/20/2018

Dense Linear Algebra (Part 1)

 

2/22/2018

2/22/2018

Dense Linear Algebra (Part 2): Comm Avoiding Algorithms

 

2/27/2018

2/27/2018

Graph Partitioning

 

3/1/2018

3/62018

Programming Homework 2 (Part 2) Parallelizing a Particle Simulation (GPU)

 

3/1/2018

3/9/2018

Automatic Performance Tuning and Sparse Matrix Vector Multiplication

 

3/6/2018

3/8/2018

Automatic Performance Tuning and Sparse Matrix Vector Multiplication (continued)

 

3/8/2018

3/8/2018

Programming Homework 3 - Parallelize Graph Algorithms

 

3/13/2018

4/6/2018

Structured Grids

 

3/13/2018

3/13/2018

Parallel Graph Algorithms, by Aydin Buluc

 

3/15/2018

3/152018

Final Project Proposal

 

 

4/6/2018

Architecting Parallel Software with Patterns, by Kurt Keutzer

 

3/20/2018

3/20/2018

Fast Fourier Transform

 

3/22/2018

3/22/2018

Modeling and Predicting Climate Change, by Michael Wehner

 

4/3/2018

4/3/2018

Scientific Software Ecosystems by Mike Heroux

 

4/5/2018

4/5/2018

Dynamic Load Balancing

 

4/10/2018

4/10/2018

Accelerated Materials Design through High-throughput First Principles Calculations by Kristin Persson

 

4/12/2018

4/12/2018

Hierarchical Methods for the N-Body Problem

 

4/17/2018

4/19/2018

Communication Lower Bounds and Optimal Algorithms

 

4/19/2018

4/19/2018

Big Bang, Big Data, Big Iron, HPC and the Cosmic Microwave Background Data Analysis by Julian Borrill

 

4/24/2018

4/24/2018

 Big Bang and Exascale: A Tale of Two Ecosystems by Kathy Yelick

 

 4/26/2018

 4/26/2018

Final Project Poster

 

 

Local

Final Project Report

 

 

Local

Past CS267 Projects: Past Projects

 

Project Coordinator

The project coordination role for the collaborative course was played by Steven Gordon at the Ohio Supercomputer Center (OSC).  The first task for this role was the recruitment of collaborating universities.  This was done through a variety of email lists, XSEDE newsletters, and personal emails sent to previous participants.

The course materials for the workshop version of the course is maintained on the Moodle course management system at OSC - moodle.xsede.org.  OSC staff are responsible for maintaining the server while the project coordinator maintains the course information.  That includes adding discussion forums for all of the participating institutions, obtaining student enrollment information from each institution, sending enrolled students instructions for connecting to the course management system, and helping to answer course questions.  The course management system has links to all of the video lectures, online quizzes, and homework assignment instructions and datasets.  In addition, it provides a mechanism for students to upload their completed assignments.  Those assignments are then available to the individual faculty members to grade.

In addition, the project coordinator arranged for an introductory online meeting with the lead instructors and participating faculty, regular online meetings with the faculty, and discussions with the faculty about the course and the collaborative model.  Arrangements were also made with the independent evaluator for the XSEDE project to evaluate the experience of students and faculty participating in the course.  The results of those efforts are summarized in the evaluation section of this site.

Collaborating Faculty

Participating institutions were solicited via newsletter posts by XSEDE and through several mailing lists of faculty interested in computational science and parallel computing.  Participating institutions have a lead faculty member responsible for local course administration.  Those faculty:

 

  • Create a local course at their institution for which students can register for credit
  • Participate in conference calls with the course instructors and coordinators
  • Provide guidance to their students via discussion of lecture materials and assistance with programming assignments
  • Create a local grading scale that includes the online quizzes, programming assignments, and final project
  • Grade the programming assignments assisted by the autograders provided by Berkeley
  • Supervise  and grade student final projects

The collaborating faculty also participated in regular online meetings to discuss the course materials and the pros and cons of course organization. 

The course was offered two times under the workshop grant in the spring semester of 2017 and 2018.  In 2017 twelve institutions participated in the workshop.  That included several minority serving institutions, one foreign institution (Universidad de Medellin), and one high school (Marmion Academy).  The participating institutions and number of students is shown in the next section.  There were 92 students who completed the course in 2017.

In 2018, thirteen institutions participated with 211 students completing the course.  The course again included several minority serving institutions and smaller colleges that might not have been able to offer this course to the few students that were interested and and sufficiently prepared.


Participating Institutions and Students

Institutions and Students Participating in the Workshops
2017 Participating Institutions No. of Students 2018 Participating Institutions No. of Students
Case Western Reserve University 14 Austin Peay University 2
Florida Southern University 4 Florida Polytechnic University 39
George Mason University 7 Franklin and Marshall University 17
Lamar University 4 George Mason University 13
Louisiana Tech 1 Indiana University Southeast 9
Marmion Academy 1 North Carolina Central University 3
Northeast Oklahoma 14 Penn State University 39
Rutgers University 9 Rutgers University 10
University of Puerto Rico Mayaguez 4 San Francisco State University 35
Universidad de Medellin 9 South Dakota State University 1
University of Houston Clear Lake 21 State University of NY Oneonta 3
Wright State University 4 University of Houston Clear Lake 25
    University of Kentucky 15
Totals 92   211

Project Evaluation

A total of 23 different institutions participated in the course over the two offerings.  Eight institutions participated in both years.  A total of 328 students participated in the courses over the two offerings.  Of those, 301 or 92% successfully completed the course.  This is one measure of the success of the program.

 

An evaluation of the course by the participating faculty included their perspectives on both the course content and the collaborative model.  Those evaluations were made through a combination of surveys, open discussions with the faculty during live discussion sessions, and selected interviews with other faculty.

 

Overall, the faculty felt the content of the course is excellent and offers a comprehensive view of parallel computing.  They indicated that the course offering greatly increased the interest in parallel computing among their students.  However, a number of suggestions were made that could improve the impact of the course.  Those included suggestions that would help students taking the course and suggestions that would help faculty that had never taught such a course be better prepared to advise their students.

 

  1. Several faculty suggested that additional lectures and support materials are needed that focus on the practical aspects of running and optimizing codes on the XSEDE computers.  Materials on debugging code, how to approach optimization problems, and backup readings or tutorials on each major subtopic would aid students who are just being introduced to parallel computing concepts.
  2. A number of the undergraduate students taking the course were not well prepared with respect to the math prerequisites.  The course assumed some understanding of calculus and linear algebra.  One suggestion was to create a pre-course assessment for undergraduates to ascertain whether they have the appropriate background.
  3. It was suggested that there be a pre-course orientation class for faculty who have never offered such a course.  Over the course of six to 10 hours over a few weeks, faculty would optionally be guided through the course materials and especially the programming assignments.  That would better prepare them to help their own students.
  4. Several faculty indicated that they would have appreciated a clearer set of partial and complete solutions to the programming problems that they could study and use to advise their students.  They also noted that issues arose between the two offerings as there was a change in the available XSEDE hardware and the solutions to the problems were not updated in a timely manner to reflect those changes.

 

The other discussions with faculty centered around the efficacy of the collaborative course model.  In particular, those discussions involved their overall assessment of this approach, their willingness to participate in an on-going consortium, and the organization and terms of such a consortial arrangement.

 

  1. All of the faculty who participated in the discussions about the collaborative course model felt it was a valuable approach to offer specialized courses.  Several indicated that they would not have been able to offer a parallel computing course on their own.  Several also indicated that experience in the course would contribute to their ability to offer a course in the future.
  2. All but two of the participating faculty agreed that a collaborative course consortium would definitely or probably benefit their institution.
  3. There was a range of opinions on the nature of the agreements that would comprise an ongoing consortium.  All agreed that some exchange of services in the form of course preparation for the consortium would be an acceptable arrangement.  However, the faculty were split on the nature of that exchange.  Most felt that each campus should take some responsibility for course preparation every 3-2 years.  However, they were split on whether a single institution should take responsibility for an entire course versus having each institution be responsible for a portion of the course preparation.  The strongest consensus was for each institution to be responsible for only a portion of each course preparation.
  4. Parallel computing was among several courses that the faculty thought should be part of a collaborative consortium.  The others mentioned in order of preference were introduction to high performance computing, data analytics, modeling and simulation, techniques for many core computing, and bioinformatics.

 

The results indicate that further efforts should be made to continue the efforts for the current course and to pursue options for creating an ongoing collaborative consortium modeled on this effort.