Parallel and Distributed Computing

Parallel and Distributed Computing Winter 2015/16

Belegnummer: 36.3518 Modulbeschreibung

Parallel and Distributed Computing

A master's level lecture course with lab ("Praktikum"), in English.

Topics covered include:

  • Concepts and architectures for parallel and distributed computing
  • Shared Memory Programming Paradigm with Java, Posix threads and OpenMP
  • Message-Passing Programming with MPI
  • 21st century trends such as (for example)

    • Grid & Cloud Computing (particularly Apache Hadoop & Co.),
    • GPGPUs (General Purpose Graphic Processing Units)
    • 21st century programming languages such as Golang and Scala/Akka -- as time permits (!).


The Lecture starts on Wednesday, 11 October.

The lab starts on Thursday 12 October (x-Raster or 19 October for the y-Raster).

Attendance at the lab is mandatory! If you do not appear at the first meeting, your seat in the lab can be given to another student. 

Corollary: This also means that if you did not get a seat in the lab through the OBS, come to the meeting anyway! You may be able to take over a seat from someone who didn''t attend the first meeting!

Please read the explanation of "Exercise 0" below before coming to the first lab meeting!

Lecture Notes and Exams: Please Log In

The Lecture Notes and Information concerning the Final Exam (including copies of old exams) are only available for members of the h_da.  Students and employees of the h_da (including professors), please log in (see the bottom left corner of this web page), and then return to this web page (and hit "reload", if necessary).

Lab ("Praktikum")

Exercise 0 ("Zero"):

WARNING: This material is left over from last year and is subject to change!

Material: Exercise0.tar.gz. Warning: Changes are possible in this material in the days before the first meeting of the Lab (please check back here from time to time).

This file contains three example programs (and one Makefile).  All three programs calculate the value of pi (3.1415926...) using the same algorithm, but very different parallel technologies, namely:

  1. MPI - Message Passing Interface
  2. C++11 Threads - the Thread library from the new C++11 standard.
  3. OpenMP.

(MPI is a library that must be installed; C++11 and OpenMP are supported by modern C++ Compilers).

Preparation for Exercise 0 - Prepare Your Team and Your Development Environments

  • Consider who you would like to be in a team with. The lab exercises will be performed by 2-person teams (3-person teams will be allowed only if we have an odd number of participants).
  • Set up your Development Environment

    • Every group needs at least one development computer for software development between appointments (whenever the laboratory is not available).
    • It is highly recommended (but not strictly speaking required) that development be done under Linux.  Any Linux Distribution (e.g. Ubuntu, SUSE, etc.) should do.  (Linux will be used in the lab).
    • Once you have Linux installed, you should have the C++ Compilers installed, and thus you should have C++11 and OpenMP.
    • You will also need to install an implementation of MPI. We recommend MPICH (Version 3!). See   (Binary packets are available for almost every Linux Distribution; Compiling from source is also highly recommended).

  • Test your Development Environment

    • Download and unpack the three example programs (see above).
    • Enter "make tests" (after unpacking the file). 
    • If all goes well, all three programs will be compiled, linked and run.
    • If not, fix them as best you can.
    • Whether the programs run "out of the box" or not, familiarize yourself with the code.

In the lab, we will:

  1. Determine who is present.  

    • Attendance is mandatory! 
    • Students who do not attend, and who have not sent the instructor an email explaining their absence (in advance!) will lose their seat in the lab!
    • Students who have not received a seat in the lab from the OBS should attend as well - extra seats will be distributed where possible.

  2. Build teams. 
  3. Introduce Moodle

    • After this Exercise, all subsequent Exercises (i.e. Exercise n for all where 0 < <= 4) are described in Moodle s. Course Nr. 1370.
    • Moodle will be used for submitting Lab Reports (for all exercises, including this one) for grading.

  4. Introduce the Lab Workstations

    • We will make sure everyone can log in to the computers in the laboratory.
    • We will make sure every group can build and run all three example programs (see above).

  5. Benchmark the Three Sample Programs

    • Using the three example programs supplied (Exercise0.tar.gz, see above), you should measure the performance of the computers in the laboratory (and if you wish, your development computers).
    • To get interesting measurements, the programs will have to be modified! They are currently written to finish very quickly.  The programs must be modified to run longer, so that the measurments are interesting!
    • You will then collect and analyze empirical results.
    • Finally, each team will produce and upload a short lab report (document) detailing your analysis, meeting minimal scientific standards (e.g. with a title, the names of all authors, an introduction, description of your methods, presentation of the empirical results, a conclusion, perhaps a list of literature, etc.). The lab report is a mandatory part of every exercise, including Exercise 0.