Parallel Algorithms and Distributed Systems

Course Information

Lecturer: William Spataro

Office Hours: by appointment/upon request

The syllabus of the course

Notice board

Please check the lecturer's didactic calendar (https://sv.mat.unical.it/~spataro/teaching.html) for time modifications, etc

Contact Lecturer (spataro[at]unical.it) for slides' password.

Course material

Slides

• Lecture 0 - Introduction (pdf)

• Lecture 1 - Introduction (pdf)

• Lecture 2 - Architectures (pdf)

• Lecture 3 - Programming Models - Shared - OpenMP - 1st part (pdf)

• Lecture 4 - Modelli, Partizionamento, Design (pdf)

• Lecture 5 - Programming Models - Shared - OpenMP - 2nd part (pdf)

• Lecture 6 - Programming Models - Shared - OpenMP - 3rd part (pdf)

• Lecture 7 - Programming Models (Examples) (pdf)

• Lecture 8 - MPI (1st part) (pdf)

• Lecture 9 - MPI (2nd part) (pdf)

• Lecture 10 - MPI (3rd part)(pdf)

• Lecture 11 - MPI (4th part)(pdf)

• Lecture 12 - Networks, etc (pdf)

• Lecture 13 - Performance, etc (1st part) (pdf)

• Lecture 14 - MPI (5th part) (pdf)

• Lecture 15 - Performance, etc (2nd part) (pdf)

• Lezione 16 - MPI (6th part) (pdf)

• Lecture 17 - Sorting, etc (pdf)

• Lecture 18 - Cuda HPCC Seminar(pdf)

• MPI - Code examples, etc (pdf)

Tutorials / Handouts

• A (Simple) Parallel Computing overview (zip)

• OpenMP overview (zip)

• Excellent course on MPI by PACS Training Group

Links

• Simple Parallel Computing Tutorial, from Lawrence Livermore National Laboratory

• Designing and Building Parallel Programs, by Ian Foster

• A (nice) Quick Tutorial on OpenMP, by Joel Yliluoma

• Tutorials and exercises on OpenMP (from the official site openmp.org)

• Simple but exhaustive Tutorial on MPI (from Lawrence Livermore National Laboratory)

• Posix Threads (pdf) - just to see :)

• Linux processes (pdf)

MPI

• Mini MPI HOWTO - Version 1.2.5

• Mini MPI HOWTO - Versione 2.1.2

• Libreria MPI (Version MPICH-1.2.5)

• Libreria MPI (Version MPICH-2.1.2)

• MPICH - 1.2.5 Manual

• MPICH - 2.1.2 Manual

• Mini OpenGL-for-MPI HOWTO

• MPI Mini-Manual for Dummies (for the exam!)

Exercises

• OpenMP Example Codes

• Easter bonus!!

• A simple parallel structure ... (Cellular Automata)

Exam information

The final exam consists in a written test regarding all studied topics, and development (and discussion) of a MPI/OpenMP/CUDA project.

Exam Modality / Projects

• Modality, etc

• Cellular Automata Project

• Parallel Sorting

• Graph Algorithms

• Linear Equations System Algorithms

Acknowledgements / Sources

• Almerico Murli, Università di Napoli
• Salvatore Orlando, Università di Venezia
• Rolf Rabenseifner, University of Stuttgart
• Jack Dongarra, University of Tennesse

• Slides from "Parallel Programming Techniques & Applications Using Networked Workstations & Parallel Computers", 2nd ed., by B. Wilkinson & M. Allen, 2004 Pearson Education Inc.

• Blaise Barney, Lawrence Livermore National Laboratory
• Libro "Advanced Linux Programming"
• Ananth Grama, Purdue University
• George Karypis, University of Minnesota
• Fedele Stabile, Università della Calabria
• The openmp.org project

Additional information