Lectures on Parallel Computing

This textbook is intended for an undergraduate course on the fundamentals of parallel computing or as background and reference for graduate courses on high-performance computing, shared-memory multiprocessor programming, or parallel algorithms. The author introduces theoretical concepts and tools for expressing, analyzing, and assessing parallel algorithms, and covers in detail the two most widely used concrete frameworks OpenMP and MPI, plus the threading interface pthreads for writing parallel programs for either shared or distributed memory parallel computers with an emphasis on general concepts and principles. Code examples are given in a C-like style, and the extensive exercises can be used for self-study and as inspiration for small implementation projects in OpenMP and MPI. The book is appropriate for undergraduate and graduate students, and for professionals engaged in all aspects of parallel computing.