Inside the Message Passing Interface: Creating Fast Communication Libraries

ISBN-13: 9781501515545 / Angielski / Miękka / 2018 / 384 str.

Supalov, Alexander

Spis treści

1. Introduction – Learn what expects you inside the book

1. What this book is about
2. Who should read this book
3. Notation and conventions
4. How to read this book

2. Overview

1. Parallel computer

1. Intraprocessor parallelism
2. Interprocessor parallelism
3. Exercises

2. MPI standard

1. MPI history
2. Related standards
3. Exercises

3. MPI subsetting

1. Motivation
2. Typical examples
3. Implementation practice
4. Exercises

3. Shared memory – Learn how to create a simple MPI subset capable of basic blocking point-to-point and collective operations over shared memory

1. Subset definition

1. General assumptions
2. Blocking point-to-point communication
3. Blocking collective operations
4. Exercises

2. Communication mechanisms

1. Basic communication
2. Intraprocess performance
3. Interprocess performance
4. Exercises

3. Startup and termination

1. Process creation

1. Two processes
2. More processes

2. Connection establishment
3. Process termination
4. Exercises

4. Blocking point-to-point communication

1. Limited message length

1. Blocking protocol

2. Unlimited message length

1. Double buffering
2. Eager protocol
3. Rendezvous protocol

3. Exercises

5. Blocking collective operations

1. Naive algorithms
2. Barrier
3. Broadcast
4. Reduce and Allreduce
5. Exercises

4. Sockets – Learn how to create an MPI subset capable of all point-to-point and blocking collective operations over Ethernet and other IP capable networks

1. Subset definition

1. General assumptions
2. Blocking point-to-point communication
3. Nonblocking point-to-point operations
4. Blocking collective operations
5. Exercises

2. Communication mechanisms

1. Basic communication
2. Intranode performance
3. Internode performance
4. Exercises

3. Synchronous progress engine

1. Communication establishment
2. Data transfer
3. Exercises

4. Startup and termination

1. Process creation

1. Startup command
2. Process daemon
3. Out-of-band communication
4. Host name resolution

2. Connection establishment

1. At startup (eager)
2. On request (lazy)

3. Process termination
4. Exercises

5. Blocking point-to-point communication

1. Source and tag matching
2. Unexpected messages
3. Exercises

6. Nonblocking point-to-point communication

1. Request management
2. Exercises

7. Blocking collective operations

1. Communication context
2. Basic algorithms

1. Tree based algorithms
2. Circular algorithms
3. Hypercube algorithms

3. Exercises

5. OFA libfabrics – Learn how to create an MPI subset capable of all point-to-point and collective operations over InfiniBand and upcoming future networks

1. Subset definition

1. General assumptions
2. Point-to-point operations
3. Collective operations
4. Exercises

2. Communication mechanisms

1. Basic communication
2. Intranode performance
3. Internode performance
4. Exercises

3. Startup and termination

1. Process creation
2. Credential exchange
3. Connection establishment
4. Process termination
5. Exercises

4. Point-to-point communication

1. Blocking communication
2. Nonblocking communication
3. Exercises

5. Collective operations

1. Advanced algorithms
2. Blocking operations
3. Nonblocking operations
4. Exercises

6. Advanced features – Learn how to add advanced MPI features including but not limited to heterogeneity, one-sided communication, file I/O, and language bindings

1. Communication modes

1. Standard
2. Buffered
3. Synchronous

2. Heterogeneity

1. Basic datatypes
2. Simple datatypes
3. Derived datatypes
4. Exercises

3. Groups, communicators, topologies

1. Group management
2. Communicator management
3. Process topologies
4. Exercises

4. One-sided communication

1. Mapped implementation
2. Native implementation
3. Exercises

5. File I/O

1. Standard I/O
2. MPI file I/O
3. Exercises

6. Language bindings

1. Fortran
2. C++
3. Java
4. Python
5. Exercises

7. Optimization – Learn how to optimize MPI internally by using advanced implementation techniques and available special hardware

1. Direct data transfer

1. Direct memory access
2. Remote direct memory access
3. Exercises

2. Threads

1. Thread support level
2. Threads as MPI processes
3. Shared memory extensions
4. Exercises

3. Multiple fabrics

1. Synchronous progress engine
2. Asynchronous progress engine
3. Hybrid progress engine
4. Exercises

4. Dedicated hardware

1. Synchronization
2. Special memory
3. Auxiliary networks
4. Exercises

8. Look ahead – Learn to recognize MPI advantages and drawbacks to better assess its future

1. MPI axioms

1. Reliable data transfer
2. Ordered message delivery
3. Dense process rank sequence
4. Exercises

2. MPI-4 en route

1. Fault tolerance
2. Exercises

3. Beyond MPI

1. Exascale challenge
2. Exercises

9. References – Learn about books that may further extend your knowledge

Appendices

1. MPI Families – Learn about major MPI implementation families, their genesis, architecture and relative performance

1. MPICH

1. Genesis
2. Architecture
3. Details

1. MPICH
2. MVAPICH
3. Intel MPI
4. …

4. Exercises

2. OpenMPI

1. Genesis
2. Architecture
3. Details
4. Exercises

3. Comparison

1. Market
2. Features
3. Performance
4. Exercises
1. Alternative interfaces – Learn about other popular interfaces that are used to implement MPI

1. DAPL

1. …
2. Exercises

2. SHMEM

1. …
2. Exercises

3. GasNET

1. …
2. Exercises

4. Portals

1. …
2. Exercises

2. Solutions to all exercises – Learn how to answer all those questions

O autorze

Dr. Alexander Supalov, Supalov HPC, Germany