Concurrent and Parallel Programming

  1. Parallel Programming Concepts
    1. Parallel Architectures
      1. Flynn's Taxonomy
        1. SISD Architecture
          1. SIMD Architecture
            1. MISD Architecture
              1. MIMD Architecture
              2. Shared Memory Architectures
                1. Uniform Memory Access
                  1. Non-Uniform Memory Access
                    1. Cache Coherence Protocols
                    2. Distributed Memory Architectures
                      1. Networked Nodes
                        1. Message Passing in Distributed Systems
                        2. Hybrid Architectures
                          1. Combining Shared and Distributed Memory
                            1. Clustered Multi-Core Systems
                          2. Performance Metrics and Laws
                            1. Speedup
                              1. Speedup Definition
                                1. Speedup Calculation
                                2. Efficiency
                                  1. Efficiency Definition
                                    1. Efficiency Calculation
                                    2. Scalability
                                      1. Strong Scaling
                                        1. Weak Scaling
                                        2. Amdahl's Law
                                          1. Theoretical Speedup Limit
                                            1. Identifying Serial Bottlenecks
                                              1. Implications for Parallelization
                                              2. Gustafson's Law
                                                1. Scaled Speedup
                                                  1. Implications for Large Problem Sizes
                                                2. Parallel Decomposition Strategies
                                                  1. Task Parallelism
                                                    1. Dividing Work by Function
                                                      1. Dividing Work by Task
                                                        1. Task Parallelism Use Cases
                                                          1. Task Parallelism Examples
                                                          2. Data Parallelism
                                                            1. Dividing Data Among Processing Units
                                                              1. Data Parallelism Use Cases
                                                                1. Data Parallelism Examples
                                                                2. Pipeline Parallelism
                                                                  1. Stages of Computation
                                                                    1. Throughput Considerations
                                                                      1. Latency Considerations

                                                                  Previous

                                                                  4. Concurrency Models and Patterns

                                                                  Go to top

                                                                  Next

                                                                  6. Parallel Programming Patterns and Algorithms