Programming Massively Parallel Processors: A Hands-on Approach, Fifth Edition shows both students and professionals alike the basic concepts of parallel programming and GPU architecture. Concise, intuitive, and practical, it is based on years of road-testing in the authors' own parallel computing courses. Various techniques for constructing and optimizing parallel programs are explored in detail, while case studies demonstrate the development process, which begins with computational thinking and ends with effective and efficient parallel programs. This new edition has been updated with an expanded repertoire of optimizations, new patterns and applications, ad more coverage of important CUDA features.
1. Chapter 1: Introduction - p. 1 2. Part 1: Fundamental concepts - p. P1 3. Chapter 2: Heterogeneous data-parallel computing - p. 21 4. Chapter 3: Multidimensional grids and data - p. 45 5. Chapter 4: Compute architecture and scheduling - p. 67 6. Chapter 5: Memory architecture and data locality - p. 93 7. Chapter 6: Performance considerations - p. 123 8. Part 2: Parallel patterns - p. P2 9. Chapter 7: Convolution - p. 159 10. Chapter 8: Stencil computation - p. 183 11. Chapter 9: Histogram - p. 201 12. Chapter 10: Reduction - p. 221 13. Chapter 11: Scan - p. 251 14. Chapter 12: Filter - p. 289 15. Chapter 13: Merge - p. 303 16. Chapter 14: Sorting - p. 329 17. Chapter 15: Advanced optimizations for matrix multiplication - p. 349 18. Part 3: Advanced patterns and applications - p. P3 19. Chapter 16: Dynamic programming and wavefront parallelism - p. 373 20. Chapter 17: Sparse matrix computation - p. 401 21. Chapter 18: Graph traversal - p. 425 22. Chapter 19: Convolutional neural networks - p. 453 23. Chapter 20: Large language models - p. 477 24. Chapter 21: Electrostatic potential map - p. 513 25. Chapter 22: Algorithm selection, problem decomposition, and problem formulation - p. 529 26. Chapter 23: Multi-GPU programming - p. 541 27. Chapter 24: Conclusion and future outlook - p. 577 28. Part 4: Appendices - p. P4 29. Appendix A: Numerical considerations - p. 583 30. Appendix B: Deep learning basics - p. 601 31. Appendix C: CUDA memories, address spaces, and pointers - p. 623 32. Index - p. 631
Programming Massively Parallel Processors: A Hands-on Approach, Fifth Edition shows both students and professionals alike the basic concepts of parallel programming and GPU architecture. Concise, intuitive, and practical, it is based on years of road-testing in the authors' own parallel computing courses. Various techniques for constructing and optimizing parallel programs are explored in detail, while case studies demonstrate the development process, which begins with computational thinking and ends with effective and efficient parallel programs. This new edition has been updated with an expanded repertoire of optimizations, new patterns and applications, ad more coverage of important CUDA features.
1. Chapter 1: Introduction - p. 1 2. Part 1: Fundamental concepts - p. P1 3. Chapter 2: Heterogeneous data-parallel computing - p. 21 4. Chapter 3: Multidimensional grids and data - p. 45 5. Chapter 4: Compute architecture and scheduling - p. 67 6. Chapter 5: Memory architecture and data locality - p. 93 7. Chapter 6: Performance considerations - p. 123 8. Part 2: Parallel patterns - p. P2 9. Chapter 7: Convolution - p. 159 10. Chapter 8: Stencil computation - p. 183 11. Chapter 9: Histogram - p. 201 12. Chapter 10: Reduction - p. 221 13. Chapter 11: Scan - p. 251 14. Chapter 12: Filter - p. 289 15. Chapter 13: Merge - p. 303 16. Chapter 14: Sorting - p. 329 17. Chapter 15: Advanced optimizations for matrix multiplication - p. 349 18. Part 3: Advanced patterns and applications - p. P3 19. Chapter 16: Dynamic programming and wavefront parallelism - p. 373 20. Chapter 17: Sparse matrix computation - p. 401 21. Chapter 18: Graph traversal - p. 425 22. Chapter 19: Convolutional neural networks - p. 453 23. Chapter 20: Large language models - p. 477 24. Chapter 21: Electrostatic potential map - p. 513 25. Chapter 22: Algorithm selection, problem decomposition, and problem formulation - p. 529 26. Chapter 23: Multi-GPU programming - p. 541 27. Chapter 24: Conclusion and future outlook - p. 577 28. Part 4: Appendices - p. P4 29. Appendix A: Numerical considerations - p. 583 30. Appendix B: Deep learning basics - p. 601 31. Appendix C: CUDA memories, address spaces, and pointers - p. 623 32. Index - p. 631
