CS6023 - GPU Programming

Course Data :


To learn parallel programming with graphics processing units (GPUs).
Outcomes: Students would learn concepts in parallel programming, implementation of programs on GPUs, debugging and profiling parallel programs.

Course Contents:

Topic (number of lectures + number of tutorials)
  • Introduction (2 + 1): History, graphics processors, graphics processing units, GPGPUs. Clock speeds, CPU / GPU comparisons, heterogeneity. Accelerators, parallel programming, CUDA / OpenCL / OpenACC,
  • Hello World Computation (3 + 1) Kernels, launch parameters, thread hierarchy, warps / wavefronts, thread blocks / workgroups, streaming multiprocessors, 1D / 2D / 3D thread mapping, device properties, simple programs
  • Memory (8 + 2) : Memory hierarchy, DRAM / global, local / shared, private / local, textures, constant memory. Pointers, parameter passing, arrays and dynamic memory, multi-dimensional arrays. Memory allocation, memory copying across devices. Programs with matrices, performance evaluation with different memories
  • Synchronization (6 + 2): Memory consistency. Barriers (local versus global), atomics, memory fence. Prefix sum, reduction. Programs for concurrent data structures such as worklists, linked-lists. Synchronization across CPU and GPU
  • Functions (3 + 1): Device functions, host functions, kernels, functors. Using libraries (such as Thrust), developing libraries.
  • Support (1 + 2): Debugging GPU programs. Profiling, profile tools, performance aspects
  • Streams (3 + 1): Asynchronous processing, tasks, task-dependence. Overlapped data transfers, default stream, synchronization with streams. Events, event-based-synchronization
    - overlapping data transfer and kernel execution, pitfalls.
  • Case studies (3 + 2) : Image processing. Graph algorithms. Simulations. Deep learning.
  • Advanced topics (8 + 2) : Dynamic parallelism. Unified virtual memory. Multi-GPU processing. Peer access. Heterogeneous processing
Course evaluation would involve programming assignments.

Text Books:

Programming Massively Parallel Processors: A Hands-on Approach; David Kirk, Wen-mei Hwu; Morgan Kaufman; 2010 (ISBN: 978-0123814722)

Reference Books:

CUDA Programming: A Developer's Guide to Parallel Computing with GPUs; Shane Cook; Morgan Kaufman; 2012 (ISBN: 978-0124159334)



Credits Type Date of Introduction
3-1-0-0-8-12-12 Elective Jan 2017

Previous Instances of the Course

© 2016 - All Rights Reserved - Dept of CSE, IIT Madras
Website Credits