CUDA programming Masterclass with C++

Course Curriculum

Chapter 1: Introduction to CUDA programming and CUDA programming model

Lecture 1: Very very important

Lecture 2: Introduction to parallel programming

Lecture 3: Parallel computing and Super computing

Lecture 4: How to install CUDA toolkit and first look at CUDA program

Lecture 5: Basic elements of CUDA program

Lecture 6: Organization of threads in a CUDA program – threadIdx

Lecture 7: Organization of thread in a CUDA program – blockIdx,blockDim,gridDim

Lecture 8: Programming exercise 1

Lecture 9: Unique index calculation using threadIdx blockId and blockDim

Lecture 10: Unique index calculation for 2D grid 1

Lecture 11: Unique index calculation for 2D grid 2

Lecture 12: Memory transfer between host and device

Lecture 13: Programming exercise 2

Lecture 14: Sum array example with validity check

Lecture 15: Sum array example with error handling

Lecture 16: Sum array example with timing

Lecture 17: Device properties

Lecture 18: Summary

Chapter 2: CUDA Execution model

Lecture 1: Understand the device better

Lecture 2: All about warps

Lecture 3: Warp divergence

Lecture 4: Resource partitioning and latency hiding 1

Lecture 5: Resource partitioning and latency hiding 2

Lecture 6: Occupancy

Lecture 7: Profile driven optimization with nvprof

Lecture 8: Parallel reduction as synchronization example

Lecture 9: Parallel reduction as warp divergence example

Lecture 10: Parallel reduction with loop unrolling

Lecture 11: Parallel reduction as warp unrolling

Lecture 12: Reduction with complete unrolling

Lecture 13: Performance comparison of reduction kernels

Lecture 14: CUDA Dynamic parallelism

Lecture 15: Reduction with dynamic parallelism

Lecture 16: Summary

Chapter 3: CUDA memory model

Lecture 1: CUDA memory model

Lecture 2: Different memory types in CUDA

Lecture 3: Memory management and pinned memory

Lecture 4: Zero copy memory

Lecture 5: Unified memory

Lecture 6: Global memory access patterns

Lecture 7: Global memory writes

Lecture 8: AOS vs SOA

Lecture 9: Matrix transpose

Lecture 10: Matrix transpose with unrolling

Lecture 11: Matrix transpose with diagonal coordinate system

Lecture 12: Summary

Chapter 4: CUDA Shared memory and constant memory

Lecture 1: Introduction to CUDA shared memory

Lecture 2: Shared memory access modes and memory banks

Lecture 3: Row major and Column major access to shared memory

Lecture 4: Static and Dynamic shared memory

Lecture 5: Shared memory padding

Lecture 6: Parallel reduction with shared memory

Lecture 7: Synchronization in CUDA

Lecture 8: Matrix transpose with shared memory

Lecture 9: CUDA constant memory

Lecture 10: Matrix transpose with Shared memory padding

Lecture 11: CUDA warp shuffle instructions

Lecture 12: Parallel reduction with warp shuffle instructions

Lecture 13: Summary

Chapter 5: CUDA Streams

Lecture 1: Introduction to CUDA streams and events

Lecture 2: How to use CUDA asynchronous functions

Lecture 3: How to use CUDA streams

Lecture 4: Overlapping memory transfer and kernel execution

Lecture 5: Stream synchronization and blocking behavious of NULL stream

Lecture 6: Explicit and implicit synchronization

Lecture 7: CUDA events and timing with CUDA events

Lecture 8: Creating inter stream dependencies with events

Chapter 6: Performance Tuning with CUDA instruction level primitives

Lecture 1: Introduction to different types of instructions in CUDA

Lecture 2: Floating point operations

Lecture 3: Standard and Instrict functions

Lecture 4: Atomic functions

Chapter 7: Parallel Patterns and Applications

Lecture 1: Scan algorithm introduction

Lecture 2: Simple parallel scan

Lecture 3: Work efficient parallel exclusive scan

Lecture 4: Work efficient parallel inclusive scan

Lecture 5: Parallel scan for large data sets

Lecture 6: Parallel Compact algorithm

Chapter 8: Bonus: Introduction to Image processing with CUDA

Lecture 1: Introduction part 1

Lecture 2: Introduction part 2

Lecture 3: Digital image processing

Lecture 4: Digital image fundametals : Human perception

Lecture 5: Digital image fundamentals : Image formation

Lecture 6: OpenCV installation