ML Performance Reading Group Session 1: GPU Architecture, CUDA, NCCL

ML Performance Reading Group Session 2: Flash Attention

MultiGPU + NCCL from the authors

Stanford CS149 I Parallel Computing I 2023 I Kayvon Fatahalian and Kunle Olukotun

Делаем графические процессоры по-настоящему быстрыми: глубокий анализ эффективности тренировок

What is CUDA? - Computerphile

Understanding the Transformer Architecture

Data Center Networking for Administrators

Demystifying NCCL An In depth Analysis of GPU Communication Protocols and Algorithms - Zhiyi Hu

Getting Started with CUDA and Parallel Programming | NVIDIA GTC 2025 Session

Lecture 44: NVIDIA Profiling

Distributed ML Talk @ UC Berkeley

Lecture 67: NCCL and NVSHMEM

GTC 2022 - How CUDA Programming Works - Stephen Jones, CUDA Architect, NVIDIA

How do Graphics Cards Work? Exploring GPU Architecture

The Chaotic State of GPU Programming

Trends in Deep Learning Hardware: Bill Dally (NVIDIA)

Hard и soft skills, без которых не попасть в ML

NCCL: High-Speed Inter-GPU Communication for Large-Scale Training - Sylvain Jeaugey, NVIDIA

Tutorial: GPU Communication Libraries for Accelerating HPC and AI Applications

Learn RDMA Programming: NVIDIA’s Guide to High-Performance Networking