Native Sparse Attention: Hardware-Aligned and Natively Trainable Sparse Attention

Hardware-Efficient Attention for Fast Decoding

Rotary Positional Embeddings Explained | Transformer

Intro to Attention and Its Forms

This Simple Optimizer Is Revolutionizing How We Train AI [Muon]

Pushing the Limits of Sparse Attention in LLMs - Marcos Treviso | ASAP 49

FlashAttention - Tri Dao | Stanford MLSys #67

Обзор теории DeepSeek R1 | GRPO + RL + SFT

Нужно ли нам внимание? — Линейные рекуррентные нейронные сети и модели пространства состояний (SS...

Intro to Sparse Tensors and Spatially Sparse Neural Networks

How FlashAttention Accelerates Generative AI Revolution

THIS is why large language models can understand the world

NEW DeepSeek Sparse Attention Explained - DeepSeek V3.2-Exp

Round and Round We Go! What makes Rotary Positional Encodings useful?

Sparse Block Attention

Variants of Multi-head attention: Multi-query (MQA) and Grouped-query attention (GQA)

Lecture 36: CUTLASS and Flash Attention 3

ЛУЧШАЯ БЕСПЛАТНАЯ НЕЙРОСЕТЬ Google, которой нет аналогов

Sliding Window Attention (Longformer) Explained

LLM inference optimization: Architecture, KV cache and Flash attention

FlashAttention: Ускоренное обучение LLM