MAMBA from Scratch: Neural Nets Better and Faster than Transformers
Автор: Algorithmic Simplicity
Загружено: 2024-04-30
Просмотров: 285314
Mamba is a new neural network architecture that came out this year, and it performs better than transformers at language modelling! This is probably the most exciting development in AI since 2017. In this video I explain how to derive Mamba from the perspective of linear RNNs. And don't worry, there's no state space model theory needed!
Mamba paper: https://openreview.net/forum?id=AL1fq...
Linear RNN paper: https://openreview.net/forum?id=M3Yd3...
#mamba
#deeplearning
#largelanguagemodels
00:00 Intro
01:33 Recurrent Neural Networks
05:24 Linear Recurrent Neural Networks
06:57 Parallelizing Linear RNNs
15:33 Vanishing and Exploding Gradients
19:08 Stable initialization
21:53 State Space Models
24:33 Mamba
25:26 The High Performance Memory Trick
27:35 The Mamba Drama
Доступные форматы для скачивания:
Скачать видео mp4
-
Информация по загрузке:
![Цепи Маркова — математика предсказаний [Veritasium]](https://ricktube.ru/thumbnail/QI7oUwNrQ34/mqdefault.jpg)
![The Real Reason Huge AI Models Actually Work [Prof. Andrew Wilson]](https://ricktube.ru/thumbnail/M-jTeBCEGHc/mqdefault.jpg)
![This Simple Optimizer Is Revolutionizing How We Train AI [Muon]](https://ricktube.ru/thumbnail/bO5nvE289ec/mqdefault.jpg)
![The Misconception that Almost Stopped AI [How Models Learn Part 1]](https://ricktube.ru/thumbnail/NrO20Jb-hy0/mqdefault.jpg)