How RWKV-7 "Goose" and It's Linear Inference Work with Author Eugene Cheah
Автор: Oxen
Загружено: 2025-04-15
Просмотров: 1013
Paper 📜 https://arxiv.org/abs/2503.14456
Links + Notes 📝 https://www.oxen.ai/blog/how-rwkv-7-g...
Join Arxiv Dives 🤿 https://oxen.ai/community
Discord 🗿 / discord
Use Oxen AI 🐂 https://oxen.ai/
Oxen AI makes versioning your datasets as easy as versioning your code! Even is millions of unstructured images, the tool quickly handles any type of data so you can build cutting-edge AI.
--
Chapters
0:00 Why is RWKV-7 Goose interesting
2:53 How to quickly run RWKV-7 Goose
4:04 What is RWKV-7
10:20 RNN’s forget things
12:33 First paper: Reinventing RNNs for the Transformer Era
24:22 Paper author Eugene Cheah joins the dive
36:43 The intuition behind each model layer
47:57 Parallelization during training
53:01 How well did RWKV-7 do on benchmarks?
56:50 Live evals on RWKV-7 and fine-tuning tips
1:00:38 Why they made the World Tokenizer
Доступные форматы для скачивания:
Скачать видео mp4
-
Информация по загрузке:
![Как внимание стало настолько эффективным [GQA/MLA/DSA]](https://imager.clipsaver.ru/Y-o545eYjXM/max.jpg)
