Coding a ChatGPT Like Transformer From Scratch in PyTorch

In this StatQuest we walk through the code required to code your own ChatGPT like Transformer in PyTorch and we do it one step at a time, with every little detail clearly explained.

NOTE: This StatQuest assumes that you are already familiar with the concepts behind...
Decoder-Only Transformers:    • Decoder-Only Transformers, ChatGPTs specif...  
The Essential Matrix Algebra for Neural Networks:    • Essential Matrix Algebra for Neural Networ...  
The Matrix Math Behind Transformers:    • The matrix math behind transformer neural ...  

You can get the code here: https://github.com/StatQuest/decoder_...

The full Neural Networks playlist, from the basics to AI, is here:    • The Essential Main Ideas of Neural Networks  

Learn more about GiveInternet.org: https://giveinternet.org/StatQuest NOTE: Donations up to $30 will be matched by an Angel Investor - so a $30 donation would give $60 to the organization. DOUBLE BAM!!!

For a complete index of all the StatQuest videos, check out:
https://statquest.org/video-index/

If you'd like to support StatQuest, please consider...

Patreon:   / statquest  
...or...
YouTube Membership:    / @statquest  

...buying one of my books, a study guide, a t-shirt or hoodie, or a song from the StatQuest store...
https://statquest.org/statquest-store/

...or just donating to StatQuest!
https://www.paypal.me/statquest

Lastly, if you want to keep up with me as I research and create new StatQuests, follow me on twitter:
  / joshuastarmer  

0:00 Awesome song and introduction
1:12 Loading the modules
2:04 Creating the training dataset
6:17 Coding Position Encoding
14:09 Coding Attention
21:04 Coding a Decoder-Only Transformer
26:39 Running the model (untrained)
29:18 Training and using the model

#StatQuest #PyTorch #chatgpt