Generative AI with Large Language Models: Hands-On Training feat. Hugging Face and PyTorch Lightning

TOPIC SUMMARY
Module 1: Introduction to Large Language Models (LLMs)
A Brief History of Natural Language Processing (NLP)
Transformers
Subword Tokenization
Autoregressive vs Autoencoding Models
ELMo, BERT and T5
The GPT (Generative Pre-trained Transformer) Family
LLM Application Areas

Module 2: The Breadth of LLM Capabilities
LLM Playgrounds
Staggering GPT-Family progress
Key Updates with GPT-4
Calling OpenAI APIs, including GPT-4

Module 3: Training and Deploying LLMs
Hardware Options (e.g., CPU, GPU, TPU, IPU, AWS chips)
The Hugging Face Transformers Library
Best Practices for Efficient LLM Training
Parameter-efficient fine-tuning (PEFT) with low-rank adaptation (LoRA)
Open-Source Pre-Trained LLMs
LLM Training with PyTorch Lightning
Multi-GPU Training
LLM Deployment Considerations
Monitoring LLMs in Production

Module 4: Getting Commercial Value from LLMs
Supporting ML with LLMs
Tasks that can be Automated
Tasks that can be Augmented
Best Practices for Successful A.I. Teams and Projects
What's Next for A.I.

CHAPTERS
0:00 Intro
5:56 Module 1: Introduction to Large Language Models (LLMs)
18:17 The Models that Shaped the Field
34:40 The GPT Family: A Closer Look
38:09 Module 2: The Breadth of LLM Capabilities
49:12 GPT-4 and OpenAI APIs
57:48 Module 3: Training and Deploying LLMs
1:09:52 Advanced Training Techniques and Open-Source Options
1:45:56 Training with PyTorch Lightning and Multi-GPU Training
2:09:33 Deployment and Monitoring of LLMs
2:10:30 Module 4: Getting Commercial Value from LLMs

CODE: https://github.com/jonkrohn/NLP-with-...
SLIDES: https://www.jonkrohn.com/talks

ABSTRACT
At an unprecedented pace, Large Language Models like GPT-4 are transforming the world in general and the field of data science in particular. This two-hour training introduces deep learning transformer architectures including LLMs. Critically, it also demonstrates the breadth of capabilities of state-of-the-art LLMs like GPT-4 can deliver, including for dramatically revolutionizing the development of machine learning models and commercially successful data-driven products, accelerating the creative capacities of data scientists and pushing them in the direction of being data product managers. Brought to life via hands-on code demos that leverage the Hugging Face and PyTorch Lightning Python libraries, this training covers the full lifecycle of LLM development, from training to production deployment.

ABOUT THE PRESENTER
Jon Krohn is Co-Founder and Chief Data Scientist at the machine learning company Nebula. He authored the book Deep Learning Illustrated, an instant #1 bestseller that was translated into seven languages. He is also the host of SuperDataScience, the data science industry’s most listened-to podcast. Jon is renowned for his compelling lectures, which he offers at leading universities and conferences, as well as via his award-winning YouTube channel. He holds a PhD from Oxford and has been publishing on machine learning in prominent academic journals since 2010.

STAY IN TOUCH
YouTube:    / jonkrohnlearns  
GitHub: https://github.com/jonkrohn
LinkedIn:   / jonkrohn  
Twitter:   / jonkrohnlearns  

Podcast: "Super Data Science" is available on all major podcasting platforms and a video version is on YouTube (   / @superdatasciencewithjonkrohn  )